NDLI: Convergence and Stability of Iteratively Re-weighted Least Squares Algorithms

Content Provider	IEEE Xplore Digital Library
Author	Ba, D. Babadi, B. Purdon, P.L. Brown, E.N.
Copyright Year	1991
Abstract	In this paper, we study the theoretical properties of iteratively re-weighted least squares (IRLS) algorithms and their utility in sparse signal recovery in the presence of noise. We demonstrate a one-to-one correspondence between the IRLS algorithms and a class of Expectation-Maximization (EM) algorithms for constrained maximum likelihood estimation under a Gaussian scale mixture (GSM) distribution. The EM formalism, as well as the connection to GSMs, allow us to establish that the IRLS algorithms minimize smooth versions of the lν `norms', for . We leverage EM theory to show that the limit points of the sequence of IRLS iterates are stationary points of the smooth lν “norm” minimization problem on the constraint set. We employ techniques from Compressive Sampling (CS) theory to show that the IRLS algorithm is stable, if the limit point of the iterates coincides with the global minimizer. We further characterize the convergence rate of the IRLS algorithm, which implies global linear convergence for ν = 1 and local super-linear convergence for . We demonstrate our results via simulation experiments. The simplicity of IRLS, along with the theoretical guarantees provided in this contribution, make a compelling case for its adoption as a standard tool for sparse signal recovery.
Sponsorship	IEEE Signal Processing Society
Starting Page	183
Ending Page	195
Page Count	13
File Size	3329938
File Format	PDF
ISSN	1053587X
Volume Number	62
Issue Number	1
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-01-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Signal processing algorithms Convergence Stability analysis Noise Standards Convex functions Random variables mathematical programming Compressive sampling constrained maximum likelihood estimation Gaussian scale mixtures expectation-maximization algorithms
Content Type	Text
Resource Type	Article
Subject	Signal Processing Electrical and Electronic Engineering

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Gaussian Mixtures Based IRLS for Sparse Recovery With Quadratic Convergence

Maximum-Likelihood Estimation of Delta-Domain Model Parameters From Noisy Output Signals

Convergence and Stability of a Class of Iteratively Re-weighted Least Squares Algorithms for Sparse Signal Recovery in the Presence of Noise

Asymptotic maximum likelihood estimator performance for chaotic signals in noise

Maximum likelihood estimation in linear models with a Gaussian model matrix

Linear Regression With Gaussian Model Uncertainty: Algorithms and Bounds

Estimating signal parameters using the nonlinear instantaneous least squares approach

Constrained maximum likelihood solution of linear equations

Maximum-likelihood array processing in non-Gaussian noise with Gaussian mixtures

Convergence and Stability of Iteratively Re-weighted Least Squares Algorithms

Similar Documents

Gaussian Mixtures Based IRLS for Sparse Recovery With Quadratic Convergence

Maximum-Likelihood Estimation of Delta-Domain Model Parameters From Noisy Output Signals

Convergence and Stability of a Class of Iteratively Re-weighted Least Squares Algorithms for Sparse Signal Recovery in the Presence of Noise

Asymptotic maximum likelihood estimator performance for chaotic signals in noise

Maximum likelihood estimation in linear models with a Gaussian model matrix

Linear Regression With Gaussian Model Uncertainty: Algorithms and Bounds

Estimating signal parameters using the nonlinear instantaneous least squares approach

Constrained maximum likelihood solution of linear equations

Maximum-likelihood array processing in non-Gaussian noise with Gaussian mixtures

Convergence and Stability of Iteratively Re-weighted Least Squares Algorithms