NDLI: Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using <formula formulatype=inline><tex Notation=TeX>$ell _{1}$</tex> </formula>-Constrained Quadratic Programming (Lasso)

Content Provider	IEEE Xplore Digital Library
Author	Wainwright, M.J.
Copyright Year	1963
Abstract	The problem of consistently estimating the sparsity pattern of a vector beta* isin R^p based on observations contaminated by noise arises in various contexts, including signal denoising, sparse approximation, compressed sensing, and model selection. We analyze the behavior of l₁-constrained quadratic programming (QP), also referred to as the Lasso, for recovering the sparsity pattern. Our main result is to establish precise conditions on the problem dimension p, the number k of nonzero elements in beta*, and the number of observations n that are necessary and sufficient for sparsity pattern recovery using the Lasso. We first analyze the case of observations made using deterministic design matrices and sub-Gaussian additive noise, and provide sufficient conditions for support recovery and l_infin-error bounds, as well as results showing the necessity of incoherence and bounds on the minimum value. We then turn to the case of random designs, in which each row of the design is drawn from a N (0, Sigma) ensemble. For a broad class of Gaussian ensembles satisfying mutual incoherence conditions, we compute explicit values of thresholds 0 < thetas_l(Sigma) les thetas_u(Sigma) < +infin with the following properties: for any delta > 0, if n > 2 (thetas_u + delta) klog (p- k), then the Lasso succeeds in recovering the sparsity pattern with probability converging to one for large problems, whereas for n < 2 (thetas_l - delta)klog (p - k), then the probability of successful recovery converges to zero. For the special case of the uniform Gaussian ensemble (Sigma = I_ptimesp), we show that thetas_l = thetas<_u = 1, so that the precise threshold n = 2 klog(p- k) is exactly determined.
Sponsorship	IEEE Information Theory Society
Starting Page	2183
Ending Page	2202
Page Count	20
File Size	762219
File Format	PDF
ISSN	00189448
Volume Number	55
Issue Number	5
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2009-05-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Quadratic programming Signal denoising Compressed sensing Statistics Graphical models Polynomials Context modeling Pattern analysis Additive noise Sufficient conditions subset selection convex relaxation high-dimensional inference $\ell _{1}$-constraints model selection phase transitions sparse approximation signal denoising
Content Type	Text
Resource Type	Article
Subject	Library and Information Sciences Information Systems Computer Science Applications

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting

Information-Theoretic Limits on Sparse Signal Recovery: Dense versus Sparse Measurement Matrices

Simultaneous Support Recovery in High Dimensions: Benefits and Perils of Block $ell _{1}/ell _{infty} $-Regularization

Necessary and Sufficient Conditions for Sparsity Pattern Recovery

Compressed and Privacy-Sensitive Sparse Regression

Fast and Accurate Algorithms for Re-Weighted $ell _{1}$-Norm Minimization

Thresholded Basis Pursuit: LP Algorithm for Order-Wise Optimal Support Recovery for Sparse and Approximately Sparse Signals From Noisy Random Measurements

Sharp thresholds for high-dimensional and noisy recovery of sparsity

Sparse Graphical Linear Dynamical Systems

Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $ell _{1}$ -Constrained Quadratic Programming (Lasso)

Similar Documents

Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting

Information-Theoretic Limits on Sparse Signal Recovery: Dense versus Sparse Measurement Matrices

Simultaneous Support Recovery in High Dimensions: Benefits and Perils of Block $ell _{1}/ell _{infty} $-Regularization

Necessary and Sufficient Conditions for Sparsity Pattern Recovery

Compressed and Privacy-Sensitive Sparse Regression

Fast and Accurate Algorithms for Re-Weighted $ell _{1}$-Norm Minimization

Thresholded Basis Pursuit: LP Algorithm for Order-Wise Optimal Support Recovery for Sparse and Approximately Sparse Signals From Noisy Random Measurements

Sharp thresholds for high-dimensional and noisy recovery of sparsity

Sparse Graphical Linear Dynamical Systems

Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $ell _{1}$ -Constrained Quadratic Programming (Lasso)