NDLI: Regularizers for structured sparsity

Content Provider	Springer Nature Link
Author	Micchelli, Charles A. Morales, Jean M. Pontil, Massimilia
Copyright Year	2011
Abstract	We study the problem of learning a sparse linear regression vector under additional conditions on the structure of its sparsity pattern. This problem is relevant in machine learning, statistics and signal processing. It is well known that a linear regression can benefit from knowledge that the underlying regression vector is sparse. The combinatorial problem of selecting the nonzero components of this vector can be “relaxed” by regularizing the squared error with a convex penalty function like the ℓ1 norm. However, in many applications, additional conditions on the structure of the regression vector and its sparsity pattern are available. Incorporating this information into the learning method may lead to a significant decrease of the estimation error. In this paper, we present a family of convex penalty functions, which encode prior knowledge on the structure of the vector formed by the absolute values of the regression coefficients. This family subsumes the ℓ1 norm and is flexible enough to include different models of sparsity patterns, which are of practical and theoretical importance. We establish the basic properties of these penalty functions and discuss some examples where they can be computed explicitly. Moreover, we present a convergent optimization algorithm for solving regularized least squares with these penalty functions. Numerical simulations highlight the benefit of structured sparsity and the advantage offered by our approach over the Lasso method and other related methods.
Starting Page	455
Ending Page	489
Page Count	35
File Format	PDF
ISSN	10197168
Journal	Advances in Computational Mathematics
Volume Number	38
Issue Number	3
e-ISSN	15729044
Language	English
Publisher	Springer US
Publisher Date	2011-11-24
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Convex optimization Feature selection LASSO Linear regression Regularization Sparse estimation Numeric Computing Calculus of Variations and Optimal Control; Optimization Mathematics Algebra Theory of Computation
Content Type	Text
Resource Type	Article
Subject	Applied Mathematics Computational Mathematics

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

On discrete ℓ1-regularization

Preface: numerical and applied linear algebra

Regularization Networks and Support Vector Machines

Beyond sparsity: Recovering structured representations by ${\ell}^1$ minimization and greedy algorithms

A Posteriori Error Estimates for Distributed Convex Optimal Control Problems

Higher-Order Convex Approximations of Young Measures in Optimal Control

High dimensional polynomial interpolation on sparse grids

Structured Sparsity of Convolutional Neural Networks via Nonconvex Sparse Group Regularization

Learning the coordinate gradients

Regularizers for structured sparsity

Similar Documents

On discrete ℓ1-regularization

Preface: numerical and applied linear algebra

Regularization Networks and Support Vector Machines

Beyond sparsity: Recovering structured representations by ${\ell}^1$ minimization and greedy algorithms

A Posteriori Error Estimates for Distributed Convex Optimal Control Problems

Higher-Order Convex Approximations of Young Measures in Optimal Control

High dimensional polynomial interpolation on sparse grids

Structured Sparsity of Convolutional Neural Networks via Nonconvex Sparse Group Regularization

Learning the coordinate gradients

Regularizers for structured sparsity