NDLI: Gradient methods for minimizing composite functions

Content Provider	Springer Nature Link
Author	Nesterov, Yu.
Copyright Year	2012
Abstract	In this paper we analyze several new methods for solving optimization problems with the objective function formed as a sum of two terms: one is smooth and given by a black-box oracle, and another is a simple general convex function with known structure. Despite the absence of good properties of the sum, such problems, both in convex and nonconvex cases, can be solved with efficiency typical for the first part of the objective. For convex problems of the above structure, we consider primal and dual variants of the gradient method (with convergence rate $$O\left({1 \over k}\right)$$ ), and an accelerated multistep version with convergence rate $$O\left({1 \over k^2}\right)$$ , where $$k$$ is the iteration counter. For nonconvex problems with this structure, we prove convergence to a point from which there is no descent direction. In contrast, we show that for general nonsmooth, nonconvex problems, even resolving the question of whether a descent direction exists from a point is NP-hard. For all methods, we suggest some efficient “line search” procedures and show that the additional computational work necessary for estimating the unknown problem class parameters can only multiply the complexity of each iteration by a small constant factor. We present also the results of preliminary computational experiments, which confirm the superiority of the accelerated scheme.
Ending Page	161
Page Count	37
Starting Page	125
File Format	PDF
ISSN	00255610
e-ISSN	14364646
Journal	Mathematical Programming
Issue Number	1
Volume Number	140
Language	English
Publisher	Springer-Verlag
Publisher Date	2012-12-21
Publisher Place	Berlin, Heidelberg
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Theoretical, Mathematical and Computational Physics Complexity theory Black-box model Convex programming Local optimization Optimal methods Analysis of algorithms and problem complexity Mathematical Methods in Physics Structural optimization Calculus of Variations and Optimal Control; Optimization Mathematics of Computing Numerical Analysis $$l_1$$ -Regularization Convex Optimization Nonsmooth optimization Combinatorics Minimax problems
Content Type	Text
Resource Type	Article
Subject	Mathematics Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Universal gradient methods for convex optimization problems

Subgradient methods for huge-scale optimization problems

Primal-dual subgradient methods for convex problems

Barrier subgradient method

Smoothing Technique and its Applications in Semidefinite Optimization

Excessive Gap Technique in Nonsmooth Convex Minimization

Block coordinate proximal gradient methods with variable Bregman functions for nonsmooth separable optimization

Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions

Gradient sliding for composite optimization

Gradient methods for minimizing composite functions

Similar Documents

Universal gradient methods for convex optimization problems

Subgradient methods for huge-scale optimization problems

Primal-dual subgradient methods for convex problems

Barrier subgradient method

Smoothing Technique and its Applications in Semidefinite Optimization

Excessive Gap Technique in Nonsmooth Convex Minimization

Block coordinate proximal gradient methods with variable Bregman functions for nonsmooth separable optimization

Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions

Gradient sliding for composite optimization

Gradient methods for minimizing composite functions