NDLI: Model selection for primal SVM

Content Provider	Springer Nature Link
Author	Moore, Gregory Bergeron, Charles Bennett, Kristin P.
Copyright Year	2011
Abstract	This paper introduces two types of nonsmooth optimization methods for selecting model hyperparameters in primal SVM models based on cross-validation. Unlike common grid search approaches for model selection, these approaches are scalable both in the number of hyperparameters and number of data points. Taking inspiration from linear-time primal SVM algorithms, scalability in model selection is achieved by directly working with the primal variables without introducing any dual variables. The proposed implicit primal gradient descent (ImpGrad) method can utilize existing SVM solvers. Unlike prior methods for gradient descent in hyperparameters space, all work is done in the primal space so no inversion of the kernel matrix is required. The proposed explicit penalized bilevel programming (PBP) approach optimizes both the hyperparameters and parameters simultaneously. It solves the original cross-validation problem by solving a series of least squares regression problems with simple constraints in both the hyperparameter and parameter space. Computational results on least squares support vector regression problems with multiple hyperparameters establish that both the implicit and explicit methods perform quite well in terms of generalization and computational time. These methods are directly applicable to other learning tasks with differentiable loss functions and regularization functions. Both the implicit and explicit algorithms investigated represent powerful new approaches to solving large bilevel programs involving nonsmooth loss functions.
Starting Page	175
Ending Page	208
Page Count	34
File Format	PDF
ISSN	08856125
Journal	Machine Learning
Volume Number	85
Issue Number	1-2
e-ISSN	15730565
Language	English
Publisher	Springer US
Publisher Date	2011-04-22
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Model selection Cross-validation Nonconvex optimization Bilevel programming Support vector machines Language Translation and Linguistics Computing Methodologies Artificial Intelligence (incl. Robotics) Simulation and Modeling Control, Robotics, Mechatronics
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Guest editorial: model selection and optimization in machine learning

Multi-parametric solution-path algorithm for instance-weighted support vector machines

Feature selection via sensitivity analysis of SVM probabilistic outputs

An efficient primal dual prox method for non-smooth optimization

Selective switching mechanism in virtual machines via support vector machines and transfer learning

Graph kernels based on tree patterns for molecules

Ensemble clustering using semidefinite programming with applications

Consensus hashing

High-probability minimax probability machines

Model selection for primal SVM

Similar Documents

Guest editorial: model selection and optimization in machine learning

Multi-parametric solution-path algorithm for instance-weighted support vector machines

Feature selection via sensitivity analysis of SVM probabilistic outputs

An efficient primal dual prox method for non-smooth optimization

Selective switching mechanism in virtual machines via support vector machines and transfer learning

Graph kernels based on tree patterns for molecules

Ensemble clustering using semidefinite programming with applications

Consensus hashing

High-probability minimax probability machines

Model selection for primal SVM