NDLI: From Convex to Nonconvex: A Loss Function Analysis for Binary Classification

Content Provider	IEEE Xplore Digital Library
Author	Lei Zhao Mammadov, M. Yearwood, J.
Copyright Year	2010
Abstract	Problems of data classification can be studied in the framework of regularization theory as ill-posed problems. In this framework, loss functions play an important role in the application of regularization theory to classification. In this paper, we review some important convex loss functions, including hinge loss, square loss, modified square loss, exponential loss, logistic regression loss, as well as some non-convex loss functions, such as sigmoid loss, $\phi$-loss, ramp loss, normalized sigmoid loss, and the loss function of 2 layer neural network. Based on the analysis of these loss functions, we propose a new differentiable non-convex loss function, called smoothed 0-1 loss function, which is a natural approximation of the 0-1 loss function. To compare the performance of different loss functions, we propose two binary classification algorithms for binary classification, one for convex loss functions, the other for non-convex loss functions. A set of experiments are launched on several binary data sets from the UCI repository. The results show that the proposed smoothed 0-1 loss function is robust, especially for those noisy data sets with many outliers.
Starting Page	1281
Ending Page	1288
File Size	592570
Page Count	8
File Format	PDF
ISBN	9781424492442
DOI	10.1109/ICDMW.2010.57
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2010-12-13
Publisher Place	Australia
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Training Support vector machines loss function non-convex optimization regularization Fasteners Robustness Loss measurement classification Optimization Logistics
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Penalty and margin decomposition - an inspection of loss function regularization in SVM

To be or not to be convex? A study on regularization in hyperspectral image classification

Admissibility of fuzzy support vector machine through loss function

An optimization method for selecting parameters in support vector machines

Convex-Concave Hull for Classification with Support Vector Machine

Convex multi-task relationship learning using hinge loss

A Robust Proximal Support Vector Machines for Classification

Nearest Class Vector Classification for Large-Scale Learning Problems

Semi-supervised image classification with huberized Laplacian Support Vector Machines

From Convex to Nonconvex: A Loss Function Analysis for Binary Classification

Similar Documents

Penalty and margin decomposition - an inspection of loss function regularization in SVM

To be or not to be convex? A study on regularization in hyperspectral image classification

Admissibility of fuzzy support vector machine through loss function

An optimization method for selecting parameters in support vector machines

Convex-Concave Hull for Classification with Support Vector Machine

Convex multi-task relationship learning using hinge loss

A Robust Proximal Support Vector Machines for Classification

Nearest Class Vector Classification for Large-Scale Learning Problems

Semi-supervised image classification with huberized Laplacian Support Vector Machines

From Convex to Nonconvex: A Loss Function Analysis for Binary Classification