NDLI: Novel Maximum-Margin Training Algorithms for Supervised Neural Networks

Content Provider	IEEE Xplore Digital Library
Author	Ludwig, O. Nunes, U.
Copyright Year	1990
Abstract	This paper proposes three novel training methods, two of them based on the backpropagation approach and a third one based on information theory for multilayer perceptron (MLP) binary classifiers. Both backpropagation methods are based on the maximal-margin (MM) principle. The first one, based on the gradient descent with adaptive learning rate algorithm (GDX) and named maximum-margin GDX (MMGDX), directly increases the margin of the MLP output-layer hyperplane. The proposed method jointly optimizes both MLP layers in a single process, backpropagating the gradient of an MM-based objective function, through the output and hidden layers, in order to create a hidden-layer space that enables a higher margin for the output-layer hyperplane, avoiding the testing of many arbitrary kernels, as occurs in case of support vector machine (SVM) training. The proposed MM-based objective function aims to stretch out the margin to its limit. An objective function based on Lp-norm is also proposed in order to take into account the idea of support vectors, however, overcoming the complexity involved in solving a constrained optimization problem, usually in SVM training. In fact, all the training methods proposed in this paper have time and space complexities O(N) while usual SVM training methods have time complexity O(N ³) and space complexity O(N ²) , where N is the training-data-set size. The second approach, named minimization of interclass interference (MICI), has an objective function inspired on the Fisher discriminant analysis. Such algorithm aims to create an MLP hidden output where the patterns have a desirable statistical distribution. In both training methods, the maximum area under ROC curve (AUC) is applied as stop criterion. The third approach offers a robust training framework able to take the best of each proposed training method. The main idea is to compose a neural model by using neurons extracted from three other neural networks, each one previously trained by MICI, MMGDX, and Levenberg-Marquard (LM), respectively. The resulting neural network was named assembled neural network (ASNN). Benchmark data sets of real-world problems have been used in experiments that enable a comparison with other state-of-the-art classifiers. The results provide evidence of the effectiveness of our methods regarding accuracy, AUC, and balanced error rate.
Sponsorship	IEEE Computational Intelligence Society
Page Count	13
File Size	974922
Starting Page	972
Ending Page	984
File Format	PDF
ISSN	10459227
Volume Number	21
Issue Number	6
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2010-06-01
Publisher Place	U.S.A.
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Neural networks Support vector machines Backpropagation algorithms Information theory Multilayer perceptrons Optimization methods Testing Kernel Constraint optimization Interference supervised learning maximal-margin (MM) principle multilayer perceptron (MLP) pattern recognition
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence Computer Networks and Communications Computer Science Applications Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A multi-objective optimization approach for training artificial neural networks

Fast training of multilayer perceptrons

An Optimization Methodology for Neural Network Weights and Architectures

Training multilayer perceptron classifiers based on a modified support vector method

The Margitron: A Generalized Perceptron With Margin

SoftDoubleMaxMinOver: Perceptron-Like Training of Support Vector Machines

Two-Phase Construction of Multilayer Perceptrons Using Information Theory

Divide-and-conquer learning and modular perceptron networks

Mongolian character recognition using multilayer perceptron (MLP)

Novel Maximum-Margin Training Algorithms for Supervised Neural Networks

Similar Documents

A multi-objective optimization approach for training artificial neural networks

Fast training of multilayer perceptrons

An Optimization Methodology for Neural Network Weights and Architectures

Training multilayer perceptron classifiers based on a modified support vector method

The Margitron: A Generalized Perceptron With Margin

SoftDoubleMaxMinOver: Perceptron-Like Training of Support Vector Machines

Two-Phase Construction of Multilayer Perceptrons Using Information Theory

Divide-and-conquer learning and modular perceptron networks

Mongolian character recognition using multilayer perceptron (MLP)

Novel Maximum-Margin Training Algorithms for Supervised Neural Networks