NDLI: The robust backpropagation training of MLP's: an optimal rapid restart method

Content Provider	IEEE Xplore Digital Library
Author	Yue-Xian Hou Cong-Jin Yu
Copyright Year	2003
Description	Author affiliation: Dept. of Comput. Sci. & Technol., Tianjin Univ., China (Yue-Xian Hou; Cong-Jin Yu)
Abstract	It is proved that MLP's with the training algorithm of backpropagation is a universal mapper, which can, in theory, approximate any continuous decision region arbitrarily well. Yet the convergence of backpropagation algorithms is still an open problem. It is well known that the time cost of backpropagation training often exhibits a remarkable variability, which seems to be of the generic character of heavy-tailed probability distribution. It has been demonstrated that, in most cases, rapid restart (RR) method can prominently suppress the heavy-tailed nature of training instances and improve efficiency of computation. However, it is usually time-consuming to verify whether a training instance is heavy-tailed. Moreover, if the heavy-tailed distribution is confirmed and the RR method is relevant, an optimal RR threshold should be chosen to facilitate the RR mechanism. In this paper, an approximate approach is proposed to quickly check whether a training instance is heavy-tailed or not by calculating the maximal Lyapunov exponent of a few generic running trace. Then a statistical estimator is deduced to determine the optimal RR threshold. The practically experimental results are consistent with the theoretical consideration perfectly.
Sponsorship	IEEE Systems Man and Cybernetics Tech. Committee on Cybernetics, Hebei Univ. Northwestern Polytechnical Univ
Starting Page	1370
Ending Page	1373
File Size	255843
Page Count	4
File Format	PDF
ISBN	0780381319
DOI	10.1109/ICMLC.2003.1259705
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2003-11-05
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Robustness Multiprotocol label switching Costs Backpropagation algorithms Search methods Fractals Computer science Probability distribution Convergence Heuristic algorithms
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

On optimal routing with multiple traffic matrices

Method for rapid recovery path computation on mesh IP network

Evaluation of MPLS P2MP distribution tree algorithms

Robust neural network training using partial gradient probing

Dividing the Traffic Matrix to Approach Optimal Traffic Engineering

Optimal Paths Design for a GMPLS Network using the Lagrangian Relaxation Method

Robust solutions for optimal routing

Rapid Backpropagation Learning Algorithms *

Bounding the Number of Light Paths for Robust LSP Routing

The robust backpropagation training of MLP's: an optimal rapid restart method

Similar Documents

On optimal routing with multiple traffic matrices

Method for rapid recovery path computation on mesh IP network

Evaluation of MPLS P2MP distribution tree algorithms

Robust neural network training using partial gradient probing

Dividing the Traffic Matrix to Approach Optimal Traffic Engineering

Optimal Paths Design for a GMPLS Network using the Lagrangian Relaxation Method

Robust solutions for optimal routing

Rapid Backpropagation Learning Algorithms *

Bounding the Number of Light Paths for Robust LSP Routing

The robust backpropagation training of MLP's: an optimal rapid restart method