NDLI: An improved method for reducing the forgetfulness in incremental learning

Content Provider	IEEE Xplore Digital Library
Author	Shinozawa, K. Shimohara, K.
Copyright Year	1999
Description	Author affiliation: NTT Commun. Sci. Labs., Kyoto, Japan (Shinozawa, K.)
Abstract	We propose an improved method for reducing the forgetfulness in incremental learning in a feedforward multilayer perceptron. In incremental learning, the network learns using new data items one by one. Naturally, the network is affected mainly by the most recent data, and past data is forgotten. Usually, to solve this problem, all data is stored and used for repeated learning, but this is not efficient use of computing time and memory. We believe the forgetfulness is an increase in the error function's value for past training data, and that it thus can be suppressed by minimizing the change in the value. The change in the error function's value is approximated using eigenvalues and eigenvectors of the coefficient of the second order term of an error function that is expanded with the Taylor expansion. By introducing a constraint for minimizing the change in the value based on this approximation to update the weight parameters, we believe the forgetfulness can be suppressed. Based on the above idea, we proposed a method that assigns the initial value of the momentum term an eigenvector with small eigenvalue. However, this method is a weak constraint for minimizing the change in the value. Thus, while it is an effective method for learning a sine function, it is not effective for learning a chaotic sequence calculated with a logistic map. In this paper, we modify the way in which the initial value of the momentum term is estimated, and propose a method that provides a stronger constraint for updating the weight parameters to minimize the change of the value in the error function. This method can effectively suppress the forgetfulness. This method was tested on a chaotic sequence calculated with a logistic map. The result shows greater suppression of the forgetfulness than with the original method.
Starting Page	1068
Ending Page	1073
File Size	484212
Page Count	6
File Format	PDF
ISBN	0780357310
ISSN	1062922X
DOI	10.1109/ICSMC.1999.812558
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	1999-10-12
Publisher Place	Japan
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Multilayer perceptrons Training data Function approximation Costs Eigenvalues and eigenfunctions Logistics Intelligent networks Laboratories Feedforward systems Taylor series
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in