NDLI: The square root Kalman filter training of recurrent neural networks

Content Provider	IEEE Xplore Digital Library
Author	Sun, P. Marko, K.
Copyright Year	1998
Description	Author affiliation: Ford Res. Lab., Ford Motor Co., Dearborn, MI, USA (Sun, P.)
Abstract	The conventional Kalman filter suffers from the problem that the covariance matrix may not remain positive definite. In using the filter to train neural networks, the consequence of this problem is both a loss of efficiency during training and an eventual blow up. This problem has long been recognized in the application of the filter in signal processing and several forms of a square root Kalman filter have been derived to overcome it. However, the existing square root Kalman filter algorithms cannot incorporate the learning rate used in neural network training. In many situations, the incorporation of the learning rate into the training algorithms is crucial to obtaining excellent results in training time lagged recurrent neural networks (TLRNN). In this paper, a new square root Kalman filter equation is derived to train TLRNN which allows a user to incorporate the learning rate into the neural network training. With this new square root Kalman filter algorithm, we are able to train a neural network to convergence on a large and complex problem related to misfire diagnostics over a very large number of cycles, to produce extremely high classification accuracy on the diagnostic task. This training proceeded without any training problems we often experienced when a standard Kalman filter training algorithm was used for identical initializations and structures of the network. Furthermore, the neural network trained by the new method outperforms the one trained with the conventional Kalman filter algorithm by almost a factor of two.
Starting Page	1645
Ending Page	1651
File Size	616720
Page Count	7
File Format	PDF
ISBN	0780347781
ISSN	1062922X
DOI	10.1109/ICSMC.1998.728125
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	1998-10-14
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Recurrent neural networks Signal processing algorithms Neural networks Equations Sun Laboratories Covariance matrix Convergence Estimation theory Decision theory
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in