NDLI: An improved RBM based on Bayesian Regularization

Content Provider	IEEE Xplore Digital Library
Author	Guangyuan Pan Junfei Qiao Wei Chai Dimopoulos, N.
Copyright Year	2014
Description	Author affiliation: Dept. of Electr. & Comput. Eng., Univ. of Victoria, Victoria, BC, Canada (Dimopoulos, N.) \|\| Inst. of Intell. Syst., Beijing Univ. of Technol., Beijing, China (Guangyuan Pan; Junfei Qiao; Wei Chai)
Abstract	Restricted Boltzmann Machine is a fundamental method in deep learning networks. Training and generalization is an ill-defined problem in that many different networks may achieve the training goal; however each will respond differently to an unknown input. Traditional approaches include stopping the training early and/or restricting the size of the network These approaches ameliorate the problem of over-fitting where the network learns the patterns presented but is unable to generalize. Bayesian regularization addresses these issues by requiring the weights of the network to attain a minimum magnitude. This ensures that non-contributing weights are reduced significantly and the resulting network represents the essence of the inter-relations of the training. Bayesian Regularization simply introduces an additional term to the objective function. This term comprises the sum of the squares of the weights. The optimization process therefore not only achieves the objective of the original cost (i.e. the minimization of an error metric) but it also ensures that this objective is achieved with minimum-magnitude weights. We have introduced Bayesian Regularization in the training of Restricted Boltzmann Machines and have applied this method in experiments of hand-written numbers classification. Our experiments showed that by adding Bayesian regularization in the training of RBMs, we were able to improve the generalization capabilities of the trained network by reducing its recognition errors by more than 1.6%.
Starting Page	2935
Ending Page	2939
File Size	3570268
Page Count	5
File Format	PDF
ISSN	21614407
e-ISBN	9781479914845
DOI	10.1109/IJCNN.2014.6889458
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-07-06
Publisher Place	China
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Training Neurons Bayes methods Approximation methods Educational institutions Testing Feature extraction classification Restricted Boltzmann Machine over fitting regularization
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Music genre recognition with risk and rejection

Credit Risk Classification Using Discriminative Restricted Boltzmann Machines

Real-Time Keypoint Recognition Using Restricted Boltzmann Machine

The Restricted Variational Bayes Approximation in Bayesian Filtering

Comparison of different variants of Restricted Boltzmann Machines

Feature extraction using Restricted Boltzmann Machine for stock price prediction

Classifier comparison for MSER-based text classification in scene images

Scalable learning for restricted Boltzmann machines

Image classification using RBM to encode local descriptors with group sparse learning

An improved RBM based on Bayesian Regularization

Similar Documents

Music genre recognition with risk and rejection

Credit Risk Classification Using Discriminative Restricted Boltzmann Machines

Real-Time Keypoint Recognition Using Restricted Boltzmann Machine

The Restricted Variational Bayes Approximation in Bayesian Filtering

Comparison of different variants of Restricted Boltzmann Machines

Feature extraction using Restricted Boltzmann Machine for stock price prediction

Classifier comparison for MSER-based text classification in scene images

Scalable learning for restricted Boltzmann machines

Image classification using RBM to encode local descriptors with group sparse learning

An improved RBM based on Bayesian Regularization