NDLI: Improving deep neural networks for LVCSR using dropout and shrinking structure

Content Provider	IEEE Xplore Digital Library
Author	Shiliang Zhang Yebo Bao Pan Zhou Hui Jiang Lirong Dai
Copyright Year	2014
Description	Author affiliation: Dept. of Electr. Eng. & Comput. Sci., York Univ., Toronto, ON, Canada (Hui Jiang) \|\| Nat. Eng. Lab. for Speech & Language Inf. Process., Univ. of Sci. & Technol. of China, Hefei, China (Shiliang Zhang; Yebo Bao; Pan Zhou; Lirong Dai)
Abstract	Recently, the hybrid deep neural networks and hidden Markov models (DNN/HMMs) have achieved dramatic gains over the conventional GMM/HMMs method on various large vocabulary continuous speech recognition (LVCSR) tasks. In this paper, we propose two new methods to further improve the hybrid DNN/HMMs model: i) use dropout as pre-conditioner (DAP) to initialize DNN prior to back-propagation (BP) for better recognition accuracy; ii) employ a shrinking DNN structure (sDNN) with hidden layers decreasing in size from bottom to top for the purpose of reducing model size and expediting computation time. The proposed DAP method is evaluated in a 70-hour Mandarin transcription (PSC) task and the 309-hour Switchboard (SWB) task. Compared with the traditional greedy layer-wise pre-trained DNN, it can achieve about 10% and 6.8% relative recognition error reduction for PSC and SWB tasks respectively. In addition, we also evaluate sDNN as well as its combination with DAP on the SWB task. Experimental results show that these methods can reduce model size to 45% of original size and accelerate training and test time by 55%, without losing recognition accuracy.
Sponsorship	IEEE Signal Process. Soc.
Starting Page	6849
Ending Page	6853
File Size	413790
Page Count	5
File Format	PDF
ISBN	9781479928934
DOI	10.1109/ICASSP.2014.6854927
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-05-04
Publisher Place	Italy
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Hidden Markov models Training Neural networks Speech recognition Speech Computational modeling Switches DNN-HMM dropout dropout as pre-conditioner (DAP) shrinking hidden layer deep neural networks LVCSR
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Improving deep neural networks by using sparse dropout strategy

Improving deep neural networks for LVCSR using rectified linear units and dropout

Investigation of deep neural networks (DNN) for large vocabulary continuous speech recognition: Why DNN surpasses GMMS in acoustic modeling

Direct adaptation of hybrid DNN/HMM model for fast speaker adaptation in LVCSR based on speaker code

Hybrid Deep Neural Network--Hidden Markov Model (DNN-HMM) Based Speech Emotion Recognition

Comparison of syllable-based and phoneme-based DNN-HMM in Japanese speech recognition

Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition

Multiple time-span feature fusion for deep neural network modeling

A cluster-based multiple deep neural networks method for large vocabulary continuous speech recognition

Improving deep neural networks for LVCSR using dropout and shrinking structure

Similar Documents

Improving deep neural networks by using sparse dropout strategy

Improving deep neural networks for LVCSR using rectified linear units and dropout

Investigation of deep neural networks (DNN) for large vocabulary continuous speech recognition: Why DNN surpasses GMMS in acoustic modeling

Direct adaptation of hybrid DNN/HMM model for fast speaker adaptation in LVCSR based on speaker code

Hybrid Deep Neural Network--Hidden Markov Model (DNN-HMM) Based Speech Emotion Recognition

Comparison of syllable-based and phoneme-based DNN-HMM in Japanese speech recognition

Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition

Multiple time-span feature fusion for deep neural network modeling

A cluster-based multiple deep neural networks method for large vocabulary continuous speech recognition

Improving deep neural networks for LVCSR using dropout and shrinking structure