NDLI: Multi-task deep neural network acoustic models with model adaptation using discriminative speaker identity for whisper recognition

Content Provider	IEEE Xplore Digital Library
Author	Jingjie Li McLoughlin, I. Cong Liu Shaofei Xue Si Wei
Copyright Year	2015
Description	Author affiliation: Nat. Eng. Lab. of Speech & Language Inf. Process., Univ. of Sci. & Technol. of China, Hefei, China (Jingjie Li; McLoughlin, I.; Shaofei Xue) \|\| Flytek Res., China (Cong Liu; Si Wei)
Abstract	This paper presents a study on large vocabulary continuous whisper automatic recognition (wLVCSR). wLVCSR provides the ability to use ASR equipment in public places without concern for disturbing others or leaking private information. However the task of wLVCSR is much more challenging than normal LVCSR due to the absence of pitch which not only causes the signal to noise ratio (SNR) of whispers to be much lower than normal speech but also leads to flatness and formant shifts in whisper spectra. Furthermore, the amount of whisper data available for training is much less than for normal speech. In this paper, multi-task deep neural network (DNN) acoustic models are deployed to solve these problems. Moreover, model adaptation is performed on the multi-task DNN to normalize speaker and environmental variability in whispers based on discriminative speaker identity information. On a Mandarin whisper dictation task, with 55 hours of whisper data, the proposed SI multi-task DNN model can achieve 56.7% character error rate (CER) improvement over a baseline Gaussian Mixture Model (GMM), discriminatively trained only using the whisper data. Besides, the CER of the proposed model for normal speech can reach 15.2%, which is close to the performance of a state-of-the-art DNN trained with one thousand hours of speech data. From this baseline, the model-adapted DNN gains a further 10.9% CER reduction over the generic model.
Starting Page	4969
Ending Page	4973
File Size	226319
Page Count	5
File Format	PDF
ISBN	9781467369978
DOI	10.1109/ICASSP.2015.7178916
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2015-04-19
Publisher Place	Australia
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Speech Adaptation models Speech recognition Acoustics Data models Neural networks Training Silent speech interface Whisper recognition model adaption speaker code multi-task DNN
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Direct adaptation of hybrid DNN/HMM model for fast speaker adaptation in LVCSR based on speaker code

Context adaptive deep neural networks for fast acoustic model adaptation

An investigation into speaker informed DNN front-end for LVCSR

Multi-speaker modeling and speaker adaptation for DNN-based TTS synthesis

Unsupervised speaker adaptation of deep neural network based on the combination of speaker codes and singular value decomposition for speech recognition

Multi-task learning in deep neural networks for improved phoneme recognition

On combining DNN and GMM with unsupervised speaker adaptation for robust automatic speech recognition

Exploring Gaussian mixture model framework for speaker adaptation of deep neural network acoustic models

Fast speaker adaptation of hybrid NN/HMM model for speech recognition based on discriminative learning of speaker code

Multi-task deep neural network acoustic models with model adaptation using discriminative speaker identity for whisper recognition

Similar Documents

Direct adaptation of hybrid DNN/HMM model for fast speaker adaptation in LVCSR based on speaker code

Context adaptive deep neural networks for fast acoustic model adaptation

An investigation into speaker informed DNN front-end for LVCSR

Multi-speaker modeling and speaker adaptation for DNN-based TTS synthesis

Unsupervised speaker adaptation of deep neural network based on the combination of speaker codes and singular value decomposition for speech recognition

Multi-task learning in deep neural networks for improved phoneme recognition

On combining DNN and GMM with unsupervised speaker adaptation for robust automatic speech recognition

Exploring Gaussian mixture model framework for speaker adaptation of deep neural network acoustic models

Fast speaker adaptation of hybrid NN/HMM model for speech recognition based on discriminative learning of speaker code

Multi-task deep neural network acoustic models with model adaptation using discriminative speaker identity for whisper recognition