NDLI: Adaptive Reliability Measure and Optimum Integration Weight for Decision Fusion Audio-visual Speech Recognition

Content Provider	Springer Nature Link
Author	Rajavel, R. Sathidevi, P. S.
Copyright Year	2011
Abstract	Audio-visual speech recognition (AVSR) using acoustic and visual signals of speech has received attention recently because of its robustness in noisy environments. An important issue in decision fusion based AVSR system is the determination of appropriate integration weight for the speech modalities to integrate and ensure better performance under various SNR conditions. Generally, the integration weight is calculated from the relative reliability of two modalities. This paper investigates the effect of reliability measure on integration weight estimation and proposes a genetic algorithm (GA) based reliability measure which uses optimum number of best recognition hypotheses rather than N best recognition hypotheses to determine an appropriate integration weight. Further improvement in recognition accuracy is achieved by optimizing the above measured integration weight by genetic algorithm. The performance of the proposed integration weight estimation scheme is demonstrated for isolated word recognition (incorporating commonly used functions in mobile phones) via multi-speaker database experiment. The results show that the proposed schemes improve robust recognition accuracy over the conventional unimodal systems, and a couple of related existing bimodal systems, namely, the baseline reliability ratio-based system and N best recognition hypotheses reliability ratio-based system under various SNR conditions.
Starting Page	83
Ending Page	93
Page Count	11
File Format	PDF
ISSN	19398018
Journal	Journal of Signal Processing Systems
Volume Number	68
Issue Number	1
e-ISSN	19398115
Language	English
Publisher	Springer US
Publisher Date	2011-02-02
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Audio-visual speech recognition Side face visual feature extraction Audio-visual decision fusion Reliability-ratio based weight optimization GA based reliability measure Computer Imaging, Vision, Pattern Recognition and Graphics Circuits and Systems Image Processing and Computer Vision Pattern Recognition Electrical Engineering Signal, Image and Speech Processing
Content Type	Text
Resource Type	Article
Subject	Theoretical Computer Science Signal Processing Control and Systems Engineering Information Systems Modeling and Simulation Hardware and Architecture

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

The Effect of Reliability Measure on Integration Weight Estimation in Audio-Visual Speech Recognition

MPEG-4 Systems: Architecting Object-Based Audio-Visual Content

Comparison between different feature extraction techniques for audio-visual speech recognition

Audio-Visual Speaker Recognition for Video Broadcast News

Multivariate mutual information for audio video fusion

Adaptive region-based image fusion using energy evaluation model for fusion decision

Speech-to-Lip Movement Synthesis by Maximizing Audio-Visual Joint Probability Based on the EM Algorithm

Real-time Stereo Face Recognition by Fusing Appearance and Depth Fisherfaces

Face Recognition with Integrating Multiple Cues

Adaptive Reliability Measure and Optimum Integration Weight for Decision Fusion Audio-visual Speech Recognition

Similar Documents

The Effect of Reliability Measure on Integration Weight Estimation in Audio-Visual Speech Recognition

MPEG-4 Systems: Architecting Object-Based Audio-Visual Content

Comparison between different feature extraction techniques for audio-visual speech recognition

Audio-Visual Speaker Recognition for Video Broadcast News

Multivariate mutual information for audio video fusion

Adaptive region-based image fusion using energy evaluation model for fusion decision

Speech-to-Lip Movement Synthesis by Maximizing Audio-Visual Joint Probability Based on the EM Algorithm

Real-time Stereo Face Recognition by Fusing Appearance and Depth Fisherfaces

Face Recognition with Integrating Multiple Cues

Adaptive Reliability Measure and Optimum Integration Weight for Decision Fusion Audio-visual Speech Recognition