NDLI: Hypothesis testing for evaluating a multimodal pattern recognition framework applied to speaker detection

Content Provider	Springer Nature : BioMed Central
Author	Besson, Patricia Kunt, Murat
Abstract	Background Speaker detection is an important component of many human-computer interaction applications, like for example, multimedia indexing, or ambient intelligent systems. This work addresses the problem of detecting the current speaker in audio-visual sequences. The detector performs with few and simple material since a single camera and microphone meets the needs. Method A multimodal pattern recognition framework is proposed, with solutions provided for each step of the process, namely, the feature generation and extraction steps, the classification, and the evaluation of the system performance. The decision is based on the estimation of the synchrony between the audio and the video signals. Prior to the classification, an information theoretic framework is applied to extract optimized audio features using video information. The classification step is then defined through a hypothesis testing framework in order to get confidence levels associated to the classifier outputs, allowing thereby an evaluation of the performance of the whole multimodal pattern recognition system. Results Through the hypothesis testing approach, the classifier performance can be given as a ratio of detection to false-alarm probabilities. Above all, the hypothesis tests give means for measuring the whole pattern recognition process effciency. In particular, the gain offered by the proposed feature extraction step can be evaluated. As a result, it is shown that introducing such a feature extraction step increases the ability of the classifier to produce good relative instance scores, and therefore, the performance of the pattern recognition process. Conclusion The powerful capacities of hypothesis tests as an evaluation tool are exploited to assess the performance of a multimodal pattern recognition process. In particular, the advantage of performing or not a feature extraction step prior to the classification is evaluated. Although the proposed framework is used here for detecting the speaker in audiovisual sequences, it could be applied to any other classification task involving two spatio-temporal co-occurring signals.
Related Links	https://jneuroengrehab.biomedcentral.com/counter/pdf/10.1186/1743-0003-5-11.pdf
Ending Page	8
Page Count	8
Starting Page	1
File Format	HTM / HTML
ISSN	17430003
DOI	10.1186/1743-0003-5-11
Journal	Journal of NeuroEngineering and Rehabilitation
Issue Number	1
Volume Number	5
Language	English
Publisher	BioMed Central
Publisher Date	2008-03-27
Access Restriction	Open
Subject Keyword	Neurosciences Neurology Rehabilitation Medicine Biomedical Engineering and Bioengineering Mutual Information Audio Signal Mouth Region Audio Feature Video Feature
Content Type	Text
Resource Type	Article
Subject	Health Informatics Rehabilitation
Journal Impact Factor	5.2/2023
5-Year Journal Impact Factor	5.6/2023

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Hypothesis testing for evaluating a multimodal pattern recognition framework applied to speaker detection

A multimodal approach to extract optimized audio features for speaker detection (2005)

Information theoretic optimization of audio features for multimodal speaker detection (2005)

Extraction of audio features specific to speech production for multimodal speaker detection (2008)

M.: Extraction of audio features specific to speech production for multimodal speaker detection

Multimodal speaker localization in a probabilistic framework (2006)

Relevant Feature Selection for Audio-Visual Speech Recognition

Vision based interface system for hands free control of an intelligent wheelchair

Classification complexity in myoelectric pattern recognition

Hypothesis testing for evaluating a multimodal pattern recognition framework applied to speaker detection

Similar Documents

Hypothesis testing for evaluating a multimodal pattern recognition framework applied to speaker detection

A multimodal approach to extract optimized audio features for speaker detection (2005)

Information theoretic optimization of audio features for multimodal speaker detection (2005)

Extraction of audio features specific to speech production for multimodal speaker detection (2008)

M.: Extraction of audio features specific to speech production for multimodal speaker detection

Multimodal speaker localization in a probabilistic framework (2006)

Relevant Feature Selection for Audio-Visual Speech Recognition

Vision based interface system for hands free control of an intelligent wheelchair

Classification complexity in myoelectric pattern recognition

Hypothesis testing for evaluating a multimodal pattern recognition framework applied to speaker detection