NDLI: Robust Multimodal Person Identification With Limited Training Data

Please wait, while we are loading the content...

Robust Multimodal Person Identification With Limited Training Data

Content Provider	IEEE Xplore Digital Library
Author	McLaughlin, N. Ji Ming Crookes, D.
Copyright Year	2013
Abstract	This paper presents a novel method of audio-visual feature-level fusion for person identification where both the speech and facial modalities may be corrupted, and there is a lack of prior knowledge about the corruption. Furthermore, we assume there are limited amount of training data for each modality (e.g., a short training speech segment and a single training facial image for each person). A new multimodal feature representation and a modified cosine similarity are introduced to combine and compare bimodal features with limited training data, as well as vastly differing data rates and feature sizes. Optimal feature selection and multicondition training are used to reduce the mismatch between training and testing, thereby making the system robust to unknown bimodal corruption. Experiments have been carried out on a bimodal dataset created from the SPIDRE speaker recognition database and AR face recognition database with variable noise corruption of speech and occlusion in the face images. The system's speaker identification performance on the SPIDRE database, and facial identification performance on the AR database, is comparable with the literature. Combining both modalities using the new method of multimodal fusion leads to significantly improved accuracy over the unimodal systems, even when both modalities have been corrupted. The new method also shows improved identification accuracy compared with the bimodal systems based on multicondition model training or missing-feature decoding alone.
Starting Page	214
Ending Page	224
Page Count	11
File Size	909719
File Format	PDF
ISSN	21682291
Volume Number	43
Issue Number	2
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2013-01-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Training Speech Training data Face Speech recognition Robustness Noise robustness Limited training data multimodal fusion noisy speech occluded face person identification
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence Signal Processing Human Factors and Ergonomics Control and Systems Engineering Computer Networks and Communications Human-Computer Interaction Computer Science Applications

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Robust Biometric Person Identification Using Automatic Classifier Fusion of Speech, Mouth, and Face Experts

A noise-robust front-end for distributed speech recognition in mobile communications

A monocular system for person tracking: Implementation and testing

Binary Data Embedding Framework for Multiclass Classification

Recent Advances on Singlemodal and Multimodal Face Recognition: A Survey

Multimodal person recognition for human-vehicle interaction

Feature selection for robust automatic speech recognition: a temporal offset approach

Noise robust speaker verification via the fusion of SNR-independent and SNR-dependent PLDA

Exploring Fusion Methods for Multimodal Emotion Recognition with Missing Data

Robust Multimodal Person Identification With Limited Training Data

Similar Documents

Robust Biometric Person Identification Using Automatic Classifier Fusion of Speech, Mouth, and Face Experts

A noise-robust front-end for distributed speech recognition in mobile communications

A monocular system for person tracking: Implementation and testing

Binary Data Embedding Framework for Multiclass Classification

Recent Advances on Singlemodal and Multimodal Face Recognition: A Survey

Multimodal person recognition for human-vehicle interaction

Feature selection for robust automatic speech recognition: a temporal offset approach

Noise robust speaker verification via the fusion of SNR-independent and SNR-dependent PLDA

Exploring Fusion Methods for Multimodal Emotion Recognition with Missing Data

Robust Multimodal Person Identification With Limited Training Data