NDLI: Audio/visual mapping with cross-modal hidden Markov models

Content Provider	IEEE Xplore Digital Library
Author	Shengli Fu Gutierrez-Osuna, R. Esposito, A. Kakumanu, P.K. Garcia, O.N.
Copyright Year	1999
Abstract	The audio/visual mapping problem of speech-driven facial animation has intrigued researchers for years. Recent research efforts have demonstrated that hidden Markov model (HMM) techniques, which have been applied successfully to the problem of speech recognition, could achieve a similar level of success in audio/visual mapping problems. A number of HMM-based methods have been proposed and shown to be effective by the respective designers, but it is yet unclear how these techniques compare to each other on a common test bed. In this paper, we quantitatively compare three recently proposed cross-modal HMM methods, namely the remapping HMM (R-HMM), the least-mean-squared HMM (LMS-HMM), and HMM inversion (HMMI). The objective of our comparison is not only to highlight the merits and demerits of different mapping designs, but also to study the optimality of the acoustic representation and HMM structure for the purpose of speech-driven facial animation. This paper presents a brief overview of these models, followed by an analysis of their mapping capabilities on a synthetic dataset. An empirical comparison on an experimental audio-visual dataset consisting of 75 TIMIT sentences is finally presented. Our results show that HMMI provides the best performance, both on synthetic and experimental audio-visual data.
Sponsorship	IEEE Signal Processing Society IEEE Circuits and Systems Society IEEE Communications Society IEEE Computer Society
Starting Page	243
Ending Page	252
Page Count	10
File Size	793791
File Format	PDF
ISSN	15209210
Volume Number	7
Issue Number	2
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2005-04-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Hidden Markov models Facial animation Streaming media Computer science Natural languages Speech recognition Testing Speech processing Man machine systems speech reading and lip synchroization 3-D audio/video processing joint media and multimodal processing
Content Type	Text
Resource Type	Article
Subject	Signal Processing Electrical and Electronic Engineering Computer Science Applications Media Technology

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Audio/visual mapping with cross-modal hidden Markov models (2005)

Dynamic Audio-Visual Mapping using Fused Hidden Markov Model Inversion Method

Animating expressive faces across languages

Learning dynamic audio-visual mapping with input-output Hidden Markov models

Speech-driven facial animation with realistic dynamics

Using HMMs in audio-to-visual conversion

Animation of generic 3D Head models driven by speech

Audio-Visual Integration In Multimodal Communication (1998)

Visual model structures and synchrony constraints for audio-visual speech recognition

Audio/visual mapping with cross-modal hidden Markov models

Similar Documents

Audio/visual mapping with cross-modal hidden Markov models (2005)

Dynamic Audio-Visual Mapping using Fused Hidden Markov Model Inversion Method

Animating expressive faces across languages

Learning dynamic audio-visual mapping with input-output Hidden Markov models

Speech-driven facial animation with realistic dynamics

Using HMMs in audio-to-visual conversion

Animation of generic 3D Head models driven by speech

Audio-Visual Integration In Multimodal Communication (1998)

Visual model structures and synchrony constraints for audio-visual speech recognition

Audio/visual mapping with cross-modal hidden Markov models