NDLI: Effect of speed difference between time-expanded speech and moving image of talker’s face on word intelligibility

Content Provider	Springer Nature Link
Author	Sakamoto, Shuichi Tanaka, Akihiro Tsumura, Komi Suzuki, Yôiti
Copyright Year	2009
Abstract	This study investigated effects of asynchrony between speech signal and moving image of talker’s face induced by time-expansion of the speech signal on speech intelligibility. Word intelligibility test was performed to younger listeners. Japanese 4-mora words were uttered by a female speaker. Each word was processed with STRAIGHT software to expand the speech signal by from 0 to 400 ms. These signals were combined with moving image of talker’s face which was kept at original speed. This test was performed under three conditions: visual-only, auditory-only, and auditory-visual (AV) condition. Results showed that intelligibility scores under AV condition were statistically higher than those under auditory-only condition even when the speech signal was expanded by 400 ms. These results suggest that moving image of talker’s face is effective to enhance speech intelligibility if the lag between the speech signal and moving image of talker’s face does not exceed 400 ms.
Starting Page	199
Ending Page	203
Page Count	5
File Format	PDF
ISSN	17837677
Journal	Journal on Multimodal User Interfaces
Volume Number	2
Issue Number	3-4
e-ISSN	17838738
Language	English
Publisher	Springer-Verlag
Publisher Date	2009-08-01
Publisher Place	Berlin, Heidelberg
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Time-expanded speech Moving image of talker’s face Word intelligibillity Audio-visual interaction Lip-reading Image Processing and Computer Vision Signal, Image and Speech Processing User Interfaces and Human Computer Interaction
Content Type	Text
Resource Type	Article
Subject	Signal Processing Human-Computer Interaction

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Lip-reading furhat: audio visual intelligibility of a back projected animated face.

Automatic Lips Reading for Audio-Visual Speech Processing and Recognition

Auditory visual prominence : From intelligibility to behavior

Audio-visual emotion recognition using multi-directional regression and Ridgelet transform

An audio-visual dataset of human–human interactions in stressful situations

RAVEL: an annotated corpus for training robots with audiovisual abilities

Interactive design of multimodal user interfaces : Reducing technical and visual complexity

Eye moving behaviors identification for gaze tracking interaction

Non-visual identification, localization, and selection of entities of interest in a 3D environment

Effect of speed difference between time-expanded speech and moving image of talker’s face on word intelligibility

Similar Documents

Lip-reading furhat: audio visual intelligibility of a back projected animated face.

Automatic Lips Reading for Audio-Visual Speech Processing and Recognition

Auditory visual prominence : From intelligibility to behavior

Audio-visual emotion recognition using multi-directional regression and Ridgelet transform

An audio-visual dataset of human–human interactions in stressful situations

RAVEL: an annotated corpus for training robots with audiovisual abilities

Interactive design of multimodal user interfaces : Reducing technical and visual complexity

Eye moving behaviors identification for gaze tracking interaction

Non-visual identification, localization, and selection of entities of interest in a 3D environment

Effect of speed difference between time-expanded speech and moving image of talker’s face on word intelligibility