NDLI: Multimodal user’s affective state analysis in naturalistic interaction

Content Provider	Springer Nature Link
Author	Caridakis, George Karpouzis, Kostas Wallace, Malis Kessous, Loic Amir, am
Copyright Year	2009
Abstract	Affective and human-centered computing have attracted an abundance of attention during the past years, mainly due to the abundance of environments and applications able to exploit and adapt to multimodal input from the users. The combination of facial expressions with prosody information allows us to capture the users’ emotional state in an unintrusive manner, relying on the best performing modality in cases where one modality suffers from noise or bad sensing conditions. In this paper, we describe a multi-cue, dynamic approach to detect emotion in naturalistic video sequences, where input is taken from nearly real world situations, contrary to controlled recording conditions of audiovisual material. Recognition is performed via a recurrent neural network, whose short term memory and approximation capabilities cater for modeling dynamic events in facial and prosodic expressivity. This approach also differs from existing work in that it models user expressivity using a dimensional representation, instead of detecting discrete ‘universal emotions’, which are scarce in everyday human-machine interaction. The algorithm is deployed on an audiovisual database which was recorded simulating human-human discourse and, therefore, contains less extreme expressivity and subtle variations of a number of emotion labels. Results show that in turns lasting more than a few frames, recognition rates rise to 98%.
Starting Page	49
Ending Page	66
Page Count	18
File Format	PDF
ISSN	17837677
Journal	Journal on Multimodal User Interfaces
Volume Number	3
Issue Number	1-2
e-ISSN	17838738
Language	English
Publisher	Springer-Verlag
Publisher Date	2009-12-15
Publisher Place	Berlin, Heidelberg
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Image Processing and Computer Vision Signal, Image and Speech Processing User Interfaces and Human Computer Interaction
Content Type	Text
Resource Type	Article
Subject	Signal Processing Human-Computer Interaction

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

An open source integrated framework for rapid prototyping of multimodal affective applications in digital entertainment

A cross-cultural, multimodal, affective corpus for gesture expressivity analysis

Facial expression-based affective speech translation

Best of affective computing and intelligent interaction 2013 in multimodal interactions : Guest Editorial

Cross-disciplinary approaches to multimodal user interfaces

Enriching the user experience with multimodal interfaces

Multimodal user interface for the communication of the disabled

Interactive design of multimodal user interfaces : Reducing technical and visual complexity

Prototyping and evaluating glove-based multimodal interfaces

Multimodal user’s affective state analysis in naturalistic interaction

Similar Documents

An open source integrated framework for rapid prototyping of multimodal affective applications in digital entertainment

A cross-cultural, multimodal, affective corpus for gesture expressivity analysis

Facial expression-based affective speech translation

Best of affective computing and intelligent interaction 2013 in multimodal interactions : Guest Editorial

Cross-disciplinary approaches to multimodal user interfaces

Enriching the user experience with multimodal interfaces

Multimodal user interface for the communication of the disabled

Interactive design of multimodal user interfaces : Reducing technical and visual complexity

Prototyping and evaluating glove-based multimodal interfaces

Multimodal user’s affective state analysis in naturalistic interaction