NDLI: Detecting Depression using Vocal, Facial and Semantic Communication Cues

Please wait, while we are loading the content...

Multimodal Analysis of Impressions and Personality in Human-Computer and Human-Robot Interactions

Detecting Depression using Vocal, Facial and Semantic Communication Cues

Online Affect Tracking with Multimodal Kalman Filters

Decision Tree Based Depression Classification from Audio Video and Language Information

AVEC 2016: Depression, Mood, and Emotion Recognition Workshop and Challenge

Staircase Regression in OA RVM, Data Selection and Gender Dependency in AVEC 2016

Continuous Multimodal Human Affect Estimation using Echo State Networks

Multi-Modal Audio, Video and Physiological Sensor Learning for Continuous Emotion Prediction

Depression Assessment by Fusing High and Low Level Features from Audio, Video, and Text

Multimodal Emotion Recognition for AVEC 2016 Challenge

DepAudioNet: An Efficient Deep Model for Audio based Depression Classification

Exploring Multimodal Visual Features for Continuous Affect Recognition

Multimodal and Multiresolution Depression Detection from Speech and Facial Landmark Features

High-Level Geometry-based Features of Video Modality for Emotion Prediction

Detecting Depression using Vocal, Facial and Semantic Communication Cues

Content Provider	ACM Digital Library
Author	Dagli, Charlie Godoy, Elizabeth Gwon, Youngjune Williamson, James R. Quatieri, Thomas F. Kung, Hsiang-Tsung Cha, Miriam Khorrami, Pooya Schwarzentruber, Adrianne
Abstract	Major depressive disorder (MDD) is known to result in neurophysiological and neurocognitive changes that affect control of motor, linguistic, and cognitive functions. MDD's impact on these processes is reflected in an individual's communication via coupled mechanisms: vocal articulation, facial gesturing and choice of content to convey in a dialogue. In particular, MDD-induced neurophysiological changes are associated with a decline in dynamics and coordination of speech and facial motor control, while neurocognitive changes influence dialogue semantics. In this paper, biomarkers are derived from all of these modalities, drawing first from previously developed neurophysiologically-motivated speech and facial coordination and timing features. In addition, a novel indicator of lower vocal tract constriction in articulation is incorporated that relates to vocal projection. Semantic features are analyzed for subject/avatar dialogue content using a sparse coded lexical embedding space, and for contextual clues related to the subject's present or past depression status. The features and depression classification system were developed for the 6th International Audio/Video Emotion Challenge (AVEC), which provides data consisting of audio, video-based facial action units, and transcribed text of individuals communicating with the human-controlled avatar. A clinical Patient Health Questionnaire (PHQ) score and binary depression decision are provided for each participant. PHQ predictions were obtained by fusing outputs from a Gaussian staircase regressor for each feature set, with results on the development set of mean F1=0.81, RMSE=5.31, and MAE=3.34. These compare favorably to the challenge baseline development results of mean F1=0.73, RMSE=6.62, and MAE=5.52. On test set evaluation, our system obtained a mean F1=0.70, which is similar to the challenge baseline test result. Future work calls for consideration of joint feature analyses across modalities in an effort to detect neurological disorders based on the interplay of motor, linguistic, affective, and cognitive components of communication.
Starting Page	11
Ending Page	18
Page Count	8
File Format	PDF
ISBN	9781450345163
DOI	10.1145/2988257.2988263
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-10-16
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Text mining Affective computing Semantic analysis Speech Facial expression Major depressive disorder Challenge
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in