NDLI: Extraction of texture and geometrical features from informative facial regions for sign language recognition

Content Provider	Springer Nature Link
Author	Kumar, Sunil Bhuyan, M. K. Chakraborty, Biplab Ketan
Copyright Year	2017
Abstract	In general, the most common form of gestures is made up of movements of the hand and/or arm associated with facial expressions. In this, the hand is used to make different message signs, while facial movements are used to reflect the mood and emotion of the person. In this paper, some sign language gestures are recognized only with the help of associated facial expressions. Existing facial expression based sign language recognition (SLR) methods only used facial geometric features to recognize sign language gestures. However, the performance of geometric feature-based SLR methods depends on the accuracy of tracking algorithms and the number of facial landmark points. Additionally, facial textures are more informative as compared to the geometric features of a face. Inspiring from these facts, we propose to recognize sign language gestures with the help of spatio-temporal characteristics of facial texture patterns. For this, a new face model is proposed by extracting texture features only from the informative regions of a face. The proposed face model can also be employed to extract the geometrical features of a face. The features extracted from the informative regions of a face are significantly discriminative, and so the proposed face model can track/encode the facial dynamics of the associated facial expressions of a sign. Finally, a 3-state hidden conditional random field is employed to model the texture variations of facial gestures. Experimental results on RWTH-BOSTON data-set show that proposed method can achieve upto 80.06% recognition rate.
Starting Page	227
Ending Page	239
Page Count	13
File Format	PDF
ISSN	17837677
Journal	Journal on Multimodal User Interfaces
Volume Number	11
Issue Number	2
e-ISSN	17838738
Language	English
Publisher	Springer International Publishing
Publisher Date	2017-02-18
Publisher Place	Cham
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Sign language recognition Facial expression Local binary pattern Active appearance model (AAM) Hidden conditional random field (HCRF) User Interfaces and Human Computer Interaction Signal,Image and Speech Processing Image Processing and Computer Vision
Content Type	Text
Resource Type	Article
Subject	Signal Processing Human-Computer Interaction

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Speech and sliding text aided sign retrieval from hearing impaired sign news videos

Multimodal user interface for the communication of the disabled

A novel set of features for continuous hand gesture recognition

The influence of facial interface design on dynamic emotional recognition

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Synthesis of expressive facial animations: A multimodal caricatural mirror

MARC: a framework that features emotion models for facial animation during human–computer interaction

Automatic fingersign-to-speech translation system

Induction, recording and recognition of natural emotions from facial expressions and speech prosody

Extraction of texture and geometrical features from informative facial regions for sign language recognition

Similar Documents

Speech and sliding text aided sign retrieval from hearing impaired sign news videos

Multimodal user interface for the communication of the disabled

A novel set of features for continuous hand gesture recognition

The influence of facial interface design on dynamic emotional recognition

Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis

Synthesis of expressive facial animations: A multimodal caricatural mirror

MARC: a framework that features emotion models for facial animation during human–computer interaction

Automatic fingersign-to-speech translation system

Induction, recording and recognition of natural emotions from facial expressions and speech prosody

Extraction of texture and geometrical features from informative facial regions for sign language recognition