NDLI: Dynamic gesture recognition based on feature fusion network and variant ConvLSTM

Content Provider	IET Digital Library
Author	Peng, Yuqing Tao, Huifang Li, Wei Yuan, Hongtao Li, Tiejun
Abstract	Gesture is a natural form of human communication, and it is of great significance in human–computer interaction. In the dynamic gesture recognition method based on deep learning, the key is to obtain comprehensive gesture feature information. Aiming at the problem of inadequate extraction of spatiotemporal features or loss of feature information in current dynamic gesture recognition, a new gesture recognition architecture is proposed, which combines feature fusion network with variant convolutional long short-term memory (ConvLSTM). The architecture extracts spatiotemporal feature information from local, global and deep aspects, and combines feature fusion to alleviate the loss of feature information. Firstly, local spatiotemporal feature information is extracted from video sequence by 3D residual network based on channel feature fusion. Then the authors use the variant ConvLSTM to learn the global spatiotemporal information of dynamic gesture, and introduce the attention mechanism to change the gate structure of ConvLSTM. Finally, a multi-feature fusion depthwise separable network is used to learn higher-level features including depth feature information. The proposed approach obtains very competitive performance on the Jester dataset with the classification accuracies of 95.59%, achieving state-of-the-art performance with 99.65% accuracy on the SKIG (Sheffifield Kinect Gesture) dataset.
Starting Page	2480
Ending Page	2486
Page Count	7
ISSN	17519659
Volume Number	14
e-ISSN	17519667
Issue Number	Issue 11, Sep (2020)
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/iet-ipr/14/11
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/10.1049/iet-ipr.2019.1248
Journal	IET Image Processing
Publisher Date	2020-04-30
Access Restriction	Open
Rights Holder	© The Institution of Engineering and Technology
Subject Keyword	3D Residual Network Channel Feature Fusion Classification Accuracies Computer Vision And Image Processing Technique Convolutional Neural Nets Deep Aspects Deep Learning Depth Feature Information Dynamic Gesture Recognition Method Feature Extraction Feature Fusion Network Gesture Feature Information Gesture Recognition Gesture Recognition Architecture Global Aspects Global Spatiotemporal Information Higher-level Feature Human Communication Human Computer Interaction Human-computer Interaction Image Classification Image Fusion Image Recognition Image Sequence Jester Dataset Learning in AI Local Aspects Local Spatiotemporal Feature Information Multifeature Fusion Depthwise Separable Network Neural Computing Technique Recurrent Neural Nets Sheffifield Kinect Gesture Dataset SKIG Dataset Spatiotemporal Feature Extraction Spatiotemporal Phenomena User InterFace Variant ConvLSTM Variant Convolutional Long Short-term Memory Video Sequence Video Signal Processing
Content Type	Text
Resource Type	Article
Subject	Signal Processing Electrical and Electronic Engineering Computer Vision and Pattern Recognition Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

DeepFuseOSV: online signature verification using hybrid feature fusion and depthwise separable convolution neural network architecture

RGB-D static gesture recognition based on convolutional neural network

Fully convolutional networks for action recognition

Multi-mode neural network for human action recognition

HGR-Net: a fusion network for hand gesture segmentation and recognition

Gesture recognition algorithm based on multi-scale feature fusion in RGB-D images

Feature Fusion Based Hand Gesture Recognition Method for Automotive Interfaces

Joint learning of convolution neural networks for RGB-D-based human action recognition

Modulation scheme recognition using convolutional neural network

Dynamic gesture recognition based on feature fusion network and variant ConvLSTM

Similar Documents

DeepFuseOSV: online signature verification using hybrid feature fusion and depthwise separable convolution neural network architecture

RGB-D static gesture recognition based on convolutional neural network

Fully convolutional networks for action recognition

Multi-mode neural network for human action recognition

HGR-Net: a fusion network for hand gesture segmentation and recognition

Gesture recognition algorithm based on multi-scale feature fusion in RGB-D images

Feature Fusion Based Hand Gesture Recognition Method for Automotive Interfaces

Joint learning of convolution neural networks for RGB-D-based human action recognition

Modulation scheme recognition using convolutional neural network

Dynamic gesture recognition based on feature fusion network and variant ConvLSTM