NDLI: Continuous action segmentation and recognition using hybrid convolutional neural network-hidden Markov model model

Content Provider	IET Digital Library
Author	Lei, Jun Li, Guohui Zhang, Jun Guo, Qiang Tu, Dan
Abstract	Continuous action recognition in video is more complicated compared with traditional isolated action recognition. Besides the high variability of postures and appearances of each action, the complex temporal dynamics of continuous action makes this problem challenging. In this study, the authors propose a hierarchical framework combining convolutional neural network (CNN) and hidden Markov model (HMM), which recognises and segments continuous actions simultaneously. The authors utilise the CNN's powerful capacity of learning high level features directly from raw data, and use it to extract effective and robust action features. The HMM is used to model the statistical dependences over adjacent sub-actions and infer the action sequences. In order to combine the advantages of these two models, the hybrid architecture of CNN-HMM is built. The Gaussian mixture model is replaced by CNN to model the emission distribution of HMM. The CNN-HMM model is trained using embedded Viterbi algorithm, and the data used to train CNN are labelled by forced alignment. The authors test their method on two public action dataset Weizmann and KTH. Experimental results show that the authors’ method achieves improved recognition and segmentation accuracy compared with several other methods. The superior property of features learnt by CNN is also illustrated.
Starting Page	537
Ending Page	544
Page Count	8
ISSN	17519632
Volume Number	10
e-ISSN	17519640
Issue Number	Issue 6, Sep (2016)
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/iet-cvi/10/6
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/10.1049/iet-cvi.2015.0408
Journal	IET Computer Vision
Publisher Date	2016-02-19
Access Restriction	Open
Rights Holder	© The Institution of Engineering and Technology
Subject Keyword	CNN-HMM Continuous Action Recognition Continuous Action Segmentation Convolutional Neural Network Gaussian Mixture Model Gaussian Processes Hidden Markov Model HMM Hybrid Convolutional Neural Network-hidden Markov Model Model Image Recognition Image Segmentation Isolated Action Recognition Markov Processes Neural Computing Technique Neural Nets Optical, Image And Video Signal Processing Statistical Dependences Video Signal Processing Viterbi Algorithm
Content Type	Text
Resource Type	Article
Subject	Computer Vision and Pattern Recognition Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Convolutional recurrent neural networks with hidden Markov model bootstrap for scene text recognition

State-of-art analysis of image denoising methods using convolutional neural networks

Audio classification using braided convolutional neural networks

Evaluating Convolutional Neural Network and Hidden Markov Model for Recognising Surgical Phases in Sigmoid Resection

Video segmentation scheme based on AMC

Fully convolutional networks for action recognition

Foreground segmentation using multiscale convolutional neural network

Adaptive convolutional neural network for large change in video object segmentation

Turkish Speech Recognition Based On Deep Neural Networks

Continuous action segmentation and recognition using hybrid convolutional neural network-hidden Markov model model

Similar Documents

Convolutional recurrent neural networks with hidden Markov model bootstrap for scene text recognition

State-of-art analysis of image denoising methods using convolutional neural networks

Audio classification using braided convolutional neural networks

Evaluating Convolutional Neural Network and Hidden Markov Model for Recognising Surgical Phases in Sigmoid Resection

Video segmentation scheme based on AMC

Fully convolutional networks for action recognition

Foreground segmentation using multiscale convolutional neural network

Adaptive convolutional neural network for large change in video object segmentation

Turkish Speech Recognition Based On Deep Neural Networks

Continuous action segmentation and recognition using hybrid convolutional neural network-hidden Markov model model