NDLI: Attention-based spatial–temporal hierarchical ConvLSTM network for action recognition in videos

Content Provider	IET Digital Library
Author	Xue, Fei Ji, Hongbing Zhang, Wenbo Cao, Yi
Abstract	Human action recognition in videos is an important research topic in computer vision due to its wide applications. Actions naturally contain both spatial and temporal information. The key to action recognition is to model the spatial and temporal structures of actions. In this study, the authors propose an attention-based spatial–temporal hierarchical convolutional long short-term memory (ST-HConvLSTM) network to model the structures of actions in the spatial and temporal domains. The ST-HConvLSTM consists of two parts: a spatial–temporal attention module and a novel LSTM-like architecture named hierarchical ConvLSTM (HConvLSTM). The HConvLSTM can model the spatial and temporal structures of actions. The spatial–temporal attention module can figure out which part of the video is more discriminative for action recognition and makes the HConvLSTM focus on it. In addition, a weighted fusion strategy is proposed to fuse the appearance information and motion information of the video. The proposed ST-HConvLSTM is evaluated on UCF101, HMDB51 and Kinetics datasets. Experimental results show that the authors’ proposed ST-HConvLSTM achieves state-of-the-art performance compared with other recent LSTM-like architectures and attention-based methods.
Starting Page	708
Ending Page	718
Page Count	11
ISSN	17519632
Volume Number	13
e-ISSN	17519640
Issue Number	Issue 8, Dec (2019)
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/iet-cvi/13/8
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/10.1049/iet-cvi.2018.5830
Journal	IET Computer Vision
Publisher Date	2019-09-04
Access Restriction	Open
Rights Holder	© The Institution of Engineering and Technology
Subject Keyword	Appearance Information Computer Vision Computer Vision And Image Processing Technique HMDB51 Datasets Human Action Recognition Image Motion Analysis Image Recognition Kinetics Datasets Motion Information Spatial Domain Spatial Information Spatial Structure Spatial–temporal Attention Module Spatial–temporal Hierarchical ConvLSTM Network Spatial–temporal Hierarchical Convolutional Long Short-term Memory ST-HConvLSTM Temporal Domain Temporal Information Temporal Structure UCF101 Datasets Video Signal Processing Weighted Fusion Strategy
Content Type	Text
Resource Type	Article
Subject	Computer Vision and Pattern Recognition Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Fully convolutional networks for action recognition

Depth-based end-to-end deep network for human action recognition

Pedestrian motion recognition via Conv-VLAD integrated spatial-temporal-relational network

Dual attention convolutional network for action recognition

Dimensionality reduction of Fisher vectors for human action recognition

Attention-based Temporal Weighted Convolutional Neural Network for Action Recognition

Stacked residual blocks based encoder–decoder framework for human motion prediction

Skeleton-based attention-aware spatial–temporal model for action detection and recognition

Attention-Based Temporal Weighted Convolutional Neural Network for Action Recognition

Attention-based spatial–temporal hierarchical ConvLSTM network for action recognition in videos

Similar Documents

Fully convolutional networks for action recognition

Depth-based end-to-end deep network for human action recognition

Pedestrian motion recognition via Conv-VLAD integrated spatial-temporal-relational network

Dual attention convolutional network for action recognition

Dimensionality reduction of Fisher vectors for human action recognition

Attention-based Temporal Weighted Convolutional Neural Network for Action Recognition

Stacked residual blocks based encoder–decoder framework for human motion prediction

Skeleton-based attention-aware spatial–temporal model for action detection and recognition

Attention-Based Temporal Weighted Convolutional Neural Network for Action Recognition

Attention-based spatial–temporal hierarchical ConvLSTM network for action recognition in videos