NDLI: Combination of temporal-channels correlation information and bilinear feature for action recognition

Content Provider	IET Digital Library
Author	Cai, Jiahui Hu, Jianguo Li, Shiren Lin, Jialing Wang, Jun
Abstract	In this study, the authors focus on improving the spatio–temporal representation ability of three-dimensional (3D) convolutional neural networks (CNNs) in the video domain. They observe two unfavourable issues: (i) the convolutional filters only dedicate to learning local representation along input channels. Also they treat channel-wise features equally, without emphasising the important features; (ii) traditional global average pooling layer only captures first-order statistics, ignoring finer detail features useful for classification. To mitigate these problems, they proposed two modules to boost 3D CNNs’ performance, which are temporal-channel correlation (TCC) and bilinear pooling module. The TCC module can capture the information of inter-channel correlations over the temporal domain. Moreover, the TCC module generates channel-wise dependencies, which can adaptively re-weight the channel-wise features. Therefore, the network can focus on learning important features. With regards to the bilinear pooling module, it can capture more complex second-order statistics in deep features and generate a second-order classification vector. We can get more accurate classification results by combining the first-order and second-order classification vector. Extensive experiments show that adding our proposed modules to I3D network could consistently improve the performance and outperform the state-of-the-art methods. The code and models are available at https://github.com/caijh33/I3D_TCC_Bilinear.
Starting Page	634
Ending Page	641
Page Count	8
ISSN	17519632
Volume Number	14
e-ISSN	17519640
Issue Number	Issue 8, Dec (2020)
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/iet-cvi/14/8
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/10.1049/iet-cvi.2020.0023
Journal	IET Computer Vision
Publisher Date	2020-10-07
Access Restriction	Open
Rights Holder	© The Institution of Engineering and Technology
Subject Keyword	3D CNN Algebra Bilinear Feature Bilinear Pooling Module Channel-wise Dependencies Channel-wise Features Computer Vision And Image Processing Technique Convolutional Filter Convolutional Neural Nets Correlation Method Deep Features First-order Classification Vector First-order Statistics Higher Order Statistics I3D Network Image Classification Image Recognition Image Representation Learning in AI Neural Computing Technique Object Recognition Second-order Classification Vector Second-order Statistics Spatio–temporal Representation Statistics Stereo Image Processing TCC Module Temporal-channel Correlation Temporal-channels Correlation Information Three-dimensional Convolutional Neural Network Vector Video Domain Video Signal Processing
Content Type	Text
Resource Type	Article
Subject	Computer Vision and Pattern Recognition Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Fusing HOG and convolutional neural network spatial–temporal features for video-based facial expression recognition

Multi-mode neural network for human action recognition

Learning pairwise SVM on hierarchical deep features for ear recognition

Effect of fusing features from multiple DCNN architectures in image classification

Capturing the spatio-temporal continuity for video semantic segmentation

Face recognition with compressed Fisher vector on multiscale convolutional features

Bi-directional CRC algorithm using CNN-based features for face classification

Balanced principal component for 3D shape recognition using convolutional neural networks

Animal classification using facial images with score-level fusion

Combination of temporal-channels correlation information and bilinear feature for action recognition

Similar Documents

Fusing HOG and convolutional neural network spatial–temporal features for video-based facial expression recognition

Multi-mode neural network for human action recognition

Learning pairwise SVM on hierarchical deep features for ear recognition

Effect of fusing features from multiple DCNN architectures in image classification

Capturing the spatio-temporal continuity for video semantic segmentation

Face recognition with compressed Fisher vector on multiscale convolutional features

Bi-directional CRC algorithm using CNN-based features for face classification

Balanced principal component for 3D shape recognition using convolutional neural networks

Animal classification using facial images with score-level fusion

Combination of temporal-channels correlation information and bilinear feature for action recognition