NDLI: Simultaneous and Spatiotemporal Detection of Different Levels of Activity in Multidimensional Data

Content Provider	Directory of Open Access Journals (DOAJ)
Author	Natalie M. Sommer Senem Velipasalar Leanne Hirshfield Yantao Lu Burak Kakillioglu
Abstract	In this work, we present a novel and promising approach to autonomously detect different levels of simultaneous and spatiotemporal activity in multidimensional data. We introduce a new multilabeling technique, which assigns different labels to different regions of interest in the data, and thus, incorporates the spatial aspect. Each label is built to describe the level of activity/motion to be monitored in the spatial location that it represents, in contrast to existing approaches providing only a binary result as the presence or absence of activity. This novel Spatially and Motion-Level Descriptive (SMLD) labeling schema is combined with a Convolutional Long Short Term Memory-based network for classification to capture different levels of activity both spatially and temporally without the use of any foreground or object detection. The proposed approach can be applied to various types of spatiotemporal data captured for completely different application domains. In this paper, it was evaluated on video data as well as respiratory sound data. Metrics commonly associated with multilabeling, namely Hamming Loss and Subset Accuracy, as well as confusion matrix-based measurements are used to evaluate performance. Promising testing results are achieved with an overall Hamming Loss for video datasets close to 0.05, Subset Accuracy close to 80% and confusion matrix-based metrics above 0.9. In addition, our proposed approach's ability in detecting frequent motion patterns based on predicted spatiotemporal activity levels is discussed. Encouraging results have been obtained on the respiratory sound dataset as well, while detecting abnormalities in different parts of the lungs. The experimental results demonstrate that the proposed approach can be applied to various types of spatiotemporal data captured for different application domains.
e-ISSN	21693536
DOI	10.1109/ACCESS.2020.3005633
Journal	IEEE Access
Volume Number	8
Language	English
Publisher	IEEE
Publisher Date	2020-01-01
Publisher Place	United States
Access Restriction	Open
Subject Keyword	Electrical Engineering. Electronics. Nuclear Engineering Lstm Convolutional Lstm Machine Learning Multilabeling Activity Detection Spatiotemporal
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in