NDLI: Beyond audio and video retrieval: topic-oriented multimedia summarization

Please wait, while we are loading the content...

Beyond audio and video retrieval: topic-oriented multimedia summarization

Content Provider	Springer Nature Link
Author	Metze, Florian Ding, Duo Hauptmann, Alexander Younessian, Ehsan
Copyright Year	2013
Abstract	Given the deluge of multimedia content that is becoming available over the Internet, it is increasingly important to be able to effectively examine and organize these large stores of information in ways that go beyond browsing or collaborative filtering. In this paper, we review previous work on audio and video processing, and define the task of topic-oriented multimedia summarization (TOMS) using natural language generation (NLG): given a set of automatically extracted features from a video, a TOMS system will automatically generate a paragraph of natural language, which summarizes the important information in a video belonging to a certain topic, and for example provides explanations for why a video was matched and retrieved. Possible features include visual semantic concepts, objects, and actions, environmental sounds, and transcripts from automatic speech recognition (ASR). We see this as a first step towards systems that will be able to discriminate visually similar, but semantically different videos, compare two videos and provide textual output or summarize a large number of videos at once. In this paper, we introduce our approach of solving the TOMS problem. We extract various visual concept features, environmental sounds and ASR transcription features from a given video, and develop a template-based NLG system to produce a textual recounting based on the extracted features. We also propose possible experimental designs for continuously evaluating and improving TOMS systems, and present results of a pilot evaluation of our initial system.
Starting Page	131
Ending Page	144
Page Count	14
File Format	PDF
ISSN	21926611
Journal	International Journal of Multimedia Information Retrieval
Volume Number	2
Issue Number	2
e-ISSN	2192662X
Language	English
Publisher	Springer-Verlag
Publisher Date	2013-01-04
Publisher Place	London
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Natural language generation Event detection and recounting Information Storage and Retrieval Data Mining and Knowledge Discovery Image Processing and Computer Vision Computer Science Information Systems Applications (incl. Internet) Multimedia Information Systems Multimedia summarization
Content Type	Text
Resource Type	Article
Subject	Library and Information Sciences Information Systems Media Technology

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Beyond audio and video retrieval: topic-oriented multimedia summarization

On the use of commonsense ontology for multimedia event recounting

MGraph: multimodal event summarization in social media using topic models and graph-based ranking

Beyond audio and video retrieval: towards multimedia summarization

Video concept detection by audio-visual grouplets

Special issue on video retrieval

Text-to-video: a semantic search engine for internet videos

A study on video data mining

Top multimedia information retrieval papers

Beyond audio and video retrieval: topic-oriented multimedia summarization

Similar Documents

Beyond audio and video retrieval: topic-oriented multimedia summarization

On the use of commonsense ontology for multimedia event recounting

MGraph: multimodal event summarization in social media using topic models and graph-based ranking

Beyond audio and video retrieval: towards multimedia summarization

Video concept detection by audio-visual grouplets

Special issue on video retrieval

Text-to-video: a semantic search engine for internet videos

A study on video data mining

Top multimedia information retrieval papers

Beyond audio and video retrieval: topic-oriented multimedia summarization