NDLI: Multi-modal and multi-scale photo collection summarization

Content Provider	Springer Nature Link
Author	Shen, Xu Tian, Xinmei
Copyright Year	2015
Abstract	With the proliferation of digital cameras and mobile devices, people are taking much more photos than ever before. However, these photos can be redundant in content and varied in quality. Therefore there is a growing need for tools to manage the photo collections. One efficient photo management way is photo collection summarization which segments the photo collection into different events and then selects a set of representative and high quality photos (key photos) from those events. However, existing photo collection summarization methods mainly consider the low-level features for photo representation only, such as color, texture, etc, while ignore many other useful features, for example high-level semantic feature and location. Moreover, they often return fixed summarization results which provide little flexibility. In this paper, we propose a multi-modal and multi-scale photo collection summarization method by leveraging multi-modal features, including time, location and high-level semantic features. We first use Gaussian mixture model to segment photo collection into events. With images represented by those multi-modal features, our event segmentation algorithm can generate better performance since the multi-modal features can better capture the inhomogeneous structure of events. Next we propose a novel key photo ranking and selection algorithm to select representative and high quality photos from the events for summarization. Our key photo ranking algorithm takes the importance of both events and photos into consideration. Furthermore, our photo summarization method allows users to control the scale of event segmentation and number of key photos selected. We evaluate our method by extensive experiments on four photo collections. Experimental results demonstrate that our method achieves better performance than previous photo collection summarization methods.
Starting Page	2527
Ending Page	2541
Page Count	15
File Format	PDF
ISSN	13807501
Journal	Multimedia Tools and Applications
Volume Number	75
Issue Number	5
e-ISSN	15737721
Language	English
Publisher	Springer US
Publisher Date	2015-05-24
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Photo collection summarization Event segmentation Key photo selection Multimedia Information Systems Computer Communication Networks Data Structures, Cryptology and Information Theory Special Purpose and Application-Based Systems
Content Type	Text
Resource Type	Article
Subject	Computer Networks and Communications Hardware and Architecture Software Media Technology

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Unsupervised scene detection and commentator building using multi-modal chains

Sub-event recognition and summarization for structured scenario photos

Object segmentation and key-pose based summarization for motion video

Automatic report generation based on multi-modal information

Unsupervised, efficient and scalable key-frame selection for automatic summarization of surveillance videos

Unsupervised multi-graph cross-modal hashing for large-scale multimedia retrieval

Motion-Based Selection of Relevant Video Segments for Video Summarization

A knowledge-based semantic approach for image collection summarization

Robust handwriting extraction and lecture video summarization

Multi-modal and multi-scale photo collection summarization

Similar Documents

Unsupervised scene detection and commentator building using multi-modal chains

Sub-event recognition and summarization for structured scenario photos

Object segmentation and key-pose based summarization for motion video

Automatic report generation based on multi-modal information

Unsupervised, efficient and scalable key-frame selection for automatic summarization of surveillance videos

Unsupervised multi-graph cross-modal hashing for large-scale multimedia retrieval

Motion-Based Selection of Relevant Video Segments for Video Summarization

A knowledge-based semantic approach for image collection summarization

Robust handwriting extraction and lecture video summarization

Multi-modal and multi-scale photo collection summarization