NDLI: Multi-task deep visual-semantic embedding for video thumbnail selection

Content Provider	IEEE Xplore Digital Library
Author	Wu Liu Tao Mei Yongdong Zhang Che, C. Jiebo Luo
Copyright Year	2015
Description	Author affiliation: Univ. of Rochester, Rochester, NY, USA (Jiebo Luo) \|\| Microsoft Res., Beijing, China (Tao Mei; Che, C.) \|\| Inst. of Comput. Technol., Beijing, China (Wu Liu; Yongdong Zhang)
Abstract	Given the tremendous growth of online videos, video thumbnail, as the common visualization form of video content, is becoming increasingly important to influence user's browsing and searching experience. However, conventional methods for video thumbnail selection often fail to produce satisfying results as they ignore the side semantic information (e.g., title, description, and query) associated with the video. As a result, the selected thumbnail cannot always represent video semantics and the click-through rate is adversely affected even when the retrieved videos are relevant. In this paper, we have developed a multi-task deep visual-semantic embedding model, which can automatically select query-dependent video thumbnails according to both visual and side information. Different from most existing methods, the proposed approach employs the deep visual-semantic embedding model to directly compute the similarity between the query and video thumbnails by mapping them into a common latent semantic space, where even unseen query-thumbnail pairs can be correctly matched. In particular, we train the embedding model by exploring the large-scale and freely accessible click-through video and image data, as well as employing a multi-task learning strategy to holistically exploit the query-thumbnail relevance from these two highly related datasets. Finally, a thumbnail is selected by fusing both the representative and query relevance scores. The evaluations on 1,000 query-thumbnail dataset labeled by 191 workers in Amazon Mechanical Turk have demonstrated the effectiveness of our proposed method.
Starting Page	3707
Ending Page	3715
File Size	1350883
Page Count	9
File Format	PDF
ISSN	10636919
e-ISBN	9781467369640
DOI	10.1109/CVPR.2015.7298994
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2015-06-07
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Semantics Visualization Training Computational modeling Mathematical model Data models Nails
Content Type	Text
Resource Type	Article
Subject	Computer Vision and Pattern Recognition Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Semantic Transform: Weakly Supervised Semantic Inference for Relating Visual Attributes

Towards Micro-video Thumbnail Selection via a Multi-label Visual-semantic Embedding Model

Spatial-DiscLDA for visual recognition

Learning Fine-Grained Image Similarity with Deep Ranking

Label Consistent Quadratic Surrogate model for visual saliency prediction

The Shape-Time Random Field for Semantic Video Labeling

Reconfigurable models for scene recognition

Learning Video Saliency from Human Gaze Using Candidate Selection

Semi-supervised Learning of Geospatial Objects through Multi-modal Data Integration

Multi-task deep visual-semantic embedding for video thumbnail selection

Similar Documents

Semantic Transform: Weakly Supervised Semantic Inference for Relating Visual Attributes

Towards Micro-video Thumbnail Selection via a Multi-label Visual-semantic Embedding Model

Spatial-DiscLDA for visual recognition

Learning Fine-Grained Image Similarity with Deep Ranking

Label Consistent Quadratic Surrogate model for visual saliency prediction

The Shape-Time Random Field for Semantic Video Labeling

Reconfigurable models for scene recognition

Learning Video Saliency from Human Gaze Using Candidate Selection

Semi-supervised Learning of Geospatial Objects through Multi-modal Data Integration

Multi-task deep visual-semantic embedding for video thumbnail selection