NDLI: Image Annotation by Propagating Labels from Semantic Neighbourhoods

Content Provider	Springer Nature Link
Author	Verma, Yashaswi Jawahar, C. V.
Copyright Year	2016
Abstract	Automatic image annotation aims at predicting a set of semantic labels for an image. Because of large annotation vocabulary, there exist large variations in the number of images corresponding to different labels (“class-imbalance”). Additionally, due to the limitations of human annotation, several images are not annotated with all the relevant labels (“incomplete-labelling”). These two issues affect the performance of most of the existing image annotation models. In this work, we propose 2-pass k-nearest neighbour (2PKNN) algorithm. It is a two-step variant of the classical k-nearest neighbour algorithm, that tries to address these issues in the image annotation task. The first step of 2PKNN uses “image-to-label” similarities, while the second step uses “image-to-image” similarities, thus combining the benefits of both. We also propose a metric learning framework over 2PKNN. This is done in a large margin set-up by generalizing a well-known (single-label) classification metric learning algorithm for multi-label data. In addition to the features provided by Guillaumin et al. (2009) that are used by almost all the recent image annotation methods, we benchmark using new features that include features extracted from a generic convolutional neural network model and those computed using modern encoding techniques. We also learn linear and kernelized cross-modal embeddings over different feature combinations to reduce semantic gap between visual features and textual labels. Extensive evaluations on four image annotation datasets (Corel-5K, ESP-Game, IAPR-TC12 and MIRFlickr-25K) demonstrate that our method achieves promising results, and establishes a new state-of-the-art on the prevailing image annotation datasets.
Starting Page	126
Ending Page	148
Page Count	23
File Format	PDF
ISSN	09205691
Journal	International Journal of Computer Vision
Volume Number	121
Issue Number	1
e-ISSN	15731405
Language	English
Publisher	Springer US
Publisher Date	2016-07-12
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Image annotation Nearest neighbour Metric learning Cross-media analysis Computer Imaging, Vision, Pattern Recognition and Graphics Artificial Intelligence (incl. Robotics) Image Processing and Computer Vision Pattern Recognition
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence Computer Vision and Pattern Recognition Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Baselines for Image Annotation

Metric Learning for Image Alignment

Iterative Category Discovery via Multiple Kernel Metric Learning

Dense Neighborhoods on Affinity Graph

Joint Inference in Weakly-Annotated Image Datasets via Dense Correspondence

The CLEF 2005 Automatic Medical Image Annotation Task

LabelMe: A Database and Web-Based Tool for Image Annotation

Learning a Distance Metric from Relative Comparisons between Quadruplets of Images

Face Recognition from Caption-Based Supervision

Image Annotation by Propagating Labels from Semantic Neighbourhoods

Similar Documents

Baselines for Image Annotation

Metric Learning for Image Alignment

Iterative Category Discovery via Multiple Kernel Metric Learning

Dense Neighborhoods on Affinity Graph

Joint Inference in Weakly-Annotated Image Datasets via Dense Correspondence

The CLEF 2005 Automatic Medical Image Annotation Task

LabelMe: A Database and Web-Based Tool for Image Annotation

Learning a Distance Metric from Relative Comparisons between Quadruplets of Images

Face Recognition from Caption-Based Supervision

Image Annotation by Propagating Labels from Semantic Neighbourhoods