NDLI: Learning to Annotate Clothes in Everyday Photos: Multi-modal, Multi-label, Multi-instance Approach

Content Provider	IEEE Xplore Digital Library
Author	Veloso, A.A. Dos Santos, J.A. Nogueira, K.
Copyright Year	2014
Description	Author affiliation: Dept. of Comput. Sci., Univ. Fed. de Minas Gerais, Belo Horizonte, Brazil (Veloso, A.A.; Dos Santos, J.A.; Nogueira, K.)
Abstract	In this paper, we present an effective algorithm to automatically annotate clothes in everyday photos posted in online social networks, such as Facebook and Instagram. Specifically, clothing annotation can be informally stated as predicting, as accurately as possible, the garment items appearing in the target photo. This task not only poses interesting challenges for existing vision and recognition algorithms, but also brings huge opportunities for recommender and e-commerce systems. We formulate the annotation task as a multi-modal, multi-label and multi-instance classification problem: (i) both image and textual content (i.e., comments about the image) are available for learning classifiers, (ii) the classifiers must predict a set of labels (i.e., a set of garment items), and (iii) the decision on which labels to predict comes from a bag of instances that are used to build a function, which separates labels that should be predicted from those that should not be. Under this setting, we propose a classification algorithm which employs association rules in order to build a prediction model that combines image and textual information, and adopts an entropy-minimization strategy in order to the find the best set of labels to predict. We conducted a systematic evaluation of the proposed algorithm using everyday photos collected from two major fashion-related social networks, namely pose.com and chictopia.com. Our results show that the proposed algorithm provides improvements when compared to popular first choice multi-label algorithms that range from 2% to 40% in terms of accuracy.
Starting Page	327
Ending Page	334
File Size	323442
Page Count	8
File Format	PDF
ISBN	9781479942589
ISSN	15301834
DOI	10.1109/SIBGRAPI.2014.37
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-08-26
Publisher Place	Brazil
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Clothing Prediction algorithms Entropy Visualization Image color analysis Social network services Feature extraction machine learning Automatic annotation multi-label classification
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

An empirical study of automatic image annotation through Multi-Instance Multi-Label Learning

Effective multi-modal multi-label learning for automatic image annotation

Automatic Image Annotation and Retrieval using Multi-Instance Multi-Label Learning

Image Annotation Using the SimpleDecisionTree

Multi-modal Multi-label Semantic Indexing of Images Using Unlabeled Data

A novel semi-supervised Multi-Instance learning approach for scene recognition

Multi-modal Multi-label Semantic Indexing of Images using Unlabeled

Multi-Modal Image Annotation with Multi-Instance Multi-Label LDA ∗

Multi-Label Learning Based on Label Entropy Guided Clustering

Learning to Annotate Clothes in Everyday Photos: Multi-modal, Multi-label, Multi-instance Approach

Similar Documents

An empirical study of automatic image annotation through Multi-Instance Multi-Label Learning

Effective multi-modal multi-label learning for automatic image annotation

Automatic Image Annotation and Retrieval using Multi-Instance Multi-Label Learning

Image Annotation Using the SimpleDecisionTree

Multi-modal Multi-label Semantic Indexing of Images Using Unlabeled Data

A novel semi-supervised Multi-Instance learning approach for scene recognition

Multi-modal Multi-label Semantic Indexing of Images using Unlabeled

Multi-Modal Image Annotation with Multi-Instance Multi-Label LDA ∗

Multi-Label Learning Based on Label Entropy Guided Clustering

Learning to Annotate Clothes in Everyday Photos: Multi-modal, Multi-label, Multi-instance Approach