NDLI: Multimodal Image Retrieval : Fusing modalities with multilayer multimodal pLSA

Please wait, while we are loading the content...

Multimodal Image Retrieval : Fusing modalities with multilayer multimodal pLSA

Content Provider	Springer Nature Link
Author	Lienhart, Rainer Hörster, Eva Romberg, Stefan
Copyright Year	2012
Abstract	In this work, we extend the standard single-layer probabilistic Latent Semantic Analysis (pLSA) (Hofmann in Mach Learn 42(1–2):177–196, 2001) to multiple layers. As multiple layers should naturally handle multiple modalities and a hierarchy of abstractions, we denote this new approach multilayer multimodal probabilistic Latent Semantic Analysis (mm-pLSA). We derive the training and inference rules for the smallest possible non-degenerated mm-pLSA model: a model with two leaf-pLSAs and a single top-level pLSA node merging the two leaf-pLSAs. We evaluate this approach on two pairs of different modalities: SIFT features and image annotations (tags) as well as the combination of SIFT and HOG features. We also propose a fast and strictly stepwise forward procedure to initialize the bottom–up mm-pLSA model, which in turn can then be post-optimized by the general mm-pLSA learning algorithm. The proposed approach is evaluated in a query-by-example retrieval task where various variants of our mm-pLSA system are compared to systems relying on a single modality and other ad-hoc combinations of feature histograms. We further describe possible pitfalls of the mm-pLSA training and analyze the resulting model yielding an intuitive explanation of its behaviour.
Starting Page	31
Ending Page	44
Page Count	14
File Format	PDF
ISSN	21926611
Journal	International Journal of Multimedia Information Retrieval
Volume Number	1
Issue Number	1
e-ISSN	2192662X
Language	English
Publisher	Springer-Verlag
Publisher Date	2012-03-07
Publisher Place	London
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	HOG SIFT Image retrieval Data Mining and Knowledge Discovery Information Storage and Retrieval Multimedia Information Systems Topic models Image annotation Image Processing and Computer Vision Computer Science Information Systems Applications (incl. Internet) Hierarchical pLSA pLSA
Content Type	Text
Resource Type	Article
Subject	Library and Information Sciences Information Systems Media Technology

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Multimodal Image Retrieval

User-adaptive image retrieval via fusing pointwise and pairwise labels

MGraph: multimodal event summarization in social media using topic models and graph-based ranking

Multimodal biomedical image retrieval using hierarchical classification and modality fusion

Interactive cross and multimodal biomedical image retrieval based on automatic region-of-interest (ROI) identification and classification

Top multimedia information retrieval papers

Image recommendation based on keyword relevance using absorbing Markov chain and image features

A sparse kernel relevance model for automatic image annotation

Beyond audio and video retrieval: topic-oriented multimedia summarization

Multimodal Image Retrieval : Fusing modalities with multilayer multimodal pLSA

Similar Documents

Multimodal Image Retrieval

User-adaptive image retrieval via fusing pointwise and pairwise labels

MGraph: multimodal event summarization in social media using topic models and graph-based ranking

Multimodal biomedical image retrieval using hierarchical classification and modality fusion

Interactive cross and multimodal biomedical image retrieval based on automatic region-of-interest (ROI) identification and classification

Top multimedia information retrieval papers

Image recommendation based on keyword relevance using absorbing Markov chain and image features

A sparse kernel relevance model for automatic image annotation

Beyond audio and video retrieval: topic-oriented multimedia summarization

Multimodal Image Retrieval : Fusing modalities with multilayer multimodal pLSA