NDLI: Musical Instrument Recognition in User-generated Videos using a Multimodal Convolutional Neural Network Architecture

Please wait, while we are loading the content...

Searching for A Thing

Making a Cultural Visit with a Smart Mate

Video Indexing, Search, Detection, and Description with Focus on TRECVID

Generating Video Descriptions with Topic Guidance

Transductive Visual-Semantic Embedding for Zero-shot Learning

Insiders and Outsiders: Comparing Urban Impressions between Population Groups

The Geo-Privacy Bonus of Popular Photo Enhancements

An Unsupervised Distance Learning Framework for Multimedia Retrieval

TEX-Nets: Binary Patterns Encoded Convolutional Neural Networks for Texture Recognition

Accelerated Nearest Neighbor Search with Quick ADC

On the Automatic Identification of Music for Common Activities

Musical Instrument Recognition in User-generated Videos using a Multimodal Convolutional Neural Network Architecture

Multimodal Analysis of Image Search Intent: Intent Recognition in Image Search from User Behavior and Visual Content

On the Selection of Anchors and Targets for Video Hyperlinking

How to Make an Image More Memorable?: A Deep Style Transfer Approach

Leveraging Semantic Facets for Adaptive Ranking of Social Comments

Panorama to Panorama Matching for Location Recognition

Expo: An Expectation-oriented System for Selecting Important Photos from Personal Collections

A Generic Framework for Social Event Analysis

With 5G Approaching, How will Audio/Video Technology that Serves 800 Million QQ Users Bring Forth New Ideas

Estimating the Information Gap between Textual and Visual Representations

3D Facial Video Retrieval and Management for Decision Support in Speech and Language Therapy

Deep Sentiment Features of Context and Faces for Affective Video Analysis

Light Curve Analysis From Kepler Spacecraft Collected Data

ClusterTag: Interactive Visualization, Clustering and Tagging Tool for Big Image Collections

DeepHash for Image Instance Retrieval: Getting Regularization, Depth and Fine-Tuning Right

Improving Small Object Proposals for Company Logo Detection

Improving Context-Aware Music Recommender Systems: Beyond the Pre-filtering Approach

A Spatio-Temporal Category Representation for Brand Popularity Prediction

Nested Invariance Pooling and RBM Hashing for Image Instance Retrieval

Visual Descriptors in Methods for Video Hyperlinking

Cross-modal Image-Graphics Retrieval by Neural Transfer Learning

Multi-task Deep Neural Network for Joint Face Recognition and Facial Attribute Prediction

Concept Language Models and Event-based Concept Number Selection for Zero-example Event Detection

Multimodal Video Retrieval with the 2017 IMOTION System

Semi-Automatic Retrieval of Relevant Segments from Laparoscopic Surgery Videos

Information Retrieval from Multi-Sensor Data for Enriching Location Services at HERE Technologies

MSRC: Multimodal Spatial Regression with Semantic Context for Phrase Grounding

Music-Guided Video Summarization using Quadratic Assignments

Frame-Transformer Emotion Classification Network

Health Multimedia: Lifestyle Recommendations Based on Diverse Observations

Scalable Hadoop-Based Pooled Time Series of Big Video Data from the Deep Web

Balanced Search Space Partitioning for Distributed Media Redundant Indexing

Discrete Multi-view Hashing for Effective Image Retrieval

Groups Re-identification with Temporal Context

Bridging the Aesthetic Gap: The Wild Beauty of Web Imagery

Embedding Watermarks into Deep Neural Networks

Linking Multimedia Content for Efficient News Browsing

DRAW: Deep Networks for Recognizing Styles of Artists Who Illustrate Children's Books

Finger Vein Image Retrieval via Coding Scale-varied Superpixel Feature

Tiny Transform Net for Mobile Image Stylization

The JORD System: Linking Sky and Social Multimedia Data to Natural Disasters

On the Effectiveness of Distance Measures for Similarity Search in Multi-Variate Sensory Data: Effectiveness of Distance Measures for Similarity Search

Intelligently Connecting People with Information

Leveraging Multi-modal Prior Knowledge for Large-scale Concept Learning in Noisy Web Data

PACE: Prediction-based Annotation for Crowded Environments

Deep Supervised Hashing for Multi-Label and Large-Scale Image Retrieval

Quadruplet Networks for Sketch-Based Image Retrieval

Simple, Efficient and Effective Encodings of Local Deep Features for Video Action Recognition

Learning to Detect Misleading Content on Twitter

Multi-view Manifold Learning for Media Interestingness Prediction

AMECON: Abstract Meta-Concept Features for Text-Illustration

Joint Saliency Estimation and Matching using Image Regions for Geo-Localization of Online Video

Query and Keyframe Representations for Ad-hoc Video Search

LireSolr: A Visual Information Retrieval Server

Classification of sMRI for Alzheimer's disease Diagnosis with CNN: Single Siamese Networks with 2D+? Approach and Fusion on ADNI

Utilising High-Level Features in Summarisation of Academic Presentations

Manga FaceNet: Face Detection in Manga based on Deep Neural Network

VideoAnalysis4ALL: An On-line Tool for the Automatic Fragmentation and Concept-based Annotation, and the Interactive Exploration of Videos

Generative Adversarial Networks for Multimodal Representation Learning in Video Hyperlinking

Visually Browsing Millions of Images Using Image Graphs

Efficient Indexing of Regional Maximum Activations of Convolutions using Full-Text Search Engines

Family Photo Recognition via Multiple Instance Learning

TaiChi: A Fine-Grained Action Recognition Dataset

Conditional Fast Style Transfer Network

Improving Image Classification using Coarse and Fine Labels

Fast Multi-Modal Unified Sparse Representation Learning

Badminton Video Analysis based on Spatiotemporal and Stroke Features

Musical Instrument Recognition in User-generated Videos using a Multimodal Convolutional Neural Network Architecture

Content Provider	ACM Digital Library
Author	Gómez, Emilia Haro, Gloria Slizovskaia, Olga
Abstract	This paper presents a method for recognizing musical instruments in user-generated videos. Musical instrument recognition from music signals is a well-known task in the music information retrieval (MIR) field, where current approaches rely on the analysis of the good-quality audio material. This work addresses a real-world scenario with several research challenges, i.e. the analysis of user-generated videos that are varied in terms of recording conditions and quality and may contain multiple instruments sounding simultaneously and background noise. Our approach does not only focus on the analysis of audio information, but we exploit the multimodal information embedded in the audio and visual domains. In order to do so, we develop a Convolutional Neural Network (CNN) architecture which combines learned representations from both modalities at a late fusion stage. Our approach is trained and evaluated on two large-scale video datasets: YouTube-8M and FCVID. The proposed architectures demonstrate state-of-the-art results in audio and video object recognition, provide additional robustness to missing modalities, and remains computationally cheap to train.
Starting Page	226
Ending Page	232
Page Count	7
File Format	PDF
ISBN	9781450347013
DOI	10.1145/3078971.3079002
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2017-06-06
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Multimodal musical instrument classification Multimedia information retrieval Feature fusion Multimodal video analysis Convolutional neural networks
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in