NDLI: Example-based cross-modal denoising

Content Provider	IEEE Xplore Digital Library
Author	Segev, D. Schechner, Y.Y. Elad, M.
Copyright Year	2012
Description	Author affiliation: Dept. Computer Science, Technion - Israel Inst. Technology, Haifa 32000, Israel (Elad, M.) \|\| Dept. Electrical Engineering, Technion - Israel Inst. Technology, Haifa 32000, Israel (Segev, D.; Schechner, Y.Y.)
Abstract	Widespread current cameras are part of multisensory systems with an integrated computer (smartphones). Computer vision thus starts evolving to cross-modal sensing, where vision and other sensors cooperate. This exists in humans and animals, reflecting nature, where visual events are often accompanied with sounds. Can vision assist in denoising another modality? As a case study, we demonstrate this principle by using video to denoise audio. Unimodal (audio-only) denoising is very difficult when the noise source is non-stationary, complex (e.g., another speaker or music in the background), strong and not individually accessible in any modality (unseen). Cross-modal association can help: a clear video can direct the audio estimator. We show this using an example-based approach. A training movie having clear audio provides cross-modal examples. In testing, cross-modal input segments having noisy audio rely on the examples for denoising. The video channel drives the search for relevant training examples. We demonstrate this in speech and music experiments.
Starting Page	486
Ending Page	493
File Size	625344
Page Count	8
File Format	PDF
ISBN	9781467312264
ISSN	10636919
e-ISBN	9781467312288
e-ISBN	9781467312271
DOI	10.1109/CVPR.2012.6247712
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2012-06-16
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Visualization Vectors Noise Noise reduction Training Feature extraction Speech
Content Type	Text
Resource Type	Article
Subject	Computer Vision and Pattern Recognition Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Action Recognition with Actons

Anytime Recognition of Objects and Scenes

Linked Source and Target Domain Subspace Feature Transfer Learning -- Exemplified by Speech Emotion Recognition

Cross-View Action Recognition via a Continuous Virtual Path

Rate-Invariant Analysis of Trajectories on Riemannian Manifolds with Application in Visual Speech Recognition

Modeling gender information for emotion recognition using Denoising autoencoder

Improving generation performance of speech emotion recognition by denoising autoencoders

Coevolutionary Computation for Synthesis of Recognition Systems

Zero-Shot Event Detection Using Multi-modal Fusion of Weakly Supervised Concepts

Example-based cross-modal denoising

Similar Documents

Action Recognition with Actons

Anytime Recognition of Objects and Scenes

Linked Source and Target Domain Subspace Feature Transfer Learning -- Exemplified by Speech Emotion Recognition

Cross-View Action Recognition via a Continuous Virtual Path

Rate-Invariant Analysis of Trajectories on Riemannian Manifolds with Application in Visual Speech Recognition

Modeling gender information for emotion recognition using Denoising autoencoder

Improving generation performance of speech emotion recognition by denoising autoencoders

Coevolutionary Computation for Synthesis of Recognition Systems

Zero-Shot Event Detection Using Multi-modal Fusion of Weakly Supervised Concepts

Example-based cross-modal denoising