NDLI: Multimodal integration learning of object manipulation behaviors using deep neural networks

Content Provider	IEEE Xplore Digital Library
Author	Noda, K. Arie, H. Suga, Y. Ogata, T.
Copyright Year	2013
Description	Author affiliation: Grad. Sch. of Fundamental Sci. & Eng., Waseda Univ., Tokyo, Japan (Noda, K.; Arie, H.; Suga, Y.; Ogata, T.)
Abstract	This paper presents a novel computational approach for modeling and generating multiple object manipulation behaviors by a humanoid robot. The contribution of this paper is that deep learning methods are applied not only for multimodal sensor fusion but also for sensory-motor coordination. More specifically, a time-delay deep neural network is applied for modeling multiple behavior patterns represented with multi-dimensional visuomotor temporal sequences. By using the efficient training performance of Hessian-free optimization, the proposed mechanism successfully models six different object manipulation behaviors in a single network. The generalization capability of the learning mechanism enables the acquired model to perform the functions of cross-modal memory retrieval and temporal sequence prediction. The experimental results show that the motion patterns for object manipulation behaviors are successfully generated from the corresponding image sequence, and vice versa. Moreover, the temporal sequence prediction enables the robot to interactively switch multiple behaviors in accordance with changes in the displayed objects.
Sponsorship	IEEE Ind. Electron. Soc.
Starting Page	1728
Ending Page	1733
File Size	1857616
Page Count	6
File Format	PDF
ISSN	21530858
DOI	10.1109/IROS.2013.6696582
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2013-11-03
Publisher Place	Japan
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Vectors Joints Robot sensing systems Image reconstruction Neural networks Image sequences
Content Type	Text
Resource Type	Article
Subject	Control and Systems Engineering Computer Science Applications Computer Vision and Pattern Recognition Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Intersensory Causality Modeling Using Deep Neural Networks

Multimodal concept and word learning using phoneme sequences with errors

Learning robot gait stability using neural networks as sensory feedback function for Central Pattern Generators

A comparison of encoding schemes for haptic object recognition using a biologically plausible spiking neural network

Exploiting eye-hand coordination: A novel approach to remote manipulation

Object recognition using neural networks and surface signatures

A new feedback robot control method based on position/image sensor integration

Snake robot shape sensing using micro-inertial sensors

Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition

Multimodal integration learning of object manipulation behaviors using deep neural networks

Similar Documents

Intersensory Causality Modeling Using Deep Neural Networks

Multimodal concept and word learning using phoneme sequences with errors

Learning robot gait stability using neural networks as sensory feedback function for Central Pattern Generators

A comparison of encoding schemes for haptic object recognition using a biologically plausible spiking neural network

Exploiting eye-hand coordination: A novel approach to remote manipulation

Object recognition using neural networks and surface signatures

A new feedback robot control method based on position/image sensor integration

Snake robot shape sensing using micro-inertial sensors

Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition

Multimodal integration learning of object manipulation behaviors using deep neural networks