NDLI: 3D Scene Understanding by Voxel-CRF

Content Provider	IEEE Xplore Digital Library
Author	Byung-Soo Kim Kohli, P. Savarese, S.
Copyright Year	2013
Description	Author affiliation: Stanford Univ., Stanford, CA, USA (Savarese, S.) \|\| Microsoft Res., Cambridge, UK (Kohli, P.) \|\| Univ. of Michigan, Ann Arbor, MI, USA (Byung-Soo Kim)
Abstract	Scene understanding is an important yet very challenging problem in computer vision. In the past few years, researchers have taken advantage of the recent diffusion of depth-RGB (RGB-D) cameras to help simplify the problem of inferring scene semantics. However, while the added 3D geometry is certainly useful to segment out objects with different depth values, it also adds complications in that the 3D geometry is often incorrect because of noisy depth measurements and the actual 3D extent of the objects is usually unknown because of occlusions. In this paper we propose a new method that allows us to jointly refine the 3D reconstruction of the scene (raw depth values) while accurately segmenting out the objects or scene elements from the 3D reconstruction. This is achieved by introducing a new model which we called Voxel-CRF. The Voxel-CRF model is based on the idea of constructing a conditional random field over a 3D volume of interest which captures the semantic and 3D geometric relationships among different elements (voxels) of the scene. Such model allows to jointly estimate (1) a dense voxel-based 3D reconstruction and (2) the semantic labels associated with each voxel even in presence of partial occlusions using an approximate yet efficient inference strategy. We evaluated our method on the challenging NYU Depth dataset (Version 1 and 2). Experimental results show that our method achieves competitive accuracy in inferring scene semantics and visually appealing results in improving the quality of the 3D reconstruction. We also demonstrate an interesting application of object removal and scene completion from RGB-D images.
Sponsorship	IEEE Comput. Soc.
Starting Page	1425
Ending Page	1432
File Size	949296
Page Count	8
File Format	PDF
ISBN	9781479928408
ISSN	15505499
DOI	10.1109/ICCV.2013.180
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2013-12-01
Publisher Place	Australia
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Three-dimensional displays Semantics Labeling Solid modeling Cameras Noise measurement Image reconstruction RGB-D Scene understanding 3D reconstruction
Content Type	Text
Resource Type	Article
Subject	Computer Vision and Pattern Recognition Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A Flexible Scene Representation for 3D Reconstruction Using an RGB-D Camera

Mesh Based Semantic Modelling for Indoor and Outdoor Scenes

RGB-D Based Face Reconstruction and Recognition

Holistic 3D scene understanding from a single geo-tagged image

STAR3D: Simultaneous Tracking and Reconstruction of 3D Objects Using RGB-D Data

Holistic Scene Understanding for 3D Object Detection with RGBD Cameras

Manhattan Scene Understanding via XSlit Imaging

3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding

Analysis by Synthesis: 3D Object Recognition by Object Reconstruction

3D Scene Understanding by Voxel-CRF

Similar Documents

A Flexible Scene Representation for 3D Reconstruction Using an RGB-D Camera

Mesh Based Semantic Modelling for Indoor and Outdoor Scenes

RGB-D Based Face Reconstruction and Recognition

Holistic 3D scene understanding from a single geo-tagged image

STAR3D: Simultaneous Tracking and Reconstruction of 3D Objects Using RGB-D Data

Holistic Scene Understanding for 3D Object Detection with RGBD Cameras

Manhattan Scene Understanding via XSlit Imaging

3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding

Analysis by Synthesis: 3D Object Recognition by Object Reconstruction

3D Scene Understanding by Voxel-CRF