NDLI: Real-time shading-based refinement for consumer depth cameras

Content Provider	ACM Digital Library
Author	Stamminger, Marc Wu, Chenglei Izadi, Shahram Theobalt, Christian Nießner, Matthias Zollhöfer, Michael
Copyright Year	2014
Abstract	We present the first real-time method for refinement of depth data using shape-from-shading in general uncontrolled scenes. Per frame, our real-time algorithm takes raw noisy depth data and an aligned RGB image as input, and approximates the time-varying incident lighting, which is then used for geometry refinement. This leads to dramatically enhanced depth maps at 30Hz. Our algorithm makes few scene assumptions, handling arbitrary scene objects even under motion. To enable this type of real-time depth map enhancement, we contribute a new highly parallel algorithm that reformulates the inverse rendering optimization problem in prior work, allowing us to estimate lighting and shape in a temporally coherent way at video frame-rates. Our optimization problem is minimized using a new regular grid Gauss-Newton solver implemented fully on the GPU. We demonstrate results showing enhanced depth maps, which are comparable to offline methods but are computed orders of magnitude faster, as well as baseline comparisons with online filtering-based methods. We conclude with applications of our higher quality depth maps for improved real-time surface reconstruction and performance capture.
Starting Page	1
Ending Page	10
Page Count	10
File Format	PDF
ISSN	07300301
e-ISSN	15577368
DOI	10.1145/2661229.2661232
Volume Number	33
Issue Number	6
Journal	ACM Transactions on Graphics (TOG)
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2014-11-19
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Depth camera Real-time Shading-based refinement
Content Type	Text
Resource Type	Article
Subject	Computer Graphics and Computer-Aided Design

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Shading-based refinement on volumetric signed distance functions

Real-time expression transfer for facial reenactment

Color map optimization for 3D reconstruction with consumer depth cameras

Depth error correction for projector-camera based consumer depth cameras

Real-time polygonal-light shading with linearly transformed cosines

Bayesian Depth-From-Defocus With Shading Constraints

4D frequency analysis of computational cameras for depth of field extension

Surface flows for image-based shading design

Automated reprojection-based pixel shader optimization

Real-time shading-based refinement for consumer depth cameras

Similar Documents

Shading-based refinement on volumetric signed distance functions

Real-time expression transfer for facial reenactment

Color map optimization for 3D reconstruction with consumer depth cameras

Depth error correction for projector-camera based consumer depth cameras

Real-time polygonal-light shading with linearly transformed cosines

Bayesian Depth-From-Defocus With Shading Constraints

4D frequency analysis of computational cameras for depth of field extension

Surface flows for image-based shading design

Automated reprojection-based pixel shader optimization

Real-time shading-based refinement for consumer depth cameras