NDLI: Going beyond free viewpoint: creating animatable volumetric video of human performances

Content Provider	IET Digital Library
Author	Hilsmann, Anna Fechteler, Philipp Morgenstern, Wieland Paier, Wolfgang Feldmann, Ingo Schreer, Oliver Eisert, Peter
Abstract	An end-to-end pipeline for the creation of high-quality animatable volumetric video of human performances is presented. Going beyond the application of free-viewpoint video, the authors allow re-animation and alteration of an actor's performance through the enrichment of the captured data with semantics and animation properties. Hybrid geometry- and video-based animation methods are applied that allow a direct animation of the high-quality data itself instead of creating a CG model that resembles the captured data. Semantic enrichment and animation are achieved by establishing temporal consistency followed by automatic rigging of each 3D frame using a parametric human body model. The hybrid approach combines the flexibility of classical CG animation with the realism of real captured data. For the face, coarse movements are modelled in the geometry only, while very fine and subtle details, often lacking in purely geometric methods, are captured in video textures, which can interactively be combined to form new facial expressions. On top of that, regions that are challenging to synthesise, such as the teeth or the eyes, are learned and filled in realistically in an autoencoder-based approach. This study covers the full pipeline from capturing, volumetric video production, and enrichment with semantics for the final hybrid animation.
Starting Page	350
Ending Page	358
Page Count	9
ISSN	17519632
Volume Number	14
e-ISSN	17519640
Issue Number	Issue 6, Sep (2020)
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/iet-cvi/14/6
Alternate Webpage(s)	https://digital-library.theiet.org/content/journals/10.1049/iet-cvi.2019.0786
Journal	IET Computer Vision
Publisher	The Institution of Engineering and Technology
Publisher Date	2020-04-15
Access Restriction	Open
Rights License	Creative Commons Attribution -Non Commercial License (http://creativecommons.org/licenses/by-nc/3.0/)
Subject Keyword	3D Data Actor Performance Animation Property Autoencoder-based Approach Automatic Rigging Classical CG Animation Computational Geometry Computer Animation Computer Vision And Image Processing Technique Deformation Property Facial Expressions Free-viewpoint Volumetric Video Geometric Animation Ability Graphics Technique High-quality Animatable Volumetric Video Content High-quality Data Human Performances Hybrid Animation Hybrid Geometry Image Texture Neural Computing Technique Neural Nets Optical, Image And Video Signal Processing Parametric Shape Adaptive Full Human Body Model Pose Editing Semantic Enrichment Temporal Consistency Video Signal Processing Video-based Animation Approach Video-based Textures
Content Type	Text
Resource Type	Article
Subject	Computer Vision and Pattern Recognition Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Going beyond Free Viewpoint: Creating Animatable Volumetric Video of Human Performances

Interactive facial animation with deep neural networks

FREE-VIEWPOINT VIDEO FOR INTERACTIVE CHARACTER ANIMATION

Visual tracking via ensemble autoencoder

Video-based character animation (2005).

Physics-based dynamic texture analysis and synthesis model using GPU

Multi-cue combination network for action-based video classification

Context-driven hybrid image inpainting

Structured entropy of primitive: big data-based stereoscopic image quality assessment

Going beyond free viewpoint: creating animatable volumetric video of human performances

Similar Documents

Going beyond Free Viewpoint: Creating Animatable Volumetric Video of Human Performances

Interactive facial animation with deep neural networks

FREE-VIEWPOINT VIDEO FOR INTERACTIVE CHARACTER ANIMATION

Visual tracking via ensemble autoencoder

Video-based character animation (2005).

Physics-based dynamic texture analysis and synthesis model using GPU

Multi-cue combination network for action-based video classification

Context-driven hybrid image inpainting

Structured entropy of primitive: big data-based stereoscopic image quality assessment

Going beyond free viewpoint: creating animatable volumetric video of human performances