NDLI: Barehanded music: real-time hand interaction for virtual piano

Please wait, while we are loading the content...

SSVDAGs: symmetry-aware sparse voxel DAGs

Precision: precomputing environment semantics for contact-rich character animation

Ductile fracture for clustered shape matching

Barehanded music: real-time hand interaction for virtual piano

Accurate analytic approximations for real-time specular area lighting

Real-time rendering of procedural multiscale materials

A framework to transform in-core GPU algorithms to out-of-core algorithms

Exploiting coherence in time-varying voxel data

Dynamically balanced and plausible trajectory planning for human-like characters

Adaptive impulse response modeling for interactive sound propagation

Augmented reality instruction for object assembly based on markerless tracking

Sequential Monte Carlo instant radiosity

A phenomenological scattering model for order-independent transparency

A simulation on grass swaying with dynamic wind force

Triangle reordering for reduced overdraw in animated scenes

Path patterns: analyzing and comparing real and simulated crowds

Interactive modal sound synthesis using generalized proportional damping

Data-driven adaptive history for image editing

Precomputed illuminance composition for real-time global illumination

Beyond hard shadows: moment shadow maps for single scattering, soft shadows and translucent occluders

A stochastic and scalable approach for real-time global illumination

Real-time hair mesh simulation

A multiview and multilayer approach for interactive ray tracing

A technique for matching convergence and accommodation in a fixed screen 3D VR HMD

Adaptive deferred shading

Compact stokes coordinates for cage-based shapes

Disocclusion mitigation for point cloud imposters

Extended TimeWarp latency compensation for virtual reality

Hardware accelerated mesh colors

Natural user interface for integral photography of dispersion-rendered diamond

Pressure-based touch positioning techniques for 3D objects

Progressive photon mapping with sample elimination

Progressive streaming of textured 3D models in a web browser

RGB-D IBR: rendering indoor scenes using sparse RGB-D images with local alignments

Virtually transforming detect: moving a 2-D gamified health assessment to virtual reality

Barehanded music: real-time hand interaction for virtual piano

Content Provider	ACM Digital Library
Author	Liu, Yong-Jin Wang, Jin Liang, Hui He, Ying Luo, Jun Sun, Qian Yuan, Junsong
Abstract	This paper presents an efficient data-driven approach to track fingertip and detect finger tapping for virtual piano using an RGB-D camera. We collect 7200 depth images covering the most common finger articulation for playing piano, and train a random regression forest using depth context features of randomly sampled pixels in training images. In the online tracking stage, we firstly segment the hand from the plane in contact by fusing the information from both color and depth images. Then we use the trained random forest to estimate the 3D position of fingertips and wrist in each frame, and predict finger tapping based on the estimated fingertip motion. Finally, we build a kinematic chain and recover the articulation parameters for each finger. In contrast to the existing hand tracking algorithms that often require hands are in the air and cannot interact with physical objects, our method is designed for hand interaction with planar objects, which is desired for the virtual piano application. Using our prototype system, users can put their hands on a desk, move them sideways and then tap fingers on the desk, like playing a real piano. Preliminary results show that our method can recognize most of the beginner's piano-playing gestures in realtime for soothing rhythms.
Starting Page	87
Ending Page	94
Page Count	8
File Format	PDF MP4
ISBN	9781450340434
DOI	10.1145/2856400.2856411
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-02-27
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Fingertip tracking Rgb-d images Virtual piano Human-computer interaction Finger tapping detection
Content Type	Audio Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in