a fast and robust fingertips tracking algorithm for vision based multi touch interaction
Download
Skip this Video
Download Presentation
A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction

Loading in 2 Seconds...

play fullscreen
1 / 28

A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction - PowerPoint PPT Presentation


  • 193 Views
  • Uploaded on

A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction. Qunqun Xie, Guoyuan Liang, Cheng Tang, and Xinyu Wu. 2013 10th IEEE International Conference on Control and Automation (ICCA). Outline. Introduction Related Work Proposed Method

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction' - enya


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
a fast and robust fingertips tracking algorithm for vision based multi touch interaction

A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction

Qunqun Xie, Guoyuan Liang,

Cheng Tang, and Xinyu Wu

2013 10th IEEE

International Conference on Control and Automation (ICCA)

outline
Outline
  • Introduction
  • Related Work
  • Proposed Method
    • Hand localization
    • Fingertips tracking
    • The Multi-touch system
  • Experimental Results
  • Conclusion
introduction1
Introduction
  • Multi-touch technology:
    • Sensor Based
      • Directly receive finger touch as input
      • High cost → limits its application to some extent
    • Computer Vision Based
      • Good scalability as well as good performance

Image: Oka, K, Sato, Y, Koike, H. "Real-time fingertip tracking and gesture recognition", IEEE Computer Graphics and Applications, 2012

introduction2
Introduction
  • In this paper:
    • Propose a robust fingertip tracking algorithm:
      • Real-time
      • Stereovision-based 3D multi-touch interaction system
      • Skin / Depth / Geometry structure
related work1
Related work

L. Jin, D. Yang, L. Zhen, and J. Huang. A novel vision based finger-writing character recognition system. Journal of Circuits, Systems, and Computers (JCSC), 16(3):421–436, 2007.

  • Geometry properties:
    • Curvature
    • Edge or shape
    • Build a model
  • Image Analysis
    • Template matching
    • Color Segmentation

D. Lee and S. Lee. Vision-based finger action recognition by angle detection and contour analysis. Electronics and Telecommunications Research Institute Journal, 33(3):415–422, 2011.

related work2
Related work
  • Palm Center:
  • Fingertip Detection

[b]

[a]

Geodesic

distance

GSP points

Neighbor

depth

[a]

[d]

[c]

related work3
Related work
  • [a] Hui Liang, Junsong Yuan, and Daniel Thalmann, "3D Fingertip and Palm Tracking in Depth Image Sequences", Proceedings of the 20th ACM international conference on Multimedia, 2012
  • [b]Chia-Ping Chen, Yu-Ting Chen, Ping-Han Lee, Yu-Pao Tsai, and Shawmin Lei, "Real-time Hand Tracking on Depth Images", IEEE Visual Communications and Image Processing (VCIP), 2011
  • [c] Ziyong Feng, Shaojie Xu, Xin Zhang, Lianwen Jin, Zhichao Ye, and Weixin Yang, “Real-time Fingertip Tracking and Detection using Kinect Depth Sensor for a New Writing-in-the Air System”, Proceedings of the 4th International Conference on Internet Multimedia Computing and Service, 2012
  • [d] Zhichao Ye, Xin Zhang, Lianwen Jin, Ziyong Feng, Shaojie Xu, "FINGER-WRITING-IN-THE-AIR SYSTEM USING KINECT SENSOR", IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2013
hand segmentation
Hand Segmentation

Training data:

  • Skin Color filter
    • YCbCr color space
    • Gaussian Mixture Model
      • Describe the skin-color distribute
      • Single Gaussian Model:
      • Gaussian Mixture Model:

Weight of each Gaussian model:

color vector

hand segmentation1
Hand Segmentation
  • Skin Color filter
    • : how skin-like the color is
    • Expectation Maximization(EM) algorithm
hand segmentation2
Hand Segmentation
  • Depth Cue:
    • The points with minimum depth are picked as seeds
    • Region grow algorithm

skin

depth

skin + depth

hand segmentation3
Hand Segmentation
  • Divide wrist and hand:
    • By a boundary curve [18]
      • Minimum depth
      • Boundary curve

r: row index

c :column index

z(r,c) :depth value

,

range threshold

(related to palm size)

boundary

[18] Z. Mo and U. Neumann, “Real-time hand pose recognition using low-resolution depth images,” in Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, vol. 2.

palm region extraction
Palm Region Extraction
  • Observation : Palm is a rectangle-like region
  • Method :Project the hand region in all directions
palm center localization1
Palm Center Localization
  • Palm Center:
    • The point with maximum distance from the closest palm boundary[18].
  • The size of palm R:

palm region

palm boundary

fingertip localization
Fingertip Localization
  • Fingertip :

The point with maximum distance to the palm center

(on the contour of each finger)

  • Candidate set F:

P : contour point

C0: palm center

d2:distance

R:palm size

1.2

F

fingertip localization1
Fingertip Localization
  • Assign an index to each point in candidate set:
  • Sort candidate set by
  • : index
  • F : candidate set
  • C0: palm center
    • the angle of with negative x-axis
fingertip localization2
Fingertip Localization
  • Distance between successive points :
  • If > → Start & End point subset
  • Fingertips : maximum distance in each subset
multi touch system
Multi-touch system
  • TUIO (Tangible User Interface Object)

[24] M. Kaltenbrunner, T. Bovermann, R. Bencina, and E. Costanza, “Tuio:A protocol for table-top tangible user interfaces,” in Proc. of the The 6th Intl Workshop on Gesture in Human-Computer Interaction and Simulation, 2005.

experimental results1
Experimental Results
  • Xeon 3.07Ghz workstation
  • frame rate :20Hz on average(real-time)
  • Modules
    • Fingertip tracking
    • TUIO server
    • TUIO client

[10] C. Shan, Y. Wei, T. Tan, F. Ojardias, ”Real Time Hand Tracking by Combining Particle Filtering and Mean Shift”, In: International Conference on Automatic Face and Gesture Recognition, 2004, pp. 669-674

conclusion1
Conclusion
  • Fast and robust fingertip tracking
  • Without pressuring sensing device & extra marks
  • Hand Segmentation
    • Depth / Skin
  • Fingertip Detection
    • Palm region projection
    • Palm center distance from the boundary
    • Fingertip : assign index (angle)
ad