1 / 28

A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction

A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction. Qunqun Xie, Guoyuan Liang, Cheng Tang, and Xinyu Wu. 2013 10th IEEE International Conference on Control and Automation (ICCA). Outline. Introduction Related Work Proposed Method

enya
Download Presentation

A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction Qunqun Xie, Guoyuan Liang, Cheng Tang, and Xinyu Wu 2013 10th IEEE International Conference on Control and Automation (ICCA)

  2. Outline • Introduction • Related Work • Proposed Method • Hand localization • Fingertips tracking • The Multi-touch system • Experimental Results • Conclusion

  3. Introduction

  4. Introduction • Multi-touch technology: • Sensor Based • Directly receive finger touch as input • High cost → limits its application to some extent • Computer Vision Based • Good scalability as well as good performance Image: Oka, K, Sato, Y, Koike, H. "Real-time fingertip tracking and gesture recognition", IEEE Computer Graphics and Applications, 2012

  5. Introduction • In this paper: • Propose a robust fingertip tracking algorithm: • Real-time • Stereovision-based 3D multi-touch interaction system • Skin / Depth / Geometry structure

  6. Related Work

  7. Related work L. Jin, D. Yang, L. Zhen, and J. Huang. A novel vision based finger-writing character recognition system. Journal of Circuits, Systems, and Computers (JCSC), 16(3):421–436, 2007. • Geometry properties: • Curvature • Edge or shape • Build a model • Image Analysis • Template matching • Color Segmentation D. Lee and S. Lee. Vision-based finger action recognition by angle detection and contour analysis. Electronics and Telecommunications Research Institute Journal, 33(3):415–422, 2011.

  8. Related work • Palm Center: • Fingertip Detection [b] [a] Geodesic distance GSP points Neighbor depth [a] [d] [c]

  9. Related work • [a] Hui Liang, Junsong Yuan, and Daniel Thalmann, "3D Fingertip and Palm Tracking in Depth Image Sequences", Proceedings of the 20th ACM international conference on Multimedia, 2012 • [b]Chia-Ping Chen, Yu-Ting Chen, Ping-Han Lee, Yu-Pao Tsai, and Shawmin Lei, "Real-time Hand Tracking on Depth Images", IEEE Visual Communications and Image Processing (VCIP), 2011 • [c] Ziyong Feng, Shaojie Xu, Xin Zhang, Lianwen Jin, Zhichao Ye, and Weixin Yang, “Real-time Fingertip Tracking and Detection using Kinect Depth Sensor for a New Writing-in-the Air System”, Proceedings of the 4th International Conference on Internet Multimedia Computing and Service, 2012 • [d] Zhichao Ye, Xin Zhang, Lianwen Jin, Ziyong Feng, Shaojie Xu, "FINGER-WRITING-IN-THE-AIR SYSTEM USING KINECT SENSOR", IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2013

  10. ProposedMethod

  11. Hand Segmentation Training data: • Skin Color filter • YCbCr color space • Gaussian Mixture Model • Describe the skin-color distribute • Single Gaussian Model: • Gaussian Mixture Model: Weight of each Gaussian model: color vector

  12. Hand Segmentation • Skin Color filter • : how skin-like the color is • Expectation Maximization(EM) algorithm

  13. Hand Segmentation • Depth Cue: • The points with minimum depth are picked as seeds • Region grow algorithm skin depth skin + depth

  14. Hand Segmentation • Divide wrist and hand: • By a boundary curve [18] • Minimum depth • Boundary curve r: row index c :column index z(r,c) :depth value , range threshold (related to palm size) boundary [18] Z. Mo and U. Neumann, “Real-time hand pose recognition using low-resolution depth images,” in Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, vol. 2.

  15. Palm Region Extraction • Observation : Palm is a rectangle-like region • Method :Project the hand region in all directions

  16. Palm Region Extraction Intersection

  17. Palm Center Localization X > 、、 > >

  18. Palm Center Localization • Palm Center: • The point with maximum distance from the closest palm boundary[18]. • The size of palm R: palm region palm boundary

  19. Fingertip Localization • Fingertip : The point with maximum distance to the palm center (on the contour of each finger) • Candidate set F: P : contour point C0: palm center d2:distance R:palm size 1.2 F

  20. Fingertip Localization • Assign an index to each point in candidate set: • Sort candidate set by • : index • F : candidate set • C0: palm center • the angle of with negative x-axis

  21. Fingertip Localization • Distance between successive points : • If > → Start & End point subset • Fingertips : maximum distance in each subset

  22. Multi-touch system • TUIO (Tangible User Interface Object) [24] M. Kaltenbrunner, T. Bovermann, R. Bencina, and E. Costanza, “Tuio:A protocol for table-top tangible user interfaces,” in Proc. of the The 6th Intl Workshop on Gesture in Human-Computer Interaction and Simulation, 2005.

  23. ExperimentalResults

  24. Experimental Results • Xeon 3.07Ghz workstation • frame rate :20Hz on average(real-time) • Modules • Fingertip tracking • TUIO server • TUIO client [10] C. Shan, Y. Wei, T. Tan, F. Ojardias, ”Real Time Hand Tracking by Combining Particle Filtering and Mean Shift”, In: International Conference on Automatic Face and Gesture Recognition, 2004, pp. 669-674

  25. Conclusion

  26. Conclusion • Fast and robust fingertip tracking • Without pressuring sensing device & extra marks • Hand Segmentation • Depth / Skin • Fingertip Detection • Palm region projection • Palm center distance from the boundary • Fingertip : assign index (angle)

More Related