Finger Detection for Multi-Touch Tabletop Display System 多重觸控桌面顯示系統之手指偵測

1. Finger Detection for Multi-Touch Tabletop Display System多重觸控桌面顯示系統之手指偵測 Su-ting, Chuang 2010/8/2

2. Outline • Introduction • Related Work • System and Method • Experiment • Conclusion & Future Work

3. Outline • Introduction • Related Work • System and Method • Experiments • Conclusion & Future Work

4. Introduction • Non-uniform lighting problem • Various finger touch response among different position • Low computation efficiency • No such tool that helps users determine parameters automatically

5. Outline • Introduction • Related Work • System and Method • Experiments • Conclusion & Future Work

6. Related Work • FTIR (Frustrated Total Internal Reflection) J. Y. Han, “Low-cost multi-touch sensing through frustrated total internal reflection," in Proceedings of the 18th annual ACM symposium on User interface software and technology (UIST '05). New York, NY, USA: ACM Press, 2005, pp. 115-118.

7. Related Work • DI (Diffused Illumination) J. Rekimoto and N. Matsushita, “Perceptual surfaces: Towards a human and object sensitive interactive display," Workshop on Perceptural User Interfaces (PUI'97), 1997.

8. Related Work • TouchLib • A multi-touch development kit • Finger detection processing flow chart Background Subtraction Simple Highpass Scale Threshold Finger Analysis

9. Related Work • DirectShow • Filter-based framework • GShow • GPU-accelerated framework • Combination of DirectX and DirectShow

10. Outline • Introduction • Related Work • System and Method • Experiments • Conclusion & Future Work

11. Hardware Configuration (2) IR Camera (3) IR Illuminator (1) Peripheral Projector

12. Hardware Configuration • Order of diffuser layer and touch-glass layer spot IRcamera IRcamera 4 3 Diffuser layer Touch-glass layer 2 spot 1 IR illuminator IR illuminator IRcamera IRcamera

13. Hardware Configuration • Problem: • IR rays reflected by the touch-glass will result in hot spot regions in camera views • Solution: • Use other cameras to recover the regions which are sheltered by IR spots

14. Software Architecture • Detection system • Image Stitching • FingerDetection • Finger Tracking • Parameter determination Image Stiching Finger Detection Finger Tracking

15. Software Architecture Image Stiching Finger Detection Finger Tracking

16. Image Stitching • Goal • Combine multi-camera view into a virtual camera view

17. Image Stitching • Advantages • Remove IR spot effect • Unify finger size among different position of table • Reduce matching problem • Be compatible with existent finger detection system

18. Image Stitching IR Camera(L) Undistortion HomoWarp Image Blending IR camera(R) Undistortion HomoWarp

19. Image Stitching 5 2 2 5 • HomoWarp 6 1 3 4 1 2 3 6 5 4

20. Image Stitching • Image Blending

21. Finger Detection • TouchLib • Our method Background Subtraction Simple Highpass Scale Binary Finger Analysis Background Subtraction Normalization Difference of Gaussian Binary Finger Analysis

22. Finger Detection • Normalization • Method • Model distribution of IR illumination • Use specific material to simulate foreground • Construct normalization map • Normalize foreground image • Result • Before normalization: mean = 75, standard variation = 30 • After normalization: mean = 255, standard variation = 3

23. Finger Detection • Difference of Gaussian (DoG) • Modified from simple highpass in TouchLib

24. Fingertip Tracking • Goal • Smooth the trajectory of finger • Fix lost results • Method • Kalman filter • Smooth the path • Predict the new state and its uncertainty • Correct the tracker with its new measurement • Assume white noise and uniform velocity Original After Kalman filter

25. Parameter Determination • Requirements of ideal finger detection system • Sensitive  miss↓ • Noise-free  false alarm ↓ • Goal • Find an applicable set of parameters for finger detection system fulfilling the requirements

26. Parameter Determination Touch Data Detection System Test Set Parameter Combination Detection Result Parameters Determinator Applicable set of Parameters Ground Truth (Trace)

27. Parameter Determination • Evaluation of parameters • Data Collection • Depict trace • Measurement • Minimize # of miss and false alarm

28. Parameter Determination • Ideal finger detection • Only one fingertip landing on trace • Continuity among frames

29. Outline • Introduction • Related Work • System and Method • Experiments • Conclusion & Future Work

30. Experiments • Performance evaluation

31. Experiments • Parameter determination • Decide parameters in our system • Adopt sampling-based parameter search technique Background Subtraction Normalization Difference of Gaussian Binary Finger Analysis Subtract value Smooth kernel Threshold Finger Size

32. Experiments • Parameter determination • Exhaustive search • Parameter combination • 5 (step) *5 (step) *5 (step) *5 (step) = 625 • Applicable parameter num • 16/625 = 2.56%

33. Initialize particles Experiments …… Particle generation 1 2 M • Parameter determination • Particle filtering …… Weight Computation 1 2 M Normalize Weights No output Sampling More iteration Yes Over limited iteration Yes Measure Exit No Resampling

34. Outline • Introduction • Related Work • System and Method • Experiments • Conclusion & Future Work