1 / 13

The Renci Multitouch Framework

The Renci Multitouch Framework. Because touching things is just good fun. Overview. Camera Capture Filtering Touch Detection Touch Camera to Stitching Transformation Tracking Stitching to Screen Transformation Gesture Detection. General Structure. Touch Detection & Tracking.

prince
Download Presentation

The Renci Multitouch Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The RenciMultitouch Framework Because touching things is just good fun.

  2. Overview • Camera Capture • Filtering • Touch Detection • Touch Camera to Stitching Transformation • Tracking • Stitching to Screen Transformation • Gesture Detection

  3. General Structure Touch Detection & Tracking Gesture Engine Acquire New Touches Test against In-Flight Gestures & Cull Cluster Remaining Touches Compute Contraction / Dilation Rotational Offset Clip to registered touch regions Push To Clients or Emit Mouse Events • Camera Stack • Filter Graph • Camera • Background • High Pass • Threshold • Blob Detection • Camera to Stitching Transform • Tracker • Tracking • Stitch to Screen Transform

  4. Camera Capture • Video for Windows • Prosilica • Point Grey • Sony PS3 Eye

  5. Filtering

  6. Blob Detection • Sample Screen Space given minimum blob size • Utilize a flood-fill style approach to blob region detection • Back-fill with black to avoid multiple detections • Compute Centroid & Bounding Box • Projective Transform from Camera to Stitched Coordinate System

  7. Tracking • KD-Tree Nearest Neighbor to track a blob frame to frame • Find set of nearest neighbors from KD-Tree • Search along line from previous blob’s gradient to find nearest blob from given set • Transform from Stitched to Screen Coordinate System

  8. What Are All These Transforms? • Math on all the Pixels Is Expensive • So save time by only doing the expensive mathy-bits on the touches • Optimally we want One Thread per Camera with Filter Graph & Blob Detection & Transform • Basically were mapping quadrilaterals to other quadrilaterals: • Camera System to Stitched System • Stitched System to Screen System

  9. So where do we get the Transformation Matrices? • Calibration and More: • Camera Calibration to remove radial distortion • Stitching Calibration • Screen Calibration • Touch Calibration ( Determine Filter graph Values )

  10. Camera Calibration: There is an App For That

  11. Stitching Calibration: Yep, Another App

  12. Stitching Results on Full Images

  13. Even More Apps: Touch & Screen Calibration

More Related