1 / 49

Ramesh Raskar Media Lab, MIT Cambridge, MA

Second Skin. Ramesh Raskar Media Lab, MIT Cambridge, MA. Second Skin and RFIG. Second Skin Sensing and Actuation Every mm at every ms Impercetible , natural environment Overcoming the ‘ Dark side of VR’ RFIG Marker technology Marker= RFID + photosensor (not barcode)

anka
Download Presentation

Ramesh Raskar Media Lab, MIT Cambridge, MA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Second Skin Ramesh Raskar Media Lab, MIT Cambridge, MA

  2. Second Skin and RFIG • Second Skin • Sensing and Actuation • Every mm at every ms • Impercetible, natural environment • Overcoming the ‘Dark side of VR’ • RFIG • Marker technology • Marker= RFID + photosensor (not barcode) • Locator= Projector (not camera) • Durable (10+ yrs), long range, imperceptible (zero area) • Think inverse

  3. Vicon Motion Capture Medical Rehabilitation Athlete Analysis Body-worn markers High-speed IR Camera Performance Capture Biomechanical Analysis

  4. ‘Motion’ Capture ? • Building a real time human model • Dense sampling over surface • Geometry with Id at every millisecond at every milimeter • Bio parameters • Getting intimate • Cameras .. • Wearables • Second Skin (Sensor suit) • Tapping inside • Close the loop in bio-I/O • Remote monitoring: Elderly care, training • Robot observation:learning, worker safety • Feedback for biomech/neuro interfaces

  5. Tagged Books in a Library • Id : List of books in RF range • No Precise Location Data • Are books in sorted order ?

  6. RF Tag + Photosensor

  7. Conventional RF tag Photo-sensing RF tag Projector + Photo-sensing RF tag

  8. PatternMSB PatternMSB-1 PatternLSB 0 1 1 0 0 X=12 • For each tag • From light sequence, decode xandycoordinate • Transmit back to RF reader (Id, x, y)

  9. Visual feedback of 2D position • Receive via RF {(Id1,x1,y1), (Id2,x2,y2), …} • Illuminate those positions

  10. Where are products about to expire ? Siggraph 2004

  11. G R F I R F I D (Radio Frequency Id & Geometry) Photosensing Wireless Tags Find tag location usinghandheld Projector Interactive stabilized projection Many geometric ops Siggraph 2004

  12. Mitsubishi Pocket Projector

  13. AR with Photosensing RFID and Handheld Projector

  14. Raskar, vanBaar, Beardsley, Willwacher, Rao, Forlines‘iLamps: Geometrically Aware and Self-Configurable Projectors’, SIGGRAPH 2003

  15. AR Issues • Preprocessing: • Authoring • Runtime: • Identification:Recognition of objects • Using markers and visual tags • Registration: Finding relative pose of display device • Dynamic estimate of translation and rotation • Render/Warp images • Interaction: • Widgets, Gesture recognition, Visual feedback

  16. AR Issues • Preprocessing: • Authoring • Runtime: • Identification:Recognition of objects • Using markers and visual tags • Registration: Finding relative pose of display device • Dynamic estimate of translation and rotation • Render/Warp images • Interaction: • Widgets, Gesture recognition, Visual feedback RFID?

  17. AR Issues • Preprocessing: • Authoring • Runtime: • Identification:Recognition of objects • Using markers and visual tags • Registration: Finding relative pose of display device • Dynamic estimate of translation and rotation • Render/Warp images • Interaction: • Widgets, Gesture recognition, Visual feedback RFID Photosensing RFID Projector for visual feedback

  18. Inside of Projector Condensing Optics Light Source Focusing Optics Gray code Slide The Gray code pattern

  19. Tag

  20. 2D Location 3D Location X2 data X data X data Y data Y data

  21. PatternMSB PatternMSB-1 PatternLSB 0 1 1 0 0 X=12 • For each tag • From light sequence, decode xandycoordinate • Transmit back to RF reader (Id, x, y)

  22. Imperceptible Tags under clothing, tracked under ambient light

  23. Towards Second SkinCoded Illumination Motion Capture Clothing • 500 Hz with Id for each Marker Tag • Capture in Natural Environment • Visually imperceptible tags • Photosensing Tag can be hidden under clothes • Ambient lighting is ok • Unlimited Number of Tags • Light sensitive fabric for dense sampling • Non-imaging, complete privacy • Base station and tags only a few 10’s $ • Full body scan + actions • Elderly, patients, athletes, performers • Breathing, small twists, multiple segments or people • Animation Analysis

  24. Second Skin and RFIG • Second Skin • Sensing and Actuation • Every mm at every ms • Impercetible, natural environment • Overcoming the ‘Dark side of VR’ • RFIG • Marker technology • Marker= RFID + photosensor (not barcode) • Locator= Projector (not camera) • Durable (10+ yrs), long range, imperceptible (zero area) • Think inverse http://raskar.info

  25. Acknowledgements • MERL • Jeroen van Baar, Paul Beardsley, Remo Ziegler, Thomas Willwacher, SrinivasRao, Cliff Forlines, Paul Dietz, Joe Marks, Darren Leigh • Office of the Future group at UNC Chapel Hill • Greg Welch, Kok-lim Low, Deepak B’padhyay, AditiMajumder, Michael Brown, Ruigang Yang • Henry Fuchs, Herman Towles • Wei-chao Chen

  26. END

  27. END

  28. END

  29. The Poor Man’s Palace:Special Effects in the Real World Ramesh Raskar Mitsubishi Electric Research Labs (MERL) Cambridge, MA

  30. Special Effects and Virtual Worlds • Photorealism around us ? • Stays on screens • Does it affect daily life in real time ? • Unlike other fields

  31. Changing Appearance

  32. Changing Virtual Illumination

  33. Special Effects and Virtual Worlds • Photorealism around us ? • Stays on screens • Does it affect daily life in real time ? • Unlike other fields • Fusion: real world with graphics • Next big challenge in CG/Second Life ? • Believable, seamless co-existence

  34. Changing Appearance Virtual light source Projector Projector

  35. Changing Virtual Illumination Raskar, Welch, Low, Bandyopadhyay, “Shader Lamps” (2000)

  36. Preprocessing • Scan 3D object and create model • Roughly align projector(s) • Calibrate by finding pose • Run-time • Render images of 3D model • Warp/Correct

  37. Virtual Motion

  38. Virtual Motion Raskar, Ziegler, Willwacher, “Cartoon Dioramas in Motion,” (NPAR 2002)

  39. Dynamic Augmentation Projecting on Tracked Objects

  40. Radiance Adjustment I d Virtual Real d ( x )2 k ( x ) cos( p ) Ip (x, p) = L ( x, ) , k ( x ) > 0 Pixelintensity Desired radiance Reflectance Intensity correction

  41. ShaderLamps Virtual Reflectance Virtual Illumination Virtual Motion Interaction www.ShaderLamps.com

  42. Poor Man’s Palace

  43. Poor Man’s Palace

  44. Maya: World is an Illusion • Fake World • We all live in one • Social Issues • Real-life Fakes • Not just photos and videos but surroundings • Privacy • X-reality/AR/Virtual Worlds • Delivers years of CG/Sensors/Robotics research into the real world • Time and Space shifting with non-linear distortions

  45. Beyond Gouraud Shading of White Objects • Complex Reflectance • Specular or arbitrary BRDF surfaces • View-dependent appearance • Participating Media • Simulating or in presence of smoke, fog • Complex Geometry • Spaghetti • Motion • Animation of real surfaces • NPR, Distortions, Perceptual factors • Great thesis topics ..

  46. Pieces of the PuzzleActuated Surfaces Actuated Workbench [Pangaro, Maynes-Aminzade, Ishii UIST 2002]

  47. DisplaysContenders Light Emitting Polymers Organic LED E-Ink

  48. Recap • Special Effects in Real World • Photorealism yet to impact daily life • Poor Man’s Palace • Spatial Augmented Reality • Untethered solution for fusion • Geometry, Photometry, Id • Sense, Control, Compensate • Projectors, RFID, Sensors • Open Problems • All senses: haptic, olfactory, proprioception • Natural phenomenon, complex BRDF, other displays • Next Challenge: Photorealistic AR around us

More Related