Virtual Rear Projection: Technology and Usability

1. Virtual Rear Projection:Technology and Usability Jay Summet and Matthew Flagg Prof. Jim Rehg Prof. Gregory Abowd GVU Center College of Computing Georgia Institute of Technology

2. Collaborators • Nanyang Technological University, Singapore • Prof. Tat-Jen Cham • Intel Research & Carnegie Mellon University, Pittsburgh, PA • Prof. Rahul Sukthankar • Georgia Tech, Atlanta GA • Prof. Greg Corso (Psychology) • Ramswaroop Somani

3. Every flat surface can be an interactive display Ubiquitous Interactive Displays

4. Large Displays • Large means wall sized! • Traditionanly implemented with rear projection • Other potential technologies: • Electronic Ink • Rollable LCD's • Super Plasma • Nanotech Paint

5. Rear Projection

6. Advantages • No Shadows • High Quality Display • Disadvantages • Installation & Space costs • Average office space costs $77 sq/ft • 10’x3’ = $2310 • Space Limitations – Difficulty in Retrofitting Rear Projection (Continued)

7. Bad for interactive displays due to shadows. Front Projection

8. Warped Front Projection Reduces opportunities for occlusion and severity of shadows.

9. Solution: Warp image by P-1 Keystone Correction Projected image Camera view Audience view Sukthankar et. al., ICCV01 T: projector-camera homography C: screen-camera homography P = C-1T Warp image by P-1 Screen P C T Projector Camera

10. (Passive) Virtual Rear ProjectionMultiple projectors “fill-in” occluded regions at a lower contrast level.

11. Four Projection Techniques

12. Preliminary Usability Study • Tasks • “Drag and drop” • Selection • Curve tracing • Do occlusions affect performance? • Can it replace RP? • User preference • 17 + 10 participants

13. Preliminary Usability Study • We did NOT find a significant performance effect from occlusions/shadows. • Example Trends: • FP: 1.25 sec • WFP: 1.12 sec • PVRP: 1.15 sec • RP: 1.07 sec • Users were able to cope with shadows by modifying their behavior • However, they prefered Rear Projection & Virtual Rear Projection

14. Front Projection Coping Behavior: Edge of Screen

15. Front Projection Coping Behavior: Near Center Participants would stand near the center of the screen and “sway” when unable to find the box.(No photo release.)

16. Front Projection Coping Behavior: Move on Occlusion

17. Passive Virtual Rear Projection Essentially eliminated occlusions. Eliminated coping behavior. Preferred by users (over FP,WFP) (p <= 0.05). Slight degradation in image quality. Results Projected Light is Annoying! Warped Front Projection • Reduced occlusions by 62 percent. • Reduced coping behavior significantly. • Retained image quality compared to front projection.

18. Half shadows Shadow Elimination Double Projector Case Passive VRP

19. Shadow Elimination Boosting projector outputs Sukthankar et. al., CVPR01

20. Occluder Light Suppression Detecting occluded pixels Cham et. al., CVPR03

21. Occluder Light Suppression Detecting occluded pixels Cham et. al., CVPR03

22. Occluder Light Suppression Detecting occluded pixels Cham et. al., CVPR03

23. Occluder Light Suppression Detecting occluded pixels Cham et. al., CVPR03

24. Switching Approach Pixels “owned” by one projector at a time

25. Play video Switching Approach

26. Distracting seams caused by pixel overlaps and gaps Implementation Issues: Seams

27. Projector contribution controlled with alpha masks • Filter masks before compositing and projecting • Median filtering and blurring on GPU Implementation Issues: Seams

28. Uneven brightness is a big issue because projectors are at divergent angles • Construct Luminance Attenuation Maps in photometric calibration stage Implementation Issues:Photometric Calibration Projector 1 Mask Projector 2 Mask

29. Challenges • Geometric calibration for alignment • Calibration for photometric uniformity • Seam removal Implementation Issues:Checkerboard Ownership Case Desired display: flat white Projector 1 Projector 2

30. Show Video Implementation Issues:Difficult Checkerboard Case

31. Conclusions • Demonstrated first active VRP system that performs at interactive rates (8-10 FPS) • Addressed photometric uniformity for oblique projection using attenuation maps • Developed GPU-based solution for seam blending

32. Future Work • Occlusion detection using voting method with multiple cameras • Current system limited by camera placement • Dynamic display content • Requires output prediction • Investigate a predictive rather than reactive approach • Compensate for shadows before they happen

33. BigBoard

34. Future Work (Cont.) • Developing an electronic whiteboard using VRP on the BigBoard • Evaluation of the suitability of VRP for a “real” application (electronic whiteboard) • Collaboration with others wanting to develop for large scale interactive surfaces

35. Application: Interactive Surfaces for Classrooms • Existing classrooms are commonly retrofitted with projected output • In the future, Smartboard DViT technology allows interactive surfaces to be installed • Many technical solutions are possible (e.g. standard rear projection) • Space and cost concerns make front projection very attractive

36. Conclusion • Passive VRP is better than front projection, but users still prefer rear projected displays • Active VRP may be able to achieve higher user satisfaction than passive systems • VRP can enable the deployment of large interactive surfaces in spaces inhospitable to rear projected solutions.