A PDA-based See-through Interface within an Immersive Environment

1. A PDA-based See-through Interface within an Immersive Environment Miranda Miranda Miguel*, Ogawa Takefumi**, Kiyokawa Kiyoshi*, Takemura Haruo* *Graduate School of Information Science and Technology, Osaka University **Information Technology Center, Tokyo University

2. Outline • Problem we are working on • Our approach • Previous studies • Description of a prototype • Pilot study • Discussion and results • Future work

3. Interaction within immersive virtual environments. How to select and manipulate virtual objects in a CAVE? http://www.es.jamstec.go.jp/esc/research/Perception/vr.en.html

4. Interfaces using a beam • Trackers are manipulated with 6 DOF • 3D objects are prone to occlusions • Manipulation does not provide feedback information (touch) http://www.jvrb.org/articles/34/

5. Problems • Interaction in a virtual environment is difficult. • Unstable spatial manipulation • No physical contact • Ambiguous depth perception • Unreadable text information • (Generally) Low resolution • Virtual widgets are not necessarily easy to manipulate • No physical contact • (Sometimes) Weak affordance

6. Previous studies (1/2) • Toolglass and Magic Lenses[1] • 3D Magic Lenses[2] [1] Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton D. DeRose (1993) [2] J. Viega, M. J. Conway, G. Williams, R. Pausch (1996)

7. Previous studies (2/2) • Image Plane Interaction Technique[3] [3] J. S. Pierce, A. Forsberg, M. J. Conway, S. Hong, R. Zeleznik, M. R. Mine. (1997)

8. Our approach • Employ a mobile device for interaction in an immersive environment

9. Advantages of our approach (1/2) • Built-in controls • A PDA is equipped with integrated controls that afford the interaction with the environment • Built-in display • The displayed image can be manipulated, processed and even stored for later reference

10. Advantages of our approach (2/2) • Physical contact • The screen surface can be touched directly • The interface is contained on the PDA • Programmable • GUI / button functions are changeable • Examples: • Through the lens tool • Volumetric manipulation tool

11. Architecture of the system Tracking System Rendering server - Render the image for the PDA compress decompress Jpeg compressed Image. UDP/IP Commands. TCP/IP

12. Implementation (1/2) • The prototype was programmed in a portable framework, VR Juggler that supports : • Most widely used operating systems Linux, Windows, Mac OSX, Irix, FreeBSD. • Several trackers and display devices (HiBall, Flock of birds, Trackd, projectors, monitors, CAVE walls. http://www.vrjuggler.org/features.php

13. Prototype implementation (2/2)

14. Through the lens interface

15. Selecting 3D objects • Direct selection: • The user selects the target object directly on the PDA screen • Indirect selection: • User searches for a comfortable view • User takes a snapshot • User selects the object from the snapshot

16. Direct selection

17. Indirect selection

18. Pilot study - Setup Experimental task: Target selection task Independent variables: • Type of selection: Direct, Beam, PDA • Layout: Dependent variables: Task completion time, Number of errors, Preferences • Sparse layout: • 30 cm separation, 72 cubes • Dense layout: • 10 cm separation, 504 cubes

19. The pilot study (1/3) Direct selection

20. The pilot study (2/3) Beam selection

21. The pilot study (3/3) PDA selection

22. Results (1) Average time for completion task (sparse) 2nd trial Overall 1st trial 3rd trial 2nd trial Overall 1st trial 3rd trial Average time for completion task (dense) • On Sparse layout PDA interface was most difficult to use at first, but after 2 trial became the easiest one. • On Dense layout unexpectedly we found Beam got a better performance than PDA. • The number of errors do not show significant difference.

23. Observations • Users require some time to adapt to our interface for use it effectively. • We noticed that users did not use the snapshot selection method. Instead of it, they tried to approach the target.

24. Results (2) It was easy to select the object? Dense Sparse Selecting objects with this interface is natural? Dense Sparse

25. Conclusions • We implemented a PDA-based see-through interface for an immersive environment • Users found our interface more natural for selecting objects than direct and beam interfaces (not statistically supported)

26. Future uses of the interface • Supporting navigation using teleporting windows[6]. • Clipboard tool (cut and paste) [6] K. Hirose, T. Ogawa, K. Kiyokawa and H. Takemura. （2006）

27. Q & A Thank you for your attention!

28. Reference • [1] Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, and Tony D. DeRose, “Toolglass and magic lenses: the see-through interface”, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p. 73-80. 1993. • [2] J. Viega, M. J. Conway, G. Williams, and R. Pausch, “3D Magic Lenses,” Proceedings of ACM Symposium on Users Interface Software and Technology (UIST) 1996, pp.51-58, 1996. • [3] Stanislav L. Stoev, Dieter Schmalstieg, “Application and taxonomy of through-the-lens techniques”. Proceedings of the ACM symposium on Virtual reality software and technology 2002: 57-64 • [4] Jeffrey S. Pierce, Andrew Forsberg, Matthew J. Conway, Seuung Hong, Robert Zeleznik, and Mark R. Mine, “. Image plane interaction techniques in 3D immersive environments,”. Symposium on Interactive 3D Graphics. Proceedings of the 1997 symposium on Interactive 3D graphics, pp. Pp. 39-43, 1997. • [5] Dieter Schmalstieg, L. Miguel Encarnacao and Zsolt Szalavari, “Using Transparent Props For Interaction With The Virtual Table,” Proceesings of the ACM Symposium on Interactive 3D Graphics, pp. 147-153, 1999. • [6] K. Hirose, T. Ogawa, K. Kiyokawa and H. Takemura, “Interactive Reconfiguration Techniques of Reference Frame Hierarchy in the Multi-viewport Interface,” Proceedings of IEEE Symp. on 3D User Interfaces (3DUI), pp. 75-82, 2006.