Selected Topics in 3D User Interfaces

1. Selected Topics in 3D User Interfaces Joseph J. LaViola Jr. CS 196-2 March 6, 2006

2. Lecture Outline • 2D/3D interfaces • Nonisomorphic interaction

3. Lecture Outline • 2D/3D interfaces • Nonisomorphic interaction

4. 2D Interaction • Advantages • provides a sense of feedback • very accurate • some operations that are 3D in nature are more easily done with a 2D input device (e.g. object selection) • picking objects is much easier in two dimensions • Limitations • manipulating 3D objects • have to add 3rd dimension in unconventional and unnatural ways • WIMP

5. 3D Interaction • Advantages • more natural for object manipulation once the object is taken • take advantage of 3D hand gestures and postures • stereoscopic vision • Limitations • very difficult to write and annotate • difficult to pick and place objects accurately

6. Bringing 2D and 3D Together • Goal: Let’s take the advantages from each type of interaction and bring them together to form a more usable interface • Broaden the application space

7. Seamless Integration • Critical component • Requires both physical and logical integration • Do not want the user to work hard to change modes • Tools should know what interaction technique they are used for • a device should know whether it is used for 3D interaction or 2D interaction based on context

8. 2D/3D Interface Taxonomy • Based on display surface interaction • Taxonomy • direct • hand-held indirect • hand-held direct

9. Direct Display - ErgoDesk • 3D modeling application • 2D interaction on display surface • Based on Sketch • Allows users to create, edit, view and manipulate 3D models

10. Hand-Held Indirect (1): Virtual Notepad • Tool for writing in immersive environments • Allows users to take notes and annotate documents

11. Hand-Held Indirect (2): Transparent Pad • Transparent prop for the Virtual Table • tool and object palette • window tools • through-the-plane tool • volumetric manipulation

12. Hand-Held Direct Displays • PDA’s in Immersive VEs • Watsen used PalmPilot in a CAVE-like device [IPT99] • provides camera, environment, and geometry controls • Wacom Tablet in the TAN-Cube • too heavy • wires got in the way • has potential

13. Going Beyond the 2D/3D Taxonomy • Go beyond the 2D surface and hand approach • Utilize traditional 2D concepts and extend to 3D interfaces • Step WIM – based on maps • TULIP – based on 2D menus

14. The Step WIM • Miniature version of the world placed on the floor • Motivated by Pausch and Stoakley’s WIM • Augmented roadmap • Step WIM scales up around users feet • Operations • invoking • navigating • dismissing • scaling

15. Foot-based Interface • Toe and heel tapping • “no place like home” metaphor • Developed interaction slippers • Disambiguation of navigate and dismissal • based on user gaze • derived from pilot studies

16. Body Gesture Interface • More fluid gesture/less invasive device • Use waist tracker to detect upward bouncing gestures • Algorithm • first get user’s initial waist height • monitor the waist tracker’s position • check to see if the waist is above a height delta for a given amount of time

17. Step WIM Scaling • VEs may be too large to fit within user’s walking area • Scaling implicitly provides different levels of detail

18. Foot-based Scaling • Heel click toggles Step WIM scaling mode • Center of scale is user’s initial “location” in WIM • maintain position within the WIM • Walking forward – closer look at the world • Step WIM grows larger • Walking backward – gain perspective • Step WIM grows smaller

19. Body Gesture Scaling • Avoid cue conflict of “walking in place” • Holding a crouching gesture increases Step WIM size • Holding a bouncing gesture decreases Step WIM size • Center of scaling is projection of user’s waist • Gestures must be held longer than the bounce time threshold • distinguishes between scaling and activation/dismissal

20. TULIP – Three Up Labels in Palm • Menu system using Pinch gloves • Derived from a number of iterations • Non-dominant hand controls menus • Dominant hand controls menu items

21. TULIP – Evaluation • Compared with pull-down and pen and “pen and tablet” menus • “Pen and tablet” found to be faster • Users preferred TULIP • TULIP had higher comfort level

22. Lecture Outline • 2D/3D interfaces • Nonisomorphic interaction

23. Isomorphic vs. Non-Isomorphic Philosophies • Human-Machine interaction • input device • display device • transfer function (control to display mapping) • Isomorphic – one-to-one mapping • Non-isomorphic – scaled linear/non-linear mapping

24. Non-Isomorphic 3D Spatial Rotation • Important advantages • manual control constrained by human anatomy • more effective use of limited tracking range (i.e vision-based tracking) • additional tools for fine tuning interaction techniques • Questions • faster? • more accurate?

25. Rotational Space • Rotations in 3D space are a little tricky • do not follow laws of Euclidian geometry • Space of rotations is not a vector space • Represented as a closed and curved surface • 4D sphere or manifold • Quaternions provide a tool for describing this surface

26. Quaternions • Four-dimensional vector (v,w) where v is a 3D vector and w is a real number • A quaternion of unit length can be used to represent a single rotation about a unit axis and angle as

27. Linear 0th Order 3D Rotation • Let be the orientation of the input device and be the displayed orientation then • Final equations w.r.t. identity or reference orientation are

28. Non-Linear 0th Order 3D Rotation • Consider • Let k be a non-linear function as in

29. Design Considerations • Absolute mapping – taken on i-th cycle of the simulation loop • Relative mapping – taken between the i-th and i-1th cycle of the simulation loop

30. Absolute Non-Isomorphic Mapping • Generally do not preserve directional compliance • Strictly preserve nulling compliance

31. Relative Non-Isomorphic Mapping • Always maintain directional compliance • Do not generally preserve nulling compliance

32. Experimental Usability Study • Comparison of relative non-isomorphic rotation technique with conventional technique • Hypothesis • rotation tasks will be faster with non-isomorphic approach for large rotations • moderate amplified rotations will decrease accuracy • Results • subjects performed 13% faster with non-isomorphic approach with no accuracy degradation

33. Amplified Non-Linear Rotation for VE Navigation (1) • Users expect the virtual world to exist in any direction • 3-walled Cave does not allow this • adapt expected UI to work in restricted environment • Amplified rotation allows users to see a full 360 degrees in a 3-walled display • A number of approaches were tested • important to take cybersickness into account

34. Amplified Non-Linear Rotation for VE Navigation (2) • Apply a non-linear mapping function to the user’s waist orientation and his or her distance from the back of the Cave • Calculate the rotation factor using a scaled 2D Gaussian function • The new viewing angle is

35. Amplified Non-Linear Rotation for VE Navigation (3)

36. Non-Linear Translation for VE Navigation (1) • Users lean about the waist to move small to medium distances • users can lean and look in different directions • Users can also lean to translate a floor-based interactive world in miniature (WIM) • Step WIM must be active • user’s gaze must be 25 degrees below horizontal

37. Non-Linear Translation for VE Navigation (2) • Leaning vector is the projection of the vector between the waist and the head onto the floor • gives direction and raw magnitude components • Navigation speed is dependent on the user’s physical location • Leaning sensitivity increases close to a boundary • Linear function - • Mapped velocity -

38. Non-Linear Translation for VE Navigation (3) • Navigation speed is also dependent on the user’s head orientation with respect to the vertical axis • especially useful when translating the floor-based WIM • Mapping is done with a scaled exponential function • Final leaning velocity is

39. Conclusions • Important ideas found in Chapters 5-8 of 3D UI book