Virtual Reality (Artificial Reality)

1. Virtual Reality (Artificial Reality)

2. Definition • “A computer system used to create an artificial world in which the user has the impression of being in that world and with the ability to navigate through the world and manipulate objects in the world.” • C. Manetta and R. Blade

3. Motivations • A display connected to a digital computer gives us a chance to gain familiarity with concepts not realizable in the physical world. It is a looking glass into a mathematical wonderland. • The ultimate display would, of course, be a room within which a computer can control the existence of matter. • - Ivan E. Sutherland

4. Two Requirements of VR • Immersive • Interactive

5. Two Related Courses of VR • Computer Graphics • Computer Vision

6. Two Ways to construct VR

7. Two Types of VR Immersive : 3D environment seen through a head-mounted display (HMD). In a completely immersive system the user has no visual contact with the physical world. Semi-immersive : Most advanced flight, ship and vehicle simulators are semi-immersive. The cockpit, bridge, or driving seat is a physical model, whereas the view of the world outside is computer-generated (typically projected).

8. Two Parts of VR • Augmented Virtuality • Augmented Reality

9. Evolution of VR Adventure games, MUD(Multi-Users Dungeon) : Textually described virtual worlds where the user perceives the virtual environment through mental images like reading a novel. Desktop : 3D virtual environment graphically displayed on a desktop computer monitor.

10. Evolution of VR Projected : 3D environment projected onto a screen. Enables a single user to demonstrate concepts to a group of people. A CAVE (developed by University of Illinois), where several screens are used to surround the user with images, is the most advanced form of projected VR in use today.

11. VR Technology • Hardware capable of rendering real-time 3D graphics and high-quality stereo sound. • Input devices to sense user interaction and motion. • Output devices to replace user's sensory input from the physical world with computer-generated input. • Software that handles real-time input/output processing, rendering, simulation, and access to the world database in which the environment is defined.

12. Head Mounted Display (HMD) • Can display either stereo or mono images • Stereo images (binocular disparity) • Same image twice (binocular concordance) • Single image (uniocular) • May be totally immersive or semi-immersive (see-through) • May include a built-in head-tracker • May have built-in stereo headphones

13. Each eye sees a different field of view

14. Alternative Displays • LCS (Liquid Crystal Shutter) glasses • Display shows left and right images alternately, switching at high speed between images • Stereoscopic image is seen when the display is viewed with special glasses • Typical 'Fishtank VR' • Particularly good for large audiences in a theatre

15. Alternative Displays • BOOM(Binocular Omni-Orientation Monitor) by Fakespace • Uses a CRT to provide high-resolution display • It is comfortable to use, since it does not have to be worn • Has fast, accurate, built-in tracking

16. Alternative Displays • VRD (Virtual Retinal Display) • Image is projected directly onto the retina (by Microvision)

17. Motion Tracking Types • Mechanical • Usually a mechanical arm attached to the tracked object • Very accurate, short lag, but restrict movement • Electromagnetic • Measures strength of magnetic fields in coils attached to objects • Fast, short lag, but often prone to interference • Limited range

18. Motion Tracking Types • Optical • Typically, pulsating LEDs monitored by a camera at a fixed position • Fast, reasonably short lag, but often prone to interference caused by ambient lighting conditions • Line of sight problems • Acoustic • Use ultrasound waves to measure position and orientation • Slow and often imprecise

19. Interaction Devices • 3D Mice • Spaceballs • Eye tracking • Video camera and shadows • Voice recognition • Biological sensors

20. Interaction Devices Light Pen

21. Interaction Devices • Data Gloves

22. Interaction Devices • Data Gloves • Hand and gesture tracking • Enables natural interaction with objects • Can use hand-signs to execute actions • Full body suits • Motion capture

23. Feedbacks • Motion platforms • Tactile feedback provides a sense of touch through, typically, vibrating nodules or expanding air bubbles inside a glove or suit • Force feedback provides physical constraints • Exoskeletons • Joysticks, Hand controllers

24. Sound • Important to create a sense of atmosphere • Can greatly enhance feeling of presence • Can be used to provide valuable depth cues, aiding navigation • Enables the user to perceive events that occur outside the immediate field of view • Audio feedback

25. Software • Need modeling tools to create objects (AutoCAD, 3D Studio, etc.) • Designing objects is time-consuming • Objects often need to be optimized for VR use

26. Applications • Architecture • Design and Prototyping • Education & Conferencing • Training • Medical • Business and Visualization • Art and Leisure

27. Architecture • Walkthroughs to evaluate design decisions and present designs to customers • Demonstrate how a planned construction fits into the environment in which it is intended to be built

28. Design and Prototyping • Use to create rapid prototypes rather than make clay models or full-scale mock-ups • Simulate assembly lines. For example, to evaluate the risk of interference or collision between robots, cars, and car parts

29. Education & Conferencing • Education • Visualize concepts • Visualize the past ('Virtual Heritage') • Virtual lectures and classrooms • Conferencing • Collaborative work over the Internet • Virtual work groups • Virtual conferences

30. Training • Civilian and military training simulators • Driving simulators • Flight simulators • Ship simulators • Tank simulators

31. Training • Train for hazardous or difficult operations • Nuclear plant maintenance • Learn to move in zero gravity • Practice locating and fixing faults in equipment

32. Medical • Surgery • Practice performing surgery • Perform surgery on a remote patient • Rehabilitation • Phobia therapy • Use VR input devices and tele-presence to enable handicapped people to do things that would otherwise be impossible for them to do • Enable people to visit/see places that they would be otherwise unable to experience • Use VR to teach new skills in a safe, controlled, environment

33. Business and Visualization • Business • 3D visualization of complex financial information • Demonstrate customizable products to customers • Scientific Visualization • View complex data sets to gain greater insight and understanding of structure • View complex molecular and geological structures

34. Art and Leisure • Art • Virtual galleries and museums • Virtual theatres • Leisure • Games • Sport simulators

36. HSCT

37. XVS

38. Challenges of VR

39. Hardware Technology • More realistic graphics and audio (this is also a software problem) • Greater processor power • Less lag • Less obstructive input/output devices • Better display technology

40. Building worlds • Creating models takes a long time and is a skill that needs to be acquired • We need better software tools to create objects, add behavior to objects, and to handle interaction • Optimizing models takes time and synchronizing modifications can be difficult (CAD tools to/from VR environment builder)

41. Building worlds • Implementing scripts and interaction is often difficult for non-programmers • Making accurate models of the physical world is difficult. Devices such as laser scanners are making this easier, but human intervention is still a necessary and time consuming part of the modeling process • Can we build knowledge-based systems to help us capture the physical world?

42. User Interaction Design • The user interface for a virtual environment needs to be carefully designed to take into account the functional requirements of the application and the limitations of current technology • How should the user interact with the virtual environment? • How should the computer provide feedback? • What is the most meaningful way to visualize an abstract object/process/structure? • What does cyberspace (the Internet) look like? • How do we improve navigational and spatial awareness?

43. Human Factors • There are a number of factors that should be taken into consideration when designing the user interface of a virtual environment in order to minimize physical discomfort for the user • Latency causes user discomfort (simulation sickness, nausea) • Well design and optimized models reduce latency and improve rendering consistency • Eye strain problems? • Long-term physical and psychological effects?

44. Distributed Virtual Environment • How should users communicate? • How do I know you are real and not an agent? • Social effects of distributed VR? • Technological: Lag, large numbers of users simultaneously, etc.

45. Conclusion • VR is a powerful user interface technology • Choosing the best way to visualize information is important • VR enables the user to interact directly with information • VR enables the user to see/experience things in new ways • VR does not have to be immersive to be useful