A Mixed Reality Approach for Merging Abstract and Concrete Knowledge

1. A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira Fischler Department of Psycholgy Paul Fishwick Department of CISE Benjamin Lok Department of CISE

2. Anesthesia Education • Current Learning Process The Virtual Anesthesia Machine (VAM) – an abstract 2D simulation A real Anesthesia Machine – a physical simulation

3. Transfer Knowledge? Knowledge Types

4. Problem: Merging Knowledge Types • Anesthesia machine trainees must mentally merge: • Abstract Concepts • Procedural Skills • Mixed Reality (MR) can merge real and virtual spaces • Virtual space = Abstract • Real Space = Concrete • Can MR’s Merging of spaces improve users’ merging of abstract and concrete knowledge?

5. Merging Abstract and Concrete • Merge the abstract physical simulation with the physical device

6. The Augmented Anesthesia Machine • Uses a magic lens approach to merge: • Abstract VAM simulation • Real Anesthesia Machine TUI

7. Overview • The Augmented Anesthesia Machine • The Augmented Learning Process • User Study: VAM vs. AAM • MR enables abstract + concrete knowledge merging

8. Previous Work • Tangible User Interfaces • Two Main Parts: • Computational Media • Physical Interface • Interface is meaningful • Representative of a part of the computational media • As opposed to an abstract interface (i.e. a mouse) • AAM • Computational media = VAM • Physical Interface = real anesthesia machine [Ishii 1997, Ullmer 2000]

9. Previous Work • Magic Lenses • 2D to 3D • Applications in AR/MR • AAM • Uses a magic lens to visually merge the abstract and concrete spaces [Bier 1993, Looser 2004]

10. Motivation • 75% of Anesthesia Machine Related Accidents resulting in death or brain damage are due to user error. • Many anesthesia providers do not fully understand how the machine functions internally • VAM was created to reduce this number

11. Motivation for the VAM • Created by Dr. Samsun Lampotang • Advantages of the VAM • visualize the invisible (i.e. gas flow) • Spatial layout simplified for ease of visualization • Web disseminated • Increases understanding of internal function [Fishcler et. al. 2006] • 30000 Registered Users • One Billion Hits per year (http://vam.anest.ufl.edu) • Problems still exist in knowledge transfer from the VAM to the real machine

12. Motivation • VAM to AAM: A Mapping Problem B A A B

13. MR: Enabling Multiple Represetations • Multiple Representations improve overall comprehension of a concept. • VAM: Abstract Representation • Anesthesia Machine: Concrete Representation • AAM: Combined Representation • Proposed Augmented Learning Process: Abstract Representation (VAM) Combined Representations (AAM) Concrete Representation (Anesthesia Machine)

14. The AAM Representation • Tracked 6DOF magic lens shows overlaid VAM simulation • Real machine interaction

15. Anesthesia Machine Tracking System • 2D Optical Tracking with OpenCV • Infrared Markers for knobs • Infrared LEDs connected to buttons • Color tracking for gauges (bright red) • 3 Unibrain webcams positioned around the machine in positions of minimal occlusion

16. Magic Lens Display • HP TC1100 • 3.7 lbs • 10” screen • Not See through • Uses a 3D model of the machine, registered to the real machine • Easier to enable consistent registration • Less hardware than video see through

17. Magic Lens Tracking System • Outside-Looking-in Optical Tracking • Infrared Markers • Specifications: • 2 Unibrain Fire-I Webcams 640x480 at 30 fps • Tracking Volume: 3x3x3 m • Accuracy: 1cm • Jitter: 5mm • Latency: 70 ms

18. User Study • Can AAM-Concrete help users to merge abstract and concrete knowledge? • Between subjects study (n=20) • VAM Training Group • AAM Training Group

19. Population and Environment • 20 Psychology Students • 4 males, 16 females • Received class credit • Knew nothing about anesthesia machines • Conducted in a quiet lab environment

20. Procedure Day 1 (90 min) 5 Training Exercises Introduction Machine Functions Spatial Cognition Tests Informed Consent VAM AAM Day 2 (60 min) Real Machine Intro Self Evaluation of Training VAM Hands-on Fault Test Written Test Questionnaires AAM

21. Metrics • Efficiency of Training • Time to complete 5 training exercises • Abstract Knowledge Acquisition • Written Anesthesia Machine Test • Short Answer and muliple choice • From the Anesthesia Patient Safety Foundation workbook • Concrete Knowledge Acquistion • Hands-on fault test • Faulty inspiratory valve

22. Fault Test Results • Hypothesis 1: The AAM is more effective at teaching concrete concepts. • AAM group found the faults significantly more often than VAM group • Accept Hypothesis 1: The AAM trains concrete knowledge more effectively than the VAM

23. Written Test Results • Hypothesis 2: The abstract representation of the VAM is more effective at teaching abstract concepts • There were no significant differences in Written test scores (p<0.21)

24. Training Time Results • Hypothesis 2: The abstract representation of the VAM is more effective at teaching abstract concepts • VAM Group trained significantly faster (p<0.002) • Accept Hypothesis 2: The VAM trains abstract knowledge more effectively than the AAM

25. Discussion • Hypothesis 3: The AAM improves transfer to the real machine by enabling the merging of abstract and concrete knowledge. • AAM group performed significantly better in the fault test • To solve the fault test, participants had to merge: • Concrete knowledge: notice the inspiratory valve was missing • Abstract knowledge: understand the effect on the gas flow of the machine – harmful to the patient • Accept Hypothesis 3: AAM helps to merge abstract and concrete knowledge

26. Conclusions • leverage MR to merge simulation types • Combining Abstract simulation with the corresponding physical device • i.e. merging the VAM with the real machine • MR enables multiple representations • MR can help users to merge abstract and concrete knowledge • Improves training transfer into real world domains