1 / 48

Ming Hou

Effects of Surface Characteristics on Alignment of Real and Graphic Objects in Stereoscopic Augmented Reality Environments. Ming Hou. Ergonomics in Teleoperation and Control ( ETC ) Laboratory. Department of Mechanical and Industrial Engineering. University of Toronto. January 6th, 2003.

gala
Download Presentation

Ming Hou

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Effects of Surface Characteristics on Alignment of Real and Graphic Objects in Stereoscopic Augmented Reality Environments Ming Hou Ergonomics in Teleoperation and Control (ETC) Laboratory Department of Mechanical and Industrial Engineering University of Toronto January 6th, 2003

  2. 3D Measurement in Unmodelled World Virtual Tape Measure

  3. Problem: Unknown relationshipbetween real and virtual objects in Augmented Reality displays

  4. Research on Real World Targets Cylinder LINE AREA VOLUME Hemisphere Virtual Pointer Video Image

  5. Pseudo-Transparency Phenomenon Real Object (in Video) Virtualpointer behindrealsurface Virtual pointer infrontof surface Breakdown of Fusion + Fusion Conflict “Transparency Effect”

  6. Conflict between Binocular Disparity and OcclusionCues Virtual pointer behind real surface Virtual pointer infrontof surface Fuse Real Surface Fuse Virtual Pointer

  7. Theory of Surface Interaction: Fusion Breakdown depends on Texture Density Virtual pointerbehind real surface Virtual pointer infrontof surface Low Texture Density High Texture Density

  8. Research Motivation Is this conflict really significant? If yes, can it be used as an extra cue for detecting interactions between real and virtual objects, and thus locating the real objects in AR environment more easily and accurately?

  9. Main Hypotheses #1 More accurate to indicate position on (curved) surface with high texture density (HTD) than with low texture density (LTD) HTDbetter than LTD

  10. Main Hypotheses #2 Top View Top View Real Surface Real Surface Stereo Camera Stereo Camera Virtual Pointer Virtual Pointer Orientation of observer relative to target surface will have influence Centredifferent fromOff-centre

  11. Main Hypotheses #3 Form of Virtual Pointer (VP) will have impact on alignment performance LINE AREA VOLUME

  12. C ND UC Main Hypotheses #4 Binocular disparity (i.e. crossed vs 0 vs uncrossed) will affect alignment performance No disparity (ND) > Crossed (C) or Uncrossed (UC)

  13. Stereo Camera Stereoscopic AR Display Virtual Pointer Spaceball Cylinder Indigo 2 Barrier Experimental Investigation ofAR “ Surface Effects ”

  14. Methodology

  15. Subjective Comparisons Ease of Use Transparency Ease of Fusion

  16. Methodology

  17. Placement Error vs Texture Density- 1 Farther \ \ Cylinder Surface \ \ Closer Main Experimental Results - 1

  18. Placement Error vs Texture Density - 2 Main Experimental Results - 2

  19. Placement Error vs Surface Orientation - 1 Farther Closer Main Experimental Results - 3

  20. Placement Error vs Surface Orientation - 2 1 5 2 4 3 Main Experimental Results - 4

  21. Angular Error vs Surface Orientation 1 5 2 4 3 Main Experimental Results - 5

  22. Subjective Comparison Results w.r.t. Ease of Use and Ease of Fusion VolumeVP better than other VPs, regardless of texture density w.r.t. Transparency Volume VP + Highly textured least transparent combination Main Experimental Results - 6

  23. Summary of Results - 1 • Texture DensityHigh better than Low • Surface Orientation Centre different from Off-Centre

  24. Summary of Results - 2 C ND UC VP FormVOLUME > AREA > LINE(subjective comparisons) Binocular Disparity

  25. Conclusions - 1 • Perceptual conflictdoes exist when real and virtual objects interact in 3D AR environments (a model proposed) • Perceptual conflict can be used as extra depth cue to indicate interaction between real and virtual objects • Optimal density value for Random Dot texture pattern was found as Engineering Solutionfor accurate 3D measurement

  26. R-V Interaction Process Model Perception (Stereo Matching) Cognition / Decision Making (Stereo Matching + Cue Conflict Resolution) behind (Breakdown/Conflict) VP behind, or at, real surface? Fusion difficulty at Localisation Achieved Fused Image? at (Transparency) No fusion difficulty VP behind, or at, real surface? No Stereo Matching : VP in front of real surface? Adjustment behind Yes User External World Display of virtual pointer (VP) superimposed upon real object surface AR Display VP Controller

  27. Conclusions - 1 • Perceptual conflictdoes exist when real and virtual objects interact in 3D AR environments (a model proposed) • Perceptual conflict can be used as extra depth cue to indicate interaction between real and virtual objects • Optimal density value for Random Dot texture pattern found as Engineering Solutionfor accurate 3D measurement

  28. Conclusions - 2 • Target positiondoes affect alignment task: centrally located targets benefit performance, but have disadvantage when along the line of sight • Volumetric stereo graphic cursor (more fusable features along three dimensions) is subjectively the most favoured VP • “Pseudo-transparency” contributes literature of depth perception cues (shape-from-texture and stereo)

  29. Implications for AR Interface Design • Random Dot Stereogram enhances 3D alignment performance • Perceptual conflicts can be used as extra depth cue to detect real object position • 3D VP better than other VPs • Perceptual errors always exist

  30. Limitations • Implementation • Display Mode: stationary display without motion parallax and motion perspective • Binocular Disparity : confounded with size cue and resolution • Scope • VPDesign:line thickness of wire-frame VP • Texture Pattern: square Random Dot pattern

  31. Future Work/Impact - 1 Projected lighting with random dot texture pattern Simulated Projector Stereo Cameras • Near Term Research • Projected lighting : more practical • Motion parallax with Video-HMD • Computational visionmay alleviate some error (being investigated)

  32. Future Work/Impact - 2 • Long Term Interests • Integration of Computer Assisted Object Detection in AR Displays • See-through HMD AR for Dismounted Soldiers in the Battlefield (e.g., Perceptual Conflicts, Navigational Aids, etc.) • Other Human Computer Interaction (HCI) Topics (e.g., Integration of Electronic Information with Human-Machine Systems, etc.)

  33. Acknowledgement • Dr.Julius Grodski at Defence Research & Development Canada (DRDC) – Toronto,Prof.Allison B. Sekuler and Prof.Paul Milgram at University of Toronto • Dr. Stephen Ellis at NASA and Prof. Stanley Hamstra at University of Toronto • Natural Sciences and Engineering Research Council (NSERC) Doctoral Scholarship • Institute of Robotics and Intelligent Systems (IRIS), Canada • Ontario Graduate Scholarship (OGS)

  34. Experimental #1 Main Result Farther \ \ Cylinder Surface \ \ Closer

  35. VP Form and Orientation in 1stExperiment : Vertical Diagonal Horizontal Observer

  36. VP Placement vs Texture and Target Position (Error Bar = +/- 1SD, F(1,9) = 246.33, P< 0.001) 1 0 -1 -2 -3 -4 PlacementError (cm) -5 -6 -7 High -8 -9 Low -10 -11 Centre Right Angular Displacement of Target Normal Interaction in Experiment # 1 Interaction between Surface Texture (High vs Low) and Target Position (Center vs Right)

  37. Mean Z score 0 1.15 1.71 2.26 3.16 Ease of Use Transparency Ease of Fusion Image # 1 2 3 4 5 6 0 0.25 0.89 2.97 3.82 4.37 Image # 2 1 3 6 5 4 0 0.69 0.73 2.08 2.14 2.26 Image # 1 2 3 5 4 6 Paired Comparison Result

  38. Surface Normal Estimated Target Real Surface Target Positive Error Measurement of Placement Error along Surface Normal Positive error shows the estimated target is inside the sphere along surface normal

  39. Experiment2 Experiment 3 Placement Error vs Texture Density for Experiments 2 and 3

  40. Definition of Angular Error Y Altitude Error :Angular distance between estimated normal TP and real surface normal TN Azimuth Error :Angular Error between horizontal projection OR and OS N P T Q O X R S Z

  41. Y N E T Bias Area S X Z Angular bias tilted upwards from real surface normal (TN) Angular Bias in Spherical Coordinate

  42. Example of Distribution of Altitude and Azimuth Angular bias between estimated and real surface normal

  43. Ground Truth Measurements in Real Scene Cylinder Stimulus Side View Stereo Cameras Top View Iron plate Cylinder Calibration object Real World Origin (0,0,0) Real Distance in Depth (Z)

  44. Registration Verification: Measurement of a Pin and a Cube Calibration Target Calibration Cube Stereo Cameras

  45. VP Resolution for Experiment 2 VP Resolution and Accuracy Tests

  46. Retinal Disparity in Stereoscopic Display for One Pixel Separation (exaggerated) ZD Apparent position of point C’, due to 1 pixel horizontal disparity Stereoscopic Display Monitor C’ b P = 1 pixel separation XD b /2 a d Viewer’s Eye 2e

  47. Psychophysical Standard for Texture Density Control • Spatial scale (size) • Homogeneity (spatial regularity, density is approximately constant over the surface) • Isotropy (no orientation bias, equally to be oriented in all directions) – compression

  48. Practical Augmented Reality Example Distance between point 1 and point 2 is 7mm Coordinate of Point 1 (2.3, 14.7, 96.2) Coordinate of Point 2 (1.8, 14.4, 95.8) Virtual Tape Measure for Minimally InvasiveSurgery

More Related