Spatiotemporal Information Processing No.4 3 components of Virtual Reality-3 Display System

1. Spatiotemporal Information ProcessingNo.43 components of Virtual Reality-3Display System Kazuhiko HAMAMOTO Dept. of Information Media Technology, School of Information and Telecommunication Eng., Tokai University, Japan

2. Today’s Contents • Somatic sense • Skin sense • Deep sense • Haptic sense • Display system – skin sense • The principle of tactile primary color • Display system – deep sense • Display system – haptic sense

3. The relationship among 3 components (review) real space virtual space Display system Simulation system Person Sensing system computer

4. Somatic sense (1/4) • Cutaneous sense (or sense of touch or skin sense) • A sense by some sensory receptors which lie beneath the surface of the skin • tactile sense, temperature sense, pain sense

5. Somatic sense (2/4) • Deep sense (or proprioceptive sense) • A sense of the relative position of neighboring parts of the body, whose receptors are found in muscles, tendons and joints • kinetic sense, postural sense

6. Somatic sense (3/4) • How can “iron ball” be felt by human sense ? • Shape ? • Skin sense or deep sense • Surface pattern (texture) ? • Skin sense or deep sense • Recognition of “iron” ?

7. Somatic sense (4/4) • Recognition of shape • If we wear gloves…. • We can feel -> deep sense • Recognition of pattern • If we wear gloves…. • We cannot feel -> skin sense • Recognition of “iron” • Temperature sense

8. Haptic sense • “Active touch” can improve the precision of the recognition • Haptic sense • Active touch sense, which is integrated sense of skin sense and deep sense with action of the body

9. Skin sense and receptors • Tactile sense • Merkel disk • displacement • Meissner corpuscle • velocity • Ruffini corpuscle • displacement • Sense of contact force (pressure) • Ruffine corpuscle • displacement • Pacinian corpuscle • acceleration

10. http://www.unmc.edu/physiology/Mann/mann5.html Skin sense and receptors

11. Deep sense and receptors • Receptor in muscle • Muscle spindle • Stretch receptor, which sends muscle’s length information to CNS • Receptor in tendon (joint) • Golgi tendon organ • Another type of receptor, which provides information about changes in muscle tenstion

12. Muscle spindle CNS : Central nervous system http://www.ucl.ac.uk/~sjjgsca/MuscleControl.html

13. Golgi tendon organ

14. Display system – skin sense • Virtual sandpaper (M.Minsky, MIT, 1995) • Recognition of “type of Texture” • Fine uneven surface • Vibration of small amplitude, high frequency • Rough uneven surface • Vibration of large amplitude, low frequency

15. Virtual Sandpaper Margaret Minsky, MIT, 1995 Screen image Experimental setup http://dspace.mit.edu/handle/1721.1/29094

16. Display system – skin sense • The principle of tactile primary colors • A kind of functional electrical stimulation (FES) • Each receptor is stimulated electrically and independently • 3 receptors can be stimulated independently by the differences of how deep seated or singularity of feature along axone.

17. Prototype using the principle of tactile primary colors • Sense of pressure, low and high freq. vibration can be presented (2005) Prototype of the tactile display http://www.star.t.u-tokyo.ac.jp/projects/tactile-display/index-j.html

18. Display system – deep sense • Equipment type • A display device is equipped on hand or fingers. • Action of hand or fingers is restricted by the display device. • The restriction causes a feeling of “existence of an object”

19. Display system – deep sense Cyber Grasp

20. Display system – deep sense • SPIDAR • 8 wires and 8 motors detect the position and the direction of the ball • User controls the ball in 6DOF • Sense of touch or collision is transformed to tension of the wires http://www.ddd.co.jp/product/spidar/

21. Display system – Haptic sense • Haptic display • Present tactile information • Stimulus of tactile organ • Force feedback system • Shape, hardness, softness, smoothness, friction and gravity are perceived in virtual space.

22. Display system – Haptic sense • PHANToM Desktop (SensAble Device, Inc.) • 6DOF • Refresh rate • 1000 [Hz] • Simple structure • Torque generated by motors is transmitted to active part by wire.

23. Display system – deep sense • Problems • Always feel a display’s weight and movement • Difficult to create “we don’t touch anything” • Response of a motor (actuator)

24. Novel Development in my Lab. • Deep sense • Artificial muscle is used as switching device • Skin sense • Uneven feeling can be realized by MR fluid Haptic interface Actuated with Magnetorheological fluid and Artificial muscle HAMAデバイス

25. Novel Development in my Lab. MR fluid for sense of cutaneous Power supply Artificial muscle switch for deep sense New haptic device Electric circuit for drive

26. New device for deep sense with Artificial muscle Lock by BMX Unlock by BMX movable part Restricted finger Released finger

27. New device for skin sense with MR fluid When MR fluid into electro magnetic field, the viscosity becomes higher