a cross modal electronic travel aid device n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
A Cross-modal Electronic Travel Aid Device PowerPoint Presentation
Download Presentation
A Cross-modal Electronic Travel Aid Device

Loading in 2 Seconds...

play fullscreen
1 / 16
warren-johnson

A Cross-modal Electronic Travel Aid Device - PowerPoint PPT Presentation

120 Views
Download Presentation
A Cross-modal Electronic Travel Aid Device
An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. A Cross-modal Electronic Travel Aid Device F. Fontana, A. Fusiello, M. Gobbi, V. Murino, D. Rocchesso, L. Sartor, A. Panuccio. Università di Verona Dipartimento di Informatica

  2. Overview • Motivations • The system • Visual analisys • Auditory display • Work in progress Mobile HCI 2002

  3. Motivations • Humans use several senses simultaneously to explore and experience the environment. • Multimodal displays can enhance the user experience and the sense of engagement. • The redundancy of our sensory system can be exploited in order to choose the display that is the most convenient for a given application. • Our interest: exploring new ways of transferring information across different sensorial modalities, especially in the context of interactive systems. Mobile HCI 2002

  4. Cross-modality: methods to perform analysis tasks using one modality, and synthesis (display) tasks using another modality. • Examples: give auditory display to visual information, visualize auditory scenes, and sonify haptic sensations. • Design problem: how to render a given percept using a certain modality, the latter possibly being not the most obvious for the stimulus at hand. Mobile HCI 2002

  5. A cross-modality instance • Auditory display of visual information • More effective HC interfaces • Aid devices for visually impaired people • Auditory display needs sonification of the (visual) objects being displayed. • Sonification is the acoustic analogous of visualization. Mobile HCI 2002

  6. The system • The blind person is equipped with a stereo camera and earphones. • He/she uses a laser pointer as a cane. • The system computes the 3D position of the laser spot and sonifies this piece of information Mobile HCI 2002

  7. Visual analisys • Track the position of the laser spot in two images • Compute the 3D position of the laser spot by triangulation Mobile HCI 2002

  8. Stereopsis: the concept The same 3D point projects onto two different pixels: the difference is the disparity, which is related to the depth of the point. Mobile HCI 2002

  9. Laser spot tracking • Red band-pass filter • Brightness normalization • Temporal averaging • Thresholding • Size filter • Epipolar constraint • Kalman filter Mobile HCI 2002

  10. Auditory display • Sonification: the most important visual objects are associated to sounds that “label” such objects. • Visual objects are: • The laser spot • Disparity blobs • For disparity blobs we sonify their area, using pitch and loudness. Mobile HCI 2002

  11. Spatialization: sounds are positioned in a virtual scenario, and displayed in 3D • Relative angular position • Relative distance Mobile HCI 2002

  12. Spatial attributes • Distance is conveyed by processing sounds as if they were located inside a tube • Angular position: • Interaural delay • Head diffraction (low pass) Mobile HCI 2002

  13. The prototype Mobile HCI 2002

  14. Work in progress • Empirical evaluation • Porting to PocketPC • Sketch based interface • Other sonification modes (global) • Looking for other interesting applications of cross-modality Mobile HCI 2002

  15. Questions? Mobile HCI 2002

  16. Brown and Duda model • Interaural distance  time delay • Head diffraction  low pass filter on the opposite hear • Brown&Duda, IEEE Tr. Speech and Audio, 6(5) 1998. Mobile HCI 2002