1 / 16

Sensory Transformation for the Blind: The vOICe Technology Explained

Learn about The vOICe, a groundbreaking technology by Peter B.L. Meijer that enables blind individuals to "see with their ears." Discover how multimedia computing facilitates audio-visual transformations, allowing the brain to translate video into sound. Explore the image-to-sound mapping process, frequency-time dynamics, and the potential of mental imagery. Delve into the limitations and implications for psychology, education, and neural pathways. Conclude with insights on the technology's affordability, non-invasiveness, and the need for further research. Dive into this innovative approach that offers a new perspective on visual perception.

neve-finley
Download Presentation

Sensory Transformation for the Blind: The vOICe Technology Explained

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. On cross-modal sensory streams... Invited Panel Presentation at SIGGRAPH '98, Orlando, Florida • The vOICe • by • Peter B.L. Meijer Philips Research Laboratories Peter.B.L.Meijer@philips.com

  2. The vOICe: “Seeing with your Ears” • Basic facts: • Blind people cannot see...but usually they can hear • Multimedia computing allowsaudio-visual transformations

  3. Video Audio MentalVideo The vOICe Human brain So translate video to audio!? The vOICe: “Seeing with your Ears” • Basic facts: • Blind people cannot see...but usually they can hear • Multimedia computing allowsaudio-visual transformations

  4. The vOICe implementations Java applet Hardware prototype Windows 95/98/NT http://www.seeingwithsound.com/

  5. The vOICe image-to-sound mapping Frequency Time Sum of sinusoids Amplitude

  6. The vOICe image-to-sound mapping Frequency Spectrographic synthesis Time

  7. The vOICe image-to-sound mapping Could this really work? Check: map sounds back to images Theory Yes!(atleast technically)

  8. The vOICe image reconstruction The vOICe “Soundscape” Spectrogram

  9. The vOICe image reconstruction “Soundscape”

  10. The vOICe and human speech Pronouncing one image per second

  11. The vOICe mental imagery? Representation may dramatically affect recognition!

  12. The vOICe limitations • Issues: • Frequency-Time uncertainty • Physiology / Perception: • just-noticeable difference (JND) • critical bands (human cochlea)

  13. The vOICe limitations • Issues: • Frequency-Time uncertainty • Physiology / Perception: • just-noticeable difference (JND) • critical bands (human cochlea) • auditory streaming/segregation • neural pathways & bandwidth • neural processing & plasticity

  14. The vOICe limitations • Psychology / Education: • sounds (too) unpleasant • minimum required results • acceptable training effort • motivation for learning • training programs • Issues: • Frequency-Time uncertainty • Physiology / Perception: • just-noticeable difference (JND) • critical bands (human cochlea) • auditory streaming/segregation • neural pathways & bandwidth • neural processing & plasticity

  15. Conclusions • The vOICe approach • needs more research • is cheap technology • is non-invasive

  16. Conclusions • The vOICe approach • needs more research • is cheap technology • is non-invasive • is available Now! http://www.seeingwithsound.com/

More Related