1 / 14

Enhancing Access to X Window Applications for the Visually Impaired through Auditory Interfaces

This project aims to develop auditory interfaces for X Window applications, providing blind or visually impaired users with transparent access to graphical interfaces. We will create a framework to monitor, model, and translate interfaces without altering the original applications. Our approach considers nonvisual interactions through auditory and tactile modalities, emphasizing low-cost solutions. By modeling interface structures and allowing users to navigate using a hierarchical tree structure, we ensure intuitive operation while integrating seamlessly with keyboard shortcuts.

lucie
Download Presentation

Enhancing Access to X Window Applications for the Visually Impaired through Auditory Interfaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mercator Mapping GUIs to Auditory Interfaces

  2. Goals • Provide transparent access to X Windows applications for computer users who are blind or severely visually impaired. • Build a framework to monitor, model, and translate graphical interfaces of X Window applications without modifying the application. • Develop a methodology for translating graphical interfaces into nonvisual interfaces.

  3. Mercator Design Considerations • Modality for nonvisual interface -- auditory or tactile. • Substantial research in auditory interfaces (Gaver, Bly, Blattner et al) • Users’ ability to monitor multiple auditory signals - the “cocktail party effect” (Cherry) • Active versus passive interaction • Low cost, standard audio devices • Users’ possible lack of tactile sensitivity due to diabetes

  4. Modeling Visual Interfaces

  5. Auditory Widgets • Correct level of abstraction • Convey objects … and attributes

  6. Navigation • Support two activities • Allow users to “scan” interface • Allow users to operate interface • Mouse navigation is unsuitable • Map interface structure into hierarchical tree structure • Based on widget hierarchy • Users walk tree structure to navigate • Works with existing keyboard short-cuts

  7. Mercator Architecture • Three Main Goals: • Capture -- high-level semantically meaningful information from X applications. • Store -- a semantic, off-screen model of the graphical interface. • Translate -- the stored information to present a consistent alternative interface.

  8. Spectrum of solutions • Modify the applications • Modify the toolkit • Interpose between application and window server

  9. First Architecture

  10. Second Architecture

  11. How this worked • Xt hooks, RAP, Xlib safety net • Control flow • Interface modeling • Interpreted interfaces • Simulating input

  12. How to do this now...

More Related