1 / 6

Multimodal

Multimodal. Interests. Incremental understanding and feedback Mobile multimodal systems Producing and understanding nonverbal communication (e.g. facial expression) Situated interaction Multiple channels to increase robustness Web-based systems How do gestures contribute to interaction?

illana-waller
Download Presentation

Multimodal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multimodal

  2. Interests • Incremental understanding and feedback • Mobile multimodal systems • Producing and understanding nonverbal communication (e.g. facial expression) • Situated interaction • Multiple channels to increase robustness • Web-based systems • How do gestures contribute to interaction? • Gesture production for Embodied Conversational Agents • Automotive environments • Balance between gesture and speech input • Learning from multimodal feedback • Evaluation, usability – does it help? • Next killer app? 2 years? 5 years? 10 years?

  3. 2 groups • 2 major threads of interest: • Group 1: Study how humans interact with each other multimodaly • Group 2: Study how to build multimodal interfaces now • How should we inform each others’ research? • 1-> 2 Alignment for multimodal fusion • 2->1 What is useful to study, naturalness isn’t everything

  4. Resources • Markup languages • EMMA, SALT, XHTML+Voice, SMIL • BML, FML, CML: behavorial & functional markup languages • We should get involved with standards • Platform trajectory • Step 1: deal with timing issues on multimodal alignment • Step 2: Future platforms

  5. Other Communities • Human robot interaction • Autonomous agents • Embodied Conversational Agents • Psycholinguistics • Vision, other sensors

  6. Now, 2 years, 5 years, … • Now • Using GUI for fallback strategies (error correction, text input) • Asynchronous input (e.g. pen input) • Eyes busy scenarios (automotive) • Soon • Presence-aware advertising and interaction • Robots

More Related