1 / 10

A multimodal dialogue-driven interface for accessing the content of recorded meetings

A multimodal dialogue-driven interface for accessing the content of recorded meetings. Agnes Lisowska ISSCO/TIM/ETI University of Geneva IM2.MDM

darrion-carry
Download Presentation

A multimodal dialogue-driven interface for accessing the content of recorded meetings

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A multimodal dialogue-driven interface for accessing the content of recorded meetings Agnes Lisowska ISSCO/TIM/ETI University of Geneva IM2.MDM Work done in collaboration with: Martin Rajman, Mirek Melichar, Trung H. Bui (LIA & CGC at the EPFL) and Marcin Bogobowicz (graphic artist)

  2. The data Audio and video Documents printed documents slides whiteboard activity handwritten notes Annotated transcriptions topics keywords argumentative structure The users Manager tracking project progress tracking employee performance Employee catch up on a missed meeting check facts learn about a project The Problem What type of interface?

  3. User-Centered Design (UCD) “The bottom line is that the tool should be designed around the needs of the user to allow for effective and efficient use of the application.”(Scanlon and Percival, IBM) Who will the users be? What tasks will users be doing? Which modalities? Which combinations? Which functionalities? Type of dialogue strategy? Degree of user control? Data annotation Types of data Data organization

  4. User Requirements • Requirements gathering must be an iterative process • requirements can emerge or change over time and with experience • User requirements gathering within IM2.MDM • Questionnaire-based study (Lisowska 2003) • collected ~300 queries to the system • analyzed linguistically and thematically • results used to derive user requirements and drive preliminary design of Archivus interface

  5. HCI and Design Principles in Archivus • Interaction metaphor • helps a user understand the various functionalities of a system • Multiple levels of interaction (user experience differs) • Visibility • important and frequently used elements always visible • Consistency • same elements in same location • same functionality performed in same way • Feedback to the user • user should see the results of their actions Novice user Average user Expert user System driven (dialogue system) Mixed initiative (dialogue system + user control) User driven

  6. Designed based on user requirements Interaction metaphor: using a library or archive library is the database of meetings each book in the library is a single meeting Preliminary version allows for many different modalities full range of interaction styles Modalities input natural language speech via speech recognition text via keyboard or pen pointing mouse touchscreen pen The Archivus System • output • audio • system prompts • meeting recordings • video • meeting recordings • text • graphics

  7. The Archivus System System control buttons Constant view of database (search space) System prompts Notebook Natural language input feedback Meeting ‘book’ Access to multimedia Interactive search definition buttons Interaction history

  8. Demo Problem: Denis told you that there was a discussion about red sofas for the ISSCO lounge and you want to know more about it. • All you know is: • Denis was at the meeting • the meeting was about ISSCO lounge furniture • the meeting was in early April in Martigny

  9. Testing and Evaluation • Wizard of Oz methodology • allows testing without full implementation • needs to be iterative • reveals • how modalities are used • new user requirements • flaws in the design

  10. References & Associated Papers User-Centered Design • Scanlon, J & Percival, L. UCD for different project types (Parts 1 & 2). IBM developerWorks website. www-106.ibm.com/developerworks/web/library/us-ucd/ User Requirements Studies • Lisowska A., Popescu-Belis A. & Armstrong S. (2004) - User Query Analysis for the Specification and Evaluation of a Dialogue Processing and Retrieval System. LREC 2004 (Fourth International Conference on Language Resources and Evaluation), Lisbon, Portugal, vol.III, p.993-996. • Lisowska, A. (2003) - Multimodal Interface Design for the Multimodal Meeting Domain: Preliminary Indications from a Query Analysis Study. Project Report IM2.MDM-11, November 2003, 30 pages. The Archivus Interface • Lisowska A., Rajman M., & Bui T.H. (2004) - ARCHIVUS : A System for Accesssing the Content of Recorded Multimodal Meetings. Proceedings of the Joint AMI/PASCAL/IM2/M4 Workshop on Multimodal Interaction and Related Machine Learning Algorithms, Martigny, Switzerland, 8 p. Dialogue Management • T. H. Bui, M. Rajman and M. Melichar. Rapid Dialogue Prototyping Methodology. Proceedings of the 7th International Conference on Text, Speech Dialogue (TSD 2004), P. Sojka, I. Kopecek K. Pala (Eds.), Lecture Notes in Computer Science, Springer-Verlag, Berlin Heidelberg New York, Brno, Czech Republic, September 8-11, 2004 • T. H. Bui and M. Rajman. Rapid Dialogue Prototyping Methodology. Technical Report No. 200401, Swiss Federal Institute of Technology (EPFL), Lausanne (Switzerland), January, 2004.

More Related