A multimodal dialogue-driven interface for accessing the content of recorded meetings
This presentation is the property of its rightful owner.
Sponsored Links
1 / 10

A multimodal dialogue-driven interface for accessing the content of recorded meetings PowerPoint PPT Presentation


  • 41 Views
  • Uploaded on
  • Presentation posted in: General

A multimodal dialogue-driven interface for accessing the content of recorded meetings. Agnes Lisowska ISSCO/TIM/ETI University of Geneva IM2.MDM

Download Presentation

A multimodal dialogue-driven interface for accessing the content of recorded meetings

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


A multimodal dialogue driven interface for accessing the content of recorded meetings

A multimodal dialogue-driven interface for accessing the content of recorded meetings

Agnes Lisowska

ISSCO/TIM/ETI University of Geneva

IM2.MDM

Work done in collaboration with: Martin Rajman, Mirek Melichar, Trung H. Bui (LIA & CGC at the EPFL) and Marcin Bogobowicz (graphic artist)


The problem

The data

Audio and video

Documents

printed documents

slides

whiteboard activity

handwritten notes

Annotated transcriptions

topics

keywords

argumentative structure

The users

Manager

tracking project progress

tracking employee performance

Employee

catch up on a missed meeting

check facts

learn about a project

The Problem

What type of interface?


User centered design ucd

User-Centered Design (UCD)

“The bottom line is that the tool should be designed around the needs of the user to allow for effective and efficient use of the application.”(Scanlon and Percival, IBM)

Who will the users be?

What tasks will users be doing?

Which modalities? Which combinations?

Which functionalities?

Type of dialogue strategy?

Degree of user control?

Data annotation

Types of data

Data organization


User requirements

User Requirements

  • Requirements gathering must be an iterative process

    • requirements can emerge or change over time and with experience

  • User requirements gathering within IM2.MDM

    • Questionnaire-based study (Lisowska 2003)

      • collected ~300 queries to the system

      • analyzed linguistically and thematically

      • results used to derive user requirements and drive preliminary design of Archivus interface


Hci and design principles in archivus

HCI and Design Principles in Archivus

  • Interaction metaphor

    • helps a user understand the various functionalities of a system

  • Multiple levels of interaction (user experience differs)

  • Visibility

    • important and frequently used elements always visible

  • Consistency

    • same elements in same location

    • same functionality performed in same way

  • Feedback to the user

    • user should see the results of their actions

Novice user

Average user

Expert user

System driven (dialogue system)

Mixed initiative (dialogue system + user control)

User driven


The archivus system

Designed based on user requirements

Interaction metaphor: using a library or archive

library is the database of meetings

each book in the library is a single meeting

Preliminary version allows for

many different modalities

full range of interaction styles

Modalities

input

natural language

speech via speech recognition

text via keyboard or pen

pointing

mouse

touchscreen

pen

The Archivus System

  • output

    • audio

      • system prompts

      • meeting recordings

    • video

      • meeting recordings

    • text

    • graphics


The archivus system1

The Archivus System

System control buttons

Constant view of database (search space)

System prompts

Notebook

Natural language input feedback

Meeting ‘book’

Access to multimedia

Interactive search definition buttons

Interaction history


A multimodal dialogue driven interface for accessing the content of recorded meetings

Demo

Problem: Denis told you that there was a discussion about red sofas for the ISSCO lounge and you want to know more about it.

  • All you know is:

    • Denis was at the meeting

    • the meeting was about ISSCO lounge furniture

    • the meeting was in early April in Martigny


Testing and evaluation

Testing and Evaluation

  • Wizard of Oz methodology

    • allows testing without full implementation

    • needs to be iterative

    • reveals

      • how modalities are used

      • new user requirements

      • flaws in the design


References associated papers

References & Associated Papers

User-Centered Design

  • Scanlon, J & Percival, L. UCD for different project types (Parts 1 & 2). IBM developerWorks website.

    www-106.ibm.com/developerworks/web/library/us-ucd/

    User Requirements Studies

  • Lisowska A., Popescu-Belis A. & Armstrong S. (2004) - User Query Analysis for the Specification and Evaluation of a Dialogue Processing and Retrieval System. LREC 2004 (Fourth International Conference on Language Resources and Evaluation), Lisbon, Portugal, vol.III, p.993-996.

  • Lisowska, A. (2003) - Multimodal Interface Design for the Multimodal

    Meeting Domain: Preliminary Indications from a Query Analysis Study. Project Report IM2.MDM-11, November 2003, 30 pages.

    The Archivus Interface

  • Lisowska A., Rajman M., & Bui T.H. (2004) - ARCHIVUS : A System for Accesssing the Content of Recorded Multimodal Meetings. Proceedings of the Joint AMI/PASCAL/IM2/M4 Workshop on Multimodal Interaction and

    Related Machine Learning Algorithms, Martigny, Switzerland, 8 p.

    Dialogue Management

  • T. H. Bui, M. Rajman and M. Melichar. Rapid Dialogue Prototyping Methodology. Proceedings of the 7th International Conference on Text, Speech Dialogue (TSD 2004), P. Sojka, I. Kopecek K. Pala (Eds.), Lecture Notes in Computer Science, Springer-Verlag, Berlin Heidelberg New York, Brno, Czech Republic, September 8-11, 2004

  • T. H. Bui and M. Rajman. Rapid Dialogue Prototyping Methodology. Technical Report No. 200401, Swiss Federal Institute of Technology (EPFL), Lausanne (Switzerland), January, 2004.


  • Login