Interactive dialogue for simulation with virtual characters
Download
1 / 24

Interactive Dialogue for Simulation with Virtual Characters - PowerPoint PPT Presentation


  • 131 Views
  • Uploaded on

Interactive Dialogue for Simulation with Virtual Characters. David Traum traum @ ict.usc.edu http://www.ict.usc.edu/~traum. ICT Dialogue Group - Fall 2006.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Interactive Dialogue for Simulation with Virtual Characters' - azalia-walton


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Interactive dialogue for simulation with virtual characters

Interactive Dialoguefor Simulation with Virtual Characters

David Traum

traum @ ict.usc.edu

http://www.ict.usc.edu/~traum


Ict dialogue group fall 2006
ICT Dialogue Group - Fall 2006

  • Staff: Dr David Traum (CS) Dr Anton Leuski (CS), Dr Bilyana Martinovski (Ling),Dr Mark Core (CS), Susan Robinson (Ling), Jillian Gerten (Ling)

  • CS Students: Antonio Roque, Sudeep Gandhe, Dusan Jan, Ashish Vaswani

  • Temporary visitors and Interns

  • Close collaboration with other groups at ICT, ISI, Campus, others


Dialogue terms
Dialogue: Terms

  • Dialogue

    • Communication involving:

      • Multiple contributions

      • Coherent Interaction

      • More than one participant

  • Dialogue System

    • System that engages in dialogue with users and/or other systems

  • Dialogue Model

    • Formal characterization of dialogue, evolving context, and possible/likely continuations

  • Dialogue Manager

    • Module of a system concerned with dialogue modelling and decisions of how to contribute to dialogue


Dialogue management tasks
Dialogue Management Tasks

  • Maintaining & Updating Context

  • Deciding what to say next

  • Interface with non-communicative reasoning aspects of agent

  • Provide expectations for interpretation


Dialogue research
Dialogue Research

  • Study of human conversational behavior

    • Pragmatic meaning

    • Representation and use of context

    • Understanding and Generation of communicative behavior

    • Interactional patterns

  • Methods

    • Empirical investigation

    • Theory development

    • Construction of dialogue systems

    • Empirical evaluation

  • Construction of conversational agents

    • Ability to converse and act in a specific domain

    • Integration with other abilities (perception, cognition, emotion)


Dialogue model
Dialogue Model

  • Representation of context of interaction including

    • Agent’s internal state

    • World state

    • State of others

    • Social relationships

    • interaction

  • Model of context-changing actions

    • Observation of communication

    • Performance of communication

    • Other observations

    • Thinking/reasoning


Information state approach to computational dialogue modeling
Information-State Approach to Computational Dialogue Modeling

(Larsson& Traum 2000, Traum &Larsson 2003)

  • Information Specification

  • Dialogue Moves for Update

  • Behaviors Dialogue moves

  • Toolkits for easy implementation (TrindiKit, dipper, Midiki, USC Steve/Austin)

    • Modular approach to sub-components

    • More direct relation to theory

    • Modular approach to theory


Dialogue approach layered information state

Dialogue Modeling

Acts

Dialogue

Acts

Dialogue Approach:Layered Information State

  • Layer captures coherent aspect of communicative interaction (e.g., turn, grounding, obligations)

  • Layer consists of

    • Information State components (state of interaction)

    • Dialogue Acts (Packages of changes to information state)

Input

Utterance

Recognition Rules

Update Rules

Info State

Components

Selection Rules

Output Utterance

(verbal and nonverbal)

Realization Rules

Dialogue Manager


Methodology
Methodology Modeling

Implement on the basis of model

Build models

rules

Model

Study

Build

Acquire

data

Test

Discover gaps in data

Design evaluation


Recent ict dialogue agents
Recent ICT Dialogue Agents Modeling

  • Embodied

    • Question-answering characters

      • Be a reporter

      • Sgt Blackwell

      • Tactical Questioning for Army

      • Mediation for interviews (AXL)

    • Virtual Humans for complex role-play interaction

      • MRE - characters for multi-party collaboration

      • SASO - non-team negotiation

    • Group conversation

  • Non-embodied

    • Transonics Farsi-English spoken translation for medical interviews

    • Radiobots - military radio operators for training in VR simulation


Example 1 sgt blackwell
Example 1: Sgt Blackwell Modeling

  • Focus: technology demo

  • Highlights:

    • Life-sized, mixed reality

      • Trans-screen

    • High-production quality

      • Rendering (> 60K polygons)

      • Voice

      • Authored Text

      • Robust responsiveness

        • Speech recognition and speech and non-verbal reply

    • Limited domain of interaction: responding to interview/Q&A



Sgt blackwell dialogue model
Sgt Blackwell “Dialogue Model” Modeling

  • Set of pre-constructed answers

    • In domain

    • Off-topic

    • Prompt

  • Local history

  • IR-based classification

    • Given possibly unseen question, map to best answer


Ict virtual human domains

MRE Modeling

SASO-ST

ICT Virtual Human Domains


Immersive training environment
Immersive Training Environment Modeling

  • Mission Rehearsal Exercise(Swartout et al ‘01)

    • Human lieutenant (student) faces peacekeeping dilemmas

      • Appears in video offsceen

    • Artificial agents interact with user

      • Mentor (e.g., sergeant, front left)

      • Teammates (e.g., medic, front right)

      • Locals (e.g., mother, front center)

  • VR Theatre

    • 8’ 150˚ Curved Screen,

    • Multiple Projectors

    • 10-2 3-d spatialized sound


Mre dialogue layers traum rickel aamas 2002

Communication Modeling

Contact

Attention

Conversation

Participants

Turn

Initiative

Grounding

Purpose

Rhetorical

Social

Obligations-Commitments

Negotiation-Collaboration

Social Roles

Individual

Perception

Rational

belief,desire, intention,..

Emotional

Coping strategies

MRE Dialogue Layers (Traum & Rickel AAMAS 2002)


Grounding clark schaeffer 89 traum 94
Grounding Modeling (Clark & Schaeffer 89, Traum 94)

  • Signal by B of how s is interpreted

  • Possibly revised signals by A

  • Grounded state = mutually believed common ground



Secure Area Modeling

Secure 12-4

Secure 8-12

Render Aid

Secure 4-8

Secure LZ

Secure Accident

Medevac

Sgt’s Negotiation Behavior

1

Focus=1

Lt: U9 “secure a landing zone”

Committed(lt,7,sgt), 7 authorized, Obl(sgt,U9)

Sgt: U10 “first we should secure the assembly area”

Disparaged(sgt, 7,lt), endorsed(sgt,2.lt), grounded(U9)

Lt: U11“secure the area”

Committed(lt,2,sgt), 2 authorized, Obl(sgt,U11),grounded(U10)

Sgt: U12“yes sir”

Committed(sgt,2,lt), grounded(U11), Push(2,focus)

Goal7:Announce(2,{1sldr,2sldr,3sldr,4sldr})

Goal8: Start-conversation(sgt, ,{1sldr,2sldr,…},2)

Goal8 ->Sgt: U21 “Squad leaders listen up!”

Goal7 ->Sgt:U22 “I want 360 degree security”

Committed(sgt,2,{1sldr,2sldr,3sldr,4sldr})

Push(3, focus)

Goal9:authorize 3

Goal9 ->Sgt:U23“1st squad take 12-4”

Committed(sgt,3, {1sldr,2sldr,3sldr,4sldr}), 3 authorized

Pop(3), Push(4)

Goal10: authorize 4

Goal10 ->Sgt: U24“2nd squad take 4-8”

Committed(sgt,4,{1sldr,2sldr,3sldr,4sldr}), 4 authorized

Pop(4)

A10: Squads move

Grounded(U21-U26)

ends conversation about 2, Happened(2)

Push(7,Focus)

Decomposition

Area Secure

Squads in area

A=Lt, R=Sgt

A=Lt ,R=S

2

7

Decomposition

3

4

A=Sgt,R=1sldr

A=Sgt,R=2sldr

5

6

A=Sgt,R=3sldr

A=Sgt,R=4sldr



Saso st video
SASO-ST video Modeling


Negotiation strategies traum et al iva 2005
Negotiation Strategies Modeling (Traum et al IVA 2005)

  • Result from orientations toward negotiation:

    • Avoidance

      • avoid

    • Distributive

      • attack

    • Integrative

      • negotiate

  • Govern choice of dialogue move, posture, and interpretation


Current research topics
Current Research Topics Modeling

  • Dialogue genre taxonomy

    • How is language use similar and different depending on aspects of Activity (participants, culture, goals, roles, setting,..)

  • Multi-party, multi-modal dialogue

  • Algorithms for dialogue processing

    • Multiple approaches (theory and data-driven)

  • Strategies for dialogue management

  • Parameterizable dialogue models

    • Personality models

    • Culture-specific aspects of production and interpretation

  • Affective aspects of dialogue

    • Immersion/engagement

    • Effects of emotion/coping on processing

    • Emotion-oriented language (e.g., empathy, mitigation, boosting)


Opportunities
Opportunities Modeling

  • Come talk to our agents at ICT

    • Contact [email protected]

  • ICT Summer internship program

    • Apply on ICT webpage www.ict.usc.edu

      (specify areas of interest)

  • Possibilities for directed research

    • Inquire [email protected]


ad