introduction to the trindikit
Download
Skip this Video
Download Presentation
Introduction to the TrindiKit

Loading in 2 Seconds...

play fullscreen
1 / 62

Introduction to the TrindiKit - PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on

Introduction to the TrindiKit. Dialogue Systems 2 GSLT spring 2003 Staffan Larsson [email protected] What is TrindiKit?. a toolkit for building and experimenting with dialogue move engines and systems, based on the information state approach not a dialogue system!. This lecture.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Introduction to the TrindiKit' - brenden-maynard


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
introduction to the trindikit

Introduction to the TrindiKit

Dialogue Systems 2

GSLT spring 2003

Staffan Larsson

[email protected]

what is trindikit

What is TrindiKit?

a toolkit for

building and experimenting with dialogue move engines and systems,

based on the information state approach

not a dialogue system!

this lecture
This lecture
  • architecture & concepts
  • what’s in TrindiKit?
  • extending TrindiKit
  • building a system
slide5

control

DME

module1

module…

modulei

modulej

module…

modulen

  • Total Information State
  • (TIS)
  • Information state proper (IS)
  • Module Interface Variables
  • Resource Interface Variables

resource1

resource…

resourcem

i nformation s tate is
Information State (IS)
  • an abstract data structure (record, DRS, set, stack etc.)
  • accessed by modules using conditions and operations
  • the Total Information State (TIS) includes
    • Information State proper (IS)
    • Module Interface variables
    • Resource Interface variables
dialogue move engine dme
Dialogue Move Engine (DME)
  • module or group of modules responsible for
    • updating the IS based on observed moves
    • selecting moves to be performed
  • dialogue moves are associated with IS updates using IS update rules
    • there are also update rules no directly associated with any move (e.g. for reasoning and planning)
  • update rules: rules for updating the TIS
    • rule name and class
    • preconditon list: conditions on TIS
    • effect list: operations on TIS
  • update rules are coordinated by update algorithms
modules and resources
Modules and resources
  • Modules (dialogue move engine, input, interpretation, generation, output etc.)
    • access the information state
    • no direct communication between modules
      • only via module interface variables in TIS
      • modules don’t have to know anything about other modules
      • increases modularity, reusability, reconfigurability
    • may interact with user or external processes
  • Resources (device interface, lexicons, domain knowledge etc.)
    • hooked up to the information state (TIS)
    • accessed by modules
    • defined as object of some type (e.g. ”lexicon”)
what does trindikit provide
What does TrindiKit provide?
  • High-level formalism and interpreter for implementing dialogue systems
    • promotes transparency, reusability, plug-and-play, etc.
    • allows implementation and comparison of dialogue theories
    • hides low-level software engineering issues
  • GUI, WWW-demo
  • Ready-made modules and resources
    • speech
    • interfaces to databases, devices, etc.
    • reasoning, planning
trindikit co ntents 1
TrindiKit contents (1)
  • alibrary of datatype definitions (records, DRSs, sets, stacks etc.)
    • user extendible
  • alanguage for writing information state update rules
  • GUI: methods and tools for visualising the information state
  • debugging facilities
    • typechecking
    • logs of communication modules-TIS
    • etc.
trindikit contents 2
TrindiKit contents (2)
  • A language for defining update algorithms used by TrindiKit modules to coordinate update rule application
  • A language for defining basic control structure, to coordinate modules
  • A library of basic ready-made modules for input/output, interpretation, generation etc.;
  • A library of ready-made resources and resource interfaces, e.g. to hook up databases, domain knowledge, devices etc.
special modules and resources included with trindikit
Special modules and resources included with TrindiKit
  • OAA interface resource
    • enables interaction with existing software and languages other than Prolog
  • Speech recognition and synthesis modules
    • TrindiKit shells for off-the-shelf recognisers
    • currently only ViaVoice, but more on the way
  • Possible future modules:
    • planning and reasoning modules
    • multimodal input and output
asynchronous trindikit
Asynchronous TrindiKit
  • Internal communication uses either
    • OAA (Open Agent Architecture) from SRI, or
    • AE (Agent Environment), a stripped-down version of OAA, implemented for TrindiKit
  • enables asynchronous dialogue management
    • e.g.: system can listen and interpret, plan the dialogue, and talk at the same time
how to use trindikit
How to use TrindiKit

We start from TrindiKit

Implements the information state approach

Takes care of low-level programming: dataflow, datastructures etc.

TrindiKit

information state

approach

how to build a basic system
How to build a basic system

Formulate a basic dialogue theory

Information state

Dialogue moves

Update rules

Add appropriate modules (speech recognition etc)

basic

dialoguetheory

basic system

TrindiKit

information state

approach

how to build a genre specific system
How to build a genre-specific system

Add genre-dependent IS components, moves and rules

genre-specific theory

additions

genre-specific system

basic

dialoguetheory

basic system

TrindiKit

information state

approach

how to build an application
How to build an application

Add application-specific resources

application

domain & language

resources

genre-specific theory

additions

genre-specific system

basic

dialoguetheory

basic system

TrindiKit

information state

approach

building a domain independent dialogue move engine
Building a domain-independent Dialogue Move Engine
  • Come up with a nice theory of dialogue
  • Formalise the theory, i.e. decide on
    • Type of information state (DRS, record, set of propositions, frame, ...)
    • A set of dialogue moves
    • Information state update rules, including rules for integrating and selecting moves
    • DME Module algorithm(s) and basic control algorithm
    • any extra datatypes (e.g. for semantics: proposition, question, etc.)
domain independence of the dialogue move engine
Domain independence of the Dialogue Move Engine
  • The DME is domain independent, given a certain type of dialogue
    • information-seeking
    • instructional
    • negotiative
    • ...
  • Domain independence of DME is not enforced by TrindiKit, but is good practice
    • promotes reuse of components
    • forces abstraction from domain-specific details, resulting in a more general theory of dialogue
specifying infostate type
Specifying Infostate type
  • the Total Information State contains a number of Information State Variables
    • IS, the Information State ”proper”
    • Interface Variables
      • used for communication between modules
    • Resource Variables
      • used for hooking up resources to the TIS, thus making them accessible from to modules
  • use prespecified or new datatypes
sample infostate type declaration
sample infostate type declaration

infostate_variable_of_type( is, IS ) :-

IS = record( [

private : Private,

shared : Shared ] ),

Shared = record( [

com : set( proposition ),

qud : stack( question ),

lm : set( move ) ] ),

Private = record( [

agenda: stack( action ),

plan : stackset( action ),

bel : set( proposition ),

tmp : Shared ] ) ] ).

resulting infostate type
resulting infostate type

AGENDA : stack( Action )

PLAN : stackset( Action )

PRIVATE :

BEL : set( Prop )

TMP : (same type as SHARED)

COM : set( Prop )

QUD : stack( Question )

SHARED :

LU: [SPEAKER:dp,

MOVES:set( Move )]

sample interface variable type declarations
Sample interface variable type declarations

interface_variable_of_type( input, string ).

interface_variable_of_type( output, string ).

interface_variable_of_type( latest_speaker, speaker ).

interface_variable_of_type( latest_moves, set(move) ).

interface_variable_of_type( next_moves, set(move) ).

specifying a set of moves
Specifying a set of moves
  • amounts to specifying objects of type move (a reserved type)
    • there may be type constraints on the arguments of moves
  • preconditions and effects of moves
    • formalised in update rules, not in the move definition itself
    • a move may have different effects on the IS depending e.g. on who performed it
sample move specifications
sample move specifications

% Social

of_type( quit, move ).

of_type( greet, move ).

of_type( thank, move ) .

% Q&A

of_type( ask(Q), move ) <- of_type( Q, question ).

of_type(inform(P), move ) <- of_type( P, proposition).

of_type( answer(R), move ) <-

of_type( R, proposition) or

of_type( R, ellipsis ).

writing rules
Writing rules
  • rule = conditions + updates
    • if the rule is applied to the IS and its conditions are true, the operations will be applied to the IS
    • conditions may bind variables with scope over the rule (prolog variables, with unification and backtracking)
a sample rule
A sample rule

rule( integrateUsrAnswer, [

$/shared/lu/speaker = usr,

assoc( $/shared/lu/moves, answer(R), false ),

fst( $/shared/qud, Q ),

$domain : relevant_answer( Q, R ),

$domain : reduce(Q, R, P)

], [

set_assoc( /shared/lu/moves, answer(R),true),

shared/qud := $$pop( $/shared/qud ),

add( /shared/com, P ) ] ).

a sample rule old syntax
A sample rule (old syntax)
  • rule( integrateUsrAnswer, [
    • val#rec( shared^lu^speaker, usr ),
    • assoc#rec( shared^lu^moves, answer( R ), false ),
    • fst#rec( shared^qud, Q ),
    • domain :: relevant_answer( Q, R ),
    • domain :: reduce(Q, R, P)
    • ], [
    • set_assoc#rec( shared^lu^moves, answer(R),true),
    • pop#rec( shared^qud ),
    • add#rec( shared^com, P ) ] ).
writing rules1
Writing rules
  • available conditions and operations depend on the infostate type
    • the infostate is declared to be of a certain (usually complex) type
  • datatype definitions provide
    • selectors: Sel(InObj,SelObj)
    • relations:Rel(Arg1, …, ArgN)
    • functions:Fun(Arg1, …, ArgN,Result)
    • operations:

Op(ObjIn,Arg1, …, ArgN,ObjOut)

  • New datatypes may be added
writing rules locations in tis
Writing rules: locations in TIS
  • objects may be specified by giving a path to a locationin the infostate;
    • paths are specified using selectors, which are similar to functions
      • $$Sel2($$Sel1) ~ $Sel1/Sel2
      • $$fst($/shared/qud) ~ $/shared/qud/fst
    • ”$” evaluates a path and gives the object at the location specified
    • ”$$” evaluates a function
    • $$fst($/shared/qud) = $/shared/qud/fst
  • example:
    • is/shared/com is a path, pointing to a location in the TIS
    • $is/shared/com is the object in that location
    • theiscan be left out, giving$/shared/com
writing rules conditions 1
Writing rules: conditions (1)
  • conditions do not change the information state
  • if a condition fails, backtracking ensues
  • condition syntax (incomplete)
    • Rel(Arg1, … , ArgN), e.g.
      • fst($/shared/qud,Q)
    • Arg1:Rel(Arg2,…,ArgN), e.g.
      • $/shared/qud:fst(Q)
      • $domain:relevant_answer(Q,A)
    • Arg1 = Arg2
      • Q = $$fst($/shared/qud)
    • Cond1 and Cond2
    • Cond1 or Cond2
    • not Cond1
    • forall(Cond1, Cond2)
    • (Argis object or prolog variable)
writing rules conditions 2
Writing rules: conditions (2)
  • quantification, binding and backtracking
    • if an instantiation a of a variable V in a condition C is found that makes condition C true, V is bound to a
    • backtracking occurs until a successful instantiation of all variables in the list of conditions has been found
  • example list of conditions

fst($/shared/qud,Q),

in($/shared/com,P),

$domain:relevant_answer(P,Q)

  • Explicit quantification

Q.P. fst($/shared/qud,Q) & in($/shared/com,P) & $domain:relevant_answer(P,Q)

writing rules updates
Writing rules: updates
  • operations change the information state
  • if an operation fails, an error is reported
  • variable bindings survive from conditions to operations
  • operation syntax (incomplete)
    • Op(Path,Arg1,…,ArgN)
      • push(/shared/qud, Q)
    • Path : Op(Arg1, … ,ArgN)
      • /shared/qud : push(Q)
    • Store := Fun(Obj,Arg1,…,ArgN)
      • /private/tmp/qud := $$push($/shared/qud,Q)
specifying update algorithms
Specifying update algorithms
  • uses rule classes
  • constructs include
    • Rule
    • RuleClass
    • if Cond then S else T
    • repeat R until C
    • repeat R
    • try R
    • R orelse S
    • test C
    • SubAlgorithm
sample update algorithm
Sample update algorithm

grounding,

if $latest_speaker == sys

then

try integrate,

try database,

repeat downdate_agenda,

store

else

repeat

integrate

orelse accommodate

orelse find_plan

orelse

if (empty ( $/private/agenda ) then manage_plan

else downdate_agenda

repeat downdate_agenda

if empty($/private/agenda))

then repeat manage_plan

repeat refill_agenda

repeat store_nim

try downdate_qud

specifying serial control algorithms
Specifying serial control algorithms
  • serial constructs include
    • Module{:Algorithm}
    • if Cond then S else T
    • repeat R until C
    • repeat R
    • try R
    • R orelse S
    • test C
    • SubAlgorithm
specifying concurrent control algorithms
Specifying concurrent control algorithms
  • Agent1 | Agent2 | … | AgentN
  • whereAgenti is
  • AgentName : {
    • import Module1 ,
    • import Modulep ,
    • Trigger1 => SerialAlgoritm1 ,
    • Triggerm => SerialAlgoritmm }
  • triggers:
    • condition(C) (C is a subset of the full condition set)
    • init
    • new_data(Stream)
sample control algorithm 1
Sample control algorithm (1)

repeat ( [

select,

generate,

output,

update,

test( $program_state == run ),

input,

interpret,

update ] )

sample control algorithm 2
Sample control algorithm (2)

input: {

init => input:display_prompt,

new_data(user_input) => input }

| interpretation: {

import interpret,

condition(is_set(input)) => [ interpret, print_state ] }

| dme: {

import update,

import select,

init => [ select ],

condition(not empty(latest_moves)) => [

update,

if $latest_speaker == usr then select

] }

| generation: {

condition(is_set(next_moves)) => generate }

| output: {

condition(is_set(output)) => output } )).

from dme to dialogue system
From DME to dialogue system

Build or select from existing components:

  • Modules, e.g.
    • input
    • interpretation
    • generation
    • output
  • Still domain independent
  • the choice of modules determines e.g. the format of the grammar and lexicon
domain specific system
Domain-specific system

Build or select from existing components:

  • Resources, e.g.
    • domain (device/database) interface
    • dialog-related domain knowledge, e.g. plan libraries etc.
    • grammars, lexicons
you can add
You can add
  • Datatypes
    • Whatever you need
  • Modules
    • e.g. General interfaces to speech recognizers and synthesizers
  • Resources
    • E.g. General interfaces to (passive) devices
  • Important that all things added are reasonably general, so they can be reused in other systsems
datatype definitions
Datatype definitions
  • relations
    • relations between objects; true or false
    • format: relation(Rel,Args).
    • Example
      • definition: relation(fst,[stack([E|S]),E]).
      • condition: fst($/shared/qud,Q)
datatype definitions1
Datatype definitions
  • functions
    • functions from arguments to result
    • format: function(Fun,Args,Result).
    • Example
      • definition: function(fst,[stack([E|S])],E).
      • in condition:
        • Q = $$fst($/shared/qud)
        • Q = $/shared/qud/fst
      • in effect:
        • next_move/content := $$fst($/shared/qud)
    • every function corresponds to a relation relation(Fun,[[email protected][Result]]).
datatype definitions 2
Datatype definitions (2)
  • selectors
    • selects an object (Obj) embedded in another object (Arg)
    • selector(Sel,Arg,Obj,ArgWithHole,Hole).
    • e.g. selector(fst,stack([E|S]),E,stack([H|S]),H).
    • Every selector corresponds to a function function(Sel,[Arg],Object).
datatype definitions 3
Datatype definitions (3)
  • operations
    • operation(Op,InObj,Args,OutObj).
    • e.g. operation(push,stack(S),[E],stack([E|S])).
    • every operation corresponds to a relation relation(Op,[InObj|Args]@[OutObj]).
    • push($/shared/qud,Q,$/shared/qud).
building modules
Building modules
  • DME modules
    • Specific to a certain theory of dialogue management
    • Best implemented using rules and algorithms
  • Other modules
    • Should be more general, less specific to certain theory of dialogue management
    • May be easier to implement directly in prolog or other language
      • DME-ADL only covers checking and updating the infostate
      • These modules may also need to interact with other programs or devices
building resources
Building resources
  • Resource
    • the resource itself; exports a set of predicates
  • Resource interface
    • defines the resource as a datatype T, i.e. in terms of relations, functions and operations
  • Resource interface variable
    • a TIS variable whose value is an object of the type T
  • By changing the value of the variable, resources can be switched dynamically
    • change laguage
    • change domain
sample resource variable type declarations incl resource interface
sample resource variable type declarations (incl. resource interface)

resource_type( lexiconT ).

resource_variable_of_type( lexicon, lexiconT ).

of_type( lexicon_travel_english, lexiconT ).

of_type( lexicon_autoroute_english, lexiconT ).

of_type( lexicon_travel_svenska, lexiconT ).

of_type( lexicon_cellphone_svenska, lexiconT ).

resource_condition(lexiconT,input_form(Phrase,Move),Lexicon) :- Lexicon : input_form( Phrase, Move ).

resource_condition(lexiconT,output_form(Phrase,Move),Lexicon):-

Lexicon : output_form( Phrase, Move ).

trindikit f eatures
TrindiKit features
  • explicit information state datastructure
    • makes systems more transparent
    • enable e.g. context sensitive interpretation, distributed decision making, asynchronous interaction
  • update rules
    • provide an intuitive way of formalising theories in a way which can be used by a system
    • represent domain-independent dialogue management strategies
trindikit features cont d
TrindiKit features cont’d
  • resources
    • represent domain-specific knowledge
    • can be switched dynamically
      • e.g. switching language on-line in GoDiS
  • modular architecture promotes reuse
    • basic system -> genre-specific systems
    • genre-specific system -> applications
theoretical advantages of trindikit
Theoretical advantages of TrindiKit
  • theory-independent
    • allows implementation and comparison of competing theories
    • promotes exploration of middle ground between simplistic and very complex theories of dialogue
  • intuitive formalisation and implementation of dialogue theories
    • the implementation is close to the theory
practical advantages of trindikit
Practical advantages of TrindiKit
  • promotes reuse and reconfigurability on multiple levels
  • general solutions to general phenomena enables rapid prototyping of applications
  • allows dealing with more complex dialogue phenomena not handled by current commercial systems
availability
availability
  • version 3.0a avaliable!
    • SIRIDUS deliverable D6.4
  • TrindiKit website
    • www.ling.gu.se/projects/trindi/trindikit
  • SourceForge project
    • development versions available
    • developer community?
  • licensed under GPL
  • more info in
    • Larsson & Traum: NLE Special Issue on Best Practice in Dialogue Systems Design, 2000
    • TrindiKit manual (available from website)
conclusions trindikit information state approach
Conclusions: TrindiKit & Information State Approach
  • a toolkit for dialogue systems R&D
  • freely available to researchers
  • close the gap between theory and practive of dialogue systems
  • theory-independent
  • promotes reuse and reconfigurability on multiple levels
trindikit and voicexml
TrindiKit and VoiceXML
  • VoiceXML
    • industry standard
    • form-based dialogue manager
    • web/telephony infrastructure
    • requires scripting dialogues in detail
  • Theory-specific?
    • VoiceXML implements a specific theory of dialogue
    • TrindiKit allows implementation of several different theories of dialogue
    • More complex dialogue phenomena hard to deal with in VoiceXML
trindikit and voicexml cont d
TrindiKit and VoiceXML, cont’d
  • Combine VoiceXML with TrindiKit?
    • future research area
    • support connecting TrindiKit to VoiceXML infrastructure
    • use TrindiKit system as VoiceXML server, dynamically building VoiceXML scripts
    • convert VoiceXML specifications to e.g. GoDiS dialogue plans
possible future modifications
Possible future modifications
  • regler i klasser väljs indeterministiskt
  • test C -> if C then halt / break (gå ur loop)
  • ta bort möjlighet att ha flera regler med samma namn
ad