Introduction to the trindikit
This presentation is the property of its rightful owner.
Sponsored Links
1 / 62

Introduction to the TrindiKit PowerPoint PPT Presentation


  • 76 Views
  • Uploaded on
  • Presentation posted in: General

Introduction to the TrindiKit. Dialogue Systems 2 GSLT spring 2003 Staffan Larsson [email protected] What is TrindiKit?. a toolkit for building and experimenting with dialogue move engines and systems, based on the information state approach not a dialogue system!. This lecture.

Download Presentation

Introduction to the TrindiKit

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Introduction to the trindikit

Introduction to the TrindiKit

Dialogue Systems 2

GSLT spring 2003

Staffan Larsson

[email protected]


What is trindikit

What is TrindiKit?

a toolkit for

building and experimenting with dialogue move engines and systems,

based on the information state approach

not a dialogue system!


This lecture

This lecture

  • architecture & concepts

  • what’s in TrindiKit?

  • extending TrindiKit

  • building a system


Architecture concepts

Architecture & concepts


Introduction to the trindikit

control

DME

module1

module…

modulei

modulej

module…

modulen

  • Total Information State

  • (TIS)

  • Information state proper (IS)

  • Module Interface Variables

  • Resource Interface Variables

resource1

resource…

resourcem


I nformation s tate is

Information State (IS)

  • an abstract data structure (record, DRS, set, stack etc.)

  • accessed by modules using conditions and operations

  • the Total Information State (TIS) includes

    • Information State proper (IS)

    • Module Interface variables

    • Resource Interface variables


Dialogue move engine dme

Dialogue Move Engine (DME)

  • module or group of modules responsible for

    • updating the IS based on observed moves

    • selecting moves to be performed

  • dialogue moves are associated with IS updates using IS update rules

    • there are also update rules no directly associated with any move (e.g. for reasoning and planning)

  • update rules: rules for updating the TIS

    • rule name and class

    • preconditon list: conditions on TIS

    • effect list: operations on TIS

  • update rules are coordinated by update algorithms


Modules and resources

Modules and resources

  • Modules (dialogue move engine, input, interpretation, generation, output etc.)

    • access the information state

    • no direct communication between modules

      • only via module interface variables in TIS

      • modules don’t have to know anything about other modules

      • increases modularity, reusability, reconfigurability

    • may interact with user or external processes

  • Resources (device interface, lexicons, domain knowledge etc.)

    • hooked up to the information state (TIS)

    • accessed by modules

    • defined as object of some type (e.g. ”lexicon”)


What s in trindikit

What’s in TrindiKit?


What does trindikit provide

What does TrindiKit provide?

  • High-level formalism and interpreter for implementing dialogue systems

    • promotes transparency, reusability, plug-and-play, etc.

    • allows implementation and comparison of dialogue theories

    • hides low-level software engineering issues

  • GUI, WWW-demo

  • Ready-made modules and resources

    • speech

    • interfaces to databases, devices, etc.

    • reasoning, planning


Trindikit co ntents 1

TrindiKit contents (1)

  • alibrary of datatype definitions (records, DRSs, sets, stacks etc.)

    • user extendible

  • alanguage for writing information state update rules

  • GUI: methods and tools for visualising the information state

  • debugging facilities

    • typechecking

    • logs of communication modules-TIS

    • etc.


Trindikit contents 2

TrindiKit contents (2)

  • A language for defining update algorithms used by TrindiKit modules to coordinate update rule application

  • A language for defining basic control structure, to coordinate modules

  • A library of basic ready-made modules for input/output, interpretation, generation etc.;

  • A library of ready-made resources and resource interfaces, e.g. to hook up databases, domain knowledge, devices etc.


Special modules and resources included with trindikit

Special modules and resources included with TrindiKit

  • OAA interface resource

    • enables interaction with existing software and languages other than Prolog

  • Speech recognition and synthesis modules

    • TrindiKit shells for off-the-shelf recognisers

    • currently only ViaVoice, but more on the way

  • Possible future modules:

    • planning and reasoning modules

    • multimodal input and output


Asynchronous trindikit

Asynchronous TrindiKit

  • Internal communication uses either

    • OAA (Open Agent Architecture) from SRI, or

    • AE (Agent Environment), a stripped-down version of OAA, implemented for TrindiKit

  • enables asynchronous dialogue management

    • e.g.: system can listen and interpret, plan the dialogue, and talk at the same time


How to build a system

How to build a system


How to use trindikit

How to use TrindiKit

We start from TrindiKit

Implements the information state approach

Takes care of low-level programming: dataflow, datastructures etc.

TrindiKit

information state

approach


How to build a basic system

How to build a basic system

Formulate a basic dialogue theory

Information state

Dialogue moves

Update rules

Add appropriate modules (speech recognition etc)

basic

dialoguetheory

basic system

TrindiKit

information state

approach


How to build a genre specific system

How to build a genre-specific system

Add genre-dependent IS components, moves and rules

genre-specific theory

additions

genre-specific system

basic

dialoguetheory

basic system

TrindiKit

information state

approach


How to build an application

How to build an application

Add application-specific resources

application

domain & language

resources

genre-specific theory

additions

genre-specific system

basic

dialoguetheory

basic system

TrindiKit

information state

approach


Building a domain independent dialogue move engine

Building a domain-independent Dialogue Move Engine

  • Come up with a nice theory of dialogue

  • Formalise the theory, i.e. decide on

    • Type of information state (DRS, record, set of propositions, frame, ...)

    • A set of dialogue moves

    • Information state update rules, including rules for integrating and selecting moves

    • DME Module algorithm(s) and basic control algorithm

    • any extra datatypes (e.g. for semantics: proposition, question, etc.)


Domain independence of the dialogue move engine

Domain independence of the Dialogue Move Engine

  • The DME is domain independent, given a certain type of dialogue

    • information-seeking

    • instructional

    • negotiative

    • ...

  • Domain independence of DME is not enforced by TrindiKit, but is good practice

    • promotes reuse of components

    • forces abstraction from domain-specific details, resulting in a more general theory of dialogue


Specifying infostate type

Specifying Infostate type

  • the Total Information State contains a number of Information State Variables

    • IS, the Information State ”proper”

    • Interface Variables

      • used for communication between modules

    • Resource Variables

      • used for hooking up resources to the TIS, thus making them accessible from to modules

  • use prespecified or new datatypes


Sample infostate type declaration

sample infostate type declaration

infostate_variable_of_type( is, IS ) :-

IS = record( [

private : Private,

shared : Shared ] ),

Shared = record( [

com : set( proposition ),

qud : stack( question ),

lm : set( move ) ] ),

Private = record( [

agenda: stack( action ),

plan : stackset( action ),

bel : set( proposition ),

tmp : Shared ] ) ] ).


Resulting infostate type

resulting infostate type

AGENDA : stack( Action )

PLAN : stackset( Action )

PRIVATE :

BEL : set( Prop )

TMP : (same type as SHARED)

COM : set( Prop )

QUD : stack( Question )

SHARED :

LU: [SPEAKER:dp,

MOVES:set( Move )]


Sample interface variable type declarations

Sample interface variable type declarations

interface_variable_of_type( input, string ).

interface_variable_of_type( output, string ).

interface_variable_of_type( latest_speaker, speaker ).

interface_variable_of_type( latest_moves, set(move) ).

interface_variable_of_type( next_moves, set(move) ).


Specifying a set of moves

Specifying a set of moves

  • amounts to specifying objects of type move (a reserved type)

    • there may be type constraints on the arguments of moves

  • preconditions and effects of moves

    • formalised in update rules, not in the move definition itself

    • a move may have different effects on the IS depending e.g. on who performed it


Sample move specifications

sample move specifications

% Social

of_type( quit, move ).

of_type( greet, move ).

of_type( thank, move ) .

% Q&A

of_type( ask(Q), move ) <- of_type( Q, question ).

of_type(inform(P), move ) <- of_type( P, proposition).

of_type( answer(R), move ) <-

of_type( R, proposition) or

of_type( R, ellipsis ).


Writing rules

Writing rules

  • rule = conditions + updates

    • if the rule is applied to the IS and its conditions are true, the operations will be applied to the IS

    • conditions may bind variables with scope over the rule (prolog variables, with unification and backtracking)


A sample rule

A sample rule

rule( integrateUsrAnswer, [

$/shared/lu/speaker = usr,

assoc( $/shared/lu/moves, answer(R), false ),

fst( $/shared/qud, Q ),

$domain : relevant_answer( Q, R ),

$domain : reduce(Q, R, P)

], [

set_assoc( /shared/lu/moves, answer(R),true),

shared/qud := $$pop( $/shared/qud ),

add( /shared/com, P ) ] ).


A sample rule old syntax

A sample rule (old syntax)

  • rule( integrateUsrAnswer, [

    • val#rec( shared^lu^speaker, usr ),

    • assoc#rec( shared^lu^moves, answer( R ), false ),

    • fst#rec( shared^qud, Q ),

    • domain :: relevant_answer( Q, R ),

    • domain :: reduce(Q, R, P)

    • ], [

    • set_assoc#rec( shared^lu^moves, answer(R),true),

    • pop#rec( shared^qud ),

    • add#rec( shared^com, P ) ] ).


Writing rules1

Writing rules

  • available conditions and operations depend on the infostate type

    • the infostate is declared to be of a certain (usually complex) type

  • datatype definitions provide

    • selectors: Sel(InObj,SelObj)

    • relations:Rel(Arg1, …, ArgN)

    • functions:Fun(Arg1, …, ArgN,Result)

    • operations:

      Op(ObjIn,Arg1, …, ArgN,ObjOut)

  • New datatypes may be added


Writing rules locations in tis

Writing rules: locations in TIS

  • objects may be specified by giving a path to a locationin the infostate;

    • paths are specified using selectors, which are similar to functions

      • $$Sel2($$Sel1) ~ $Sel1/Sel2

      • $$fst($/shared/qud) ~ $/shared/qud/fst

    • ”$” evaluates a path and gives the object at the location specified

    • ”$$” evaluates a function

    • $$fst($/shared/qud) = $/shared/qud/fst

  • example:

    • is/shared/com is a path, pointing to a location in the TIS

    • $is/shared/com is the object in that location

    • theiscan be left out, giving$/shared/com


Writing rules conditions 1

Writing rules: conditions (1)

  • conditions do not change the information state

  • if a condition fails, backtracking ensues

  • condition syntax (incomplete)

    • Rel(Arg1, … , ArgN), e.g.

      • fst($/shared/qud,Q)

    • Arg1:Rel(Arg2,…,ArgN), e.g.

      • $/shared/qud:fst(Q)

      • $domain:relevant_answer(Q,A)

    • Arg1 = Arg2

      • Q = $$fst($/shared/qud)

    • Cond1 and Cond2

    • Cond1 or Cond2

    • not Cond1

    • forall(Cond1, Cond2)

    • (Argis object or prolog variable)


Writing rules conditions 2

Writing rules: conditions (2)

  • quantification, binding and backtracking

    • if an instantiation a of a variable V in a condition C is found that makes condition C true, V is bound to a

    • backtracking occurs until a successful instantiation of all variables in the list of conditions has been found

  • example list of conditions

    fst($/shared/qud,Q),

    in($/shared/com,P),

    $domain:relevant_answer(P,Q)

  • Explicit quantification

    Q.P. fst($/shared/qud,Q) & in($/shared/com,P) & $domain:relevant_answer(P,Q)


Writing rules updates

Writing rules: updates

  • operations change the information state

  • if an operation fails, an error is reported

  • variable bindings survive from conditions to operations

  • operation syntax (incomplete)

    • Op(Path,Arg1,…,ArgN)

      • push(/shared/qud, Q)

    • Path : Op(Arg1, … ,ArgN)

      • /shared/qud : push(Q)

    • Store := Fun(Obj,Arg1,…,ArgN)

      • /private/tmp/qud := $$push($/shared/qud,Q)


Specifying update algorithms

Specifying update algorithms

  • uses rule classes

  • constructs include

    • Rule

    • RuleClass

    • if Cond then S else T

    • repeat R until C

    • repeat R

    • try R

    • R orelse S

    • test C

    • SubAlgorithm


Sample update algorithm

Sample update algorithm

grounding,

if $latest_speaker == sys

then

try integrate,

try database,

repeat downdate_agenda,

store

else

repeat

integrate

orelse accommodate

orelse find_plan

orelse

if (empty ( $/private/agenda ) then manage_plan

else downdate_agenda

repeat downdate_agenda

if empty($/private/agenda))

then repeat manage_plan

repeat refill_agenda

repeat store_nim

try downdate_qud


Specifying serial control algorithms

Specifying serial control algorithms

  • serial constructs include

    • Module{:Algorithm}

    • if Cond then S else T

    • repeat R until C

    • repeat R

    • try R

    • R orelse S

    • test C

    • SubAlgorithm


Specifying concurrent control algorithms

Specifying concurrent control algorithms

  • Agent1 | Agent2 | … | AgentN

  • whereAgenti is

  • AgentName : {

    • import Module1 ,

    • import Modulep ,

    • Trigger1 => SerialAlgoritm1 ,

    • Triggerm => SerialAlgoritmm }

  • triggers:

    • condition(C) (C is a subset of the full condition set)

    • init

    • new_data(Stream)


Sample control algorithm 1

Sample control algorithm (1)

repeat ( [

select,

generate,

output,

update,

test( $program_state == run ),

input,

interpret,

update ] )


Sample control algorithm 2

Sample control algorithm (2)

input: {

init => input:display_prompt,

new_data(user_input) => input }

| interpretation: {

import interpret,

condition(is_set(input)) => [ interpret, print_state ] }

| dme: {

import update,

import select,

init => [ select ],

condition(not empty(latest_moves)) => [

update,

if $latest_speaker == usr then select

] }

| generation: {

condition(is_set(next_moves)) => generate }

| output: {

condition(is_set(output)) => output } )).


From dme to dialogue system

From DME to dialogue system

Build or select from existing components:

  • Modules, e.g.

    • input

    • interpretation

    • generation

    • output

  • Still domain independent

  • the choice of modules determines e.g. the format of the grammar and lexicon


Domain specific system

Domain-specific system

Build or select from existing components:

  • Resources, e.g.

    • domain (device/database) interface

    • dialog-related domain knowledge, e.g. plan libraries etc.

    • grammars, lexicons


Extending trindikit

Extending TrindiKit


You can add

You can add

  • Datatypes

    • Whatever you need

  • Modules

    • e.g. General interfaces to speech recognizers and synthesizers

  • Resources

    • E.g. General interfaces to (passive) devices

  • Important that all things added are reasonably general, so they can be reused in other systsems


Datatype definitions

Datatype definitions

  • relations

    • relations between objects; true or false

    • format: relation(Rel,Args).

    • Example

      • definition: relation(fst,[stack([E|S]),E]).

      • condition: fst($/shared/qud,Q)


Datatype definitions1

Datatype definitions

  • functions

    • functions from arguments to result

    • format: function(Fun,Args,Result).

    • Example

      • definition: function(fst,[stack([E|S])],E).

      • in condition:

        • Q = $$fst($/shared/qud)

        • Q = $/shared/qud/fst

      • in effect:

        • next_move/content := $$fst($/shared/qud)

    • every function corresponds to a relation relation(Fun,[Args@[Result]]).


Datatype definitions 2

Datatype definitions (2)

  • selectors

    • selects an object (Obj) embedded in another object (Arg)

    • selector(Sel,Arg,Obj,ArgWithHole,Hole).

    • e.g. selector(fst,stack([E|S]),E,stack([H|S]),H).

    • Every selector corresponds to a function function(Sel,[Arg],Object).


Datatype definitions 3

Datatype definitions (3)

  • operations

    • operation(Op,InObj,Args,OutObj).

    • e.g. operation(push,stack(S),[E],stack([E|S])).

    • every operation corresponds to a relation relation(Op,[InObj|Args]@[OutObj]).

    • push($/shared/qud,Q,$/shared/qud).


Building modules

Building modules

  • DME modules

    • Specific to a certain theory of dialogue management

    • Best implemented using rules and algorithms

  • Other modules

    • Should be more general, less specific to certain theory of dialogue management

    • May be easier to implement directly in prolog or other language

      • DME-ADL only covers checking and updating the infostate

      • These modules may also need to interact with other programs or devices


Building resources

Building resources

  • Resource

    • the resource itself; exports a set of predicates

  • Resource interface

    • defines the resource as a datatype T, i.e. in terms of relations, functions and operations

  • Resource interface variable

    • a TIS variable whose value is an object of the type T

  • By changing the value of the variable, resources can be switched dynamically

    • change laguage

    • change domain


Sample resource variable type declarations incl resource interface

sample resource variable type declarations (incl. resource interface)

resource_type( lexiconT ).

resource_variable_of_type( lexicon, lexiconT ).

of_type( lexicon_travel_english, lexiconT ).

of_type( lexicon_autoroute_english, lexiconT ).

of_type( lexicon_travel_svenska, lexiconT ).

of_type( lexicon_cellphone_svenska, lexiconT ).

resource_condition(lexiconT,input_form(Phrase,Move),Lexicon) :- Lexicon : input_form( Phrase, Move ).

resource_condition(lexiconT,output_form(Phrase,Move),Lexicon):-

Lexicon : output_form( Phrase, Move ).


Conclusion

Conclusion


Trindikit f eatures

TrindiKit features

  • explicit information state datastructure

    • makes systems more transparent

    • enable e.g. context sensitive interpretation, distributed decision making, asynchronous interaction

  • update rules

    • provide an intuitive way of formalising theories in a way which can be used by a system

    • represent domain-independent dialogue management strategies


Trindikit features cont d

TrindiKit features cont’d

  • resources

    • represent domain-specific knowledge

    • can be switched dynamically

      • e.g. switching language on-line in GoDiS

  • modular architecture promotes reuse

    • basic system -> genre-specific systems

    • genre-specific system -> applications


Theoretical advantages of trindikit

Theoretical advantages of TrindiKit

  • theory-independent

    • allows implementation and comparison of competing theories

    • promotes exploration of middle ground between simplistic and very complex theories of dialogue

  • intuitive formalisation and implementation of dialogue theories

    • the implementation is close to the theory


Practical advantages of trindikit

Practical advantages of TrindiKit

  • promotes reuse and reconfigurability on multiple levels

  • general solutions to general phenomena enables rapid prototyping of applications

  • allows dealing with more complex dialogue phenomena not handled by current commercial systems


Availability

availability

  • version 3.0a avaliable!

    • SIRIDUS deliverable D6.4

  • TrindiKit website

    • www.ling.gu.se/projects/trindi/trindikit

  • SourceForge project

    • development versions available

    • developer community?

  • licensed under GPL

  • more info in

    • Larsson & Traum: NLE Special Issue on Best Practice in Dialogue Systems Design, 2000

    • TrindiKit manual (available from website)


Conclusions trindikit information state approach

Conclusions: TrindiKit & Information State Approach

  • a toolkit for dialogue systems R&D

  • freely available to researchers

  • close the gap between theory and practive of dialogue systems

  • theory-independent

  • promotes reuse and reconfigurability on multiple levels


Trindikit and voicexml

TrindiKit and VoiceXML

  • VoiceXML

    • industry standard

    • form-based dialogue manager

    • web/telephony infrastructure

    • requires scripting dialogues in detail

  • Theory-specific?

    • VoiceXML implements a specific theory of dialogue

    • TrindiKit allows implementation of several different theories of dialogue

    • More complex dialogue phenomena hard to deal with in VoiceXML


Trindikit and voicexml cont d

TrindiKit and VoiceXML, cont’d

  • Combine VoiceXML with TrindiKit?

    • future research area

    • support connecting TrindiKit to VoiceXML infrastructure

    • use TrindiKit system as VoiceXML server, dynamically building VoiceXML scripts

    • convert VoiceXML specifications to e.g. GoDiS dialogue plans


Possible future modifications

Possible future modifications

  • regler i klasser väljs indeterministiskt

  • test C -> if C then halt / break (gå ur loop)

  • ta bort möjlighet att ha flera regler med samma namn


  • Login