TrindiKit

1. TrindiKit

2. What is TrindiKit? a toolkit for building and experimenting with dialogue move engines and systems, based on the information state approach originally developed in TRINDI, considerably extended and improved in SIRIDUS

3. Key ideas • thinking in terms of IS updates • update rules • functions IS (+moves)  IS (+moves) • generic domain-independent dialogue management • requires modularity • use of global information state • defined in terms of abstract datatypes • all modules can access all information

4. control DME module1 module… modulei modulej module… modulen • Total Information State • (TIS) • Information state proper (IS) • Module Interface Variables • Resource Interface Variables resource1 resource… resourcem

5. application domain & language resources genre-specific theory additions genre-specific system basic dialoguetheory basic system TrindiKit information state approach

6. TrindiKit features • explicit information state datastructure • makes systems more transparent • enable e.g. context sensitive interpretation, distributed decision making, asynchronous interaction • update rules • provide an intuitive way of formalising theories in a way which can be used by a system • represent domain-independent dialogue management strategies • resources • represent domain-specific knowledge • can be switched dynamically • e.g. switching language on-line in GoDiS • modular architecture promotes reuse • basic system -> genre-specific systems • genre-specific system -> applications

7. technical features • interfaces to OAA (but can also run without it) • see e.g. demo by Zinn et al. • system modules can run either serially or in parallell • wrappers for off-the-shelf recognizers and synthesizers • runs on UNIX, Windows, Linux • currently uses SICStus Prolog • but considering moving to YAP prolog (freely available Sicstus clone) • possibly reimplement in other language

8. availability • version 2.1 is available • version 3.0a coming at end of 2002 • SIRIDUS deliverable D6.4 • TrindiKit website • www.ling.gu.se/projects/trindi/trindikit • SourceForge project • development versions available • developer community? • licensed under GPL • more info in • Larsson & Traum: NLE Special Issue on Best Practice in Dialogue Systems Design, 2000 • TrindiKit manual (available from website)

9. Flexible issue-based dialogue management

10. Goals • explore and implement issue-based dialogue management • adapt Ginzburg’s KOS to dialogue system (GoDiS) and implement • extend theory (incl. accommodation, action-oriented dialogue, negotiation, conditional responses) • separating general and domain-dependent phenomena helps reconfigurability • general theory of dialogue, extended into subtheories for different dialogue types • minimize effort for adapting to new domains

11. application-specific Xerox manual home device manager Travel Agency VCR manager Auto- route genre-specific GoDiS-I GoDiS-A IBDM GoDiS TrindiKit IS approach

12. Basic issue-based dialogue management • enquiry-oriented dialogue (database search) • starting point: • Ginzburg’s DGB and • related DGB update protocols • moves: ask, answer, (greet, quit) • raising and addressing issues • incl. short answers • dialogue plans • handling multiple simultaneous issues • information sharing between plans • sample domain: travel agency

13. basic infostate AGENDA : OpenQueue( Action ) PLAN : OpenStack( Action ) PRIVATE : BEL : Set( Prop ) COM : Set( Prop ) QUD : OpenStack( Question ) SHARED : ISSUES : OpenStack( Question ) LU: SPEAKER: Speaker MOVES: OpenQueue( Move ) • QUD:local, questions available for ellipsis resolution • ISSUES: global, questions which have been raised but not yet resolved

14. Semantics • Proposition: n-ary predicate-argument structure • e.g. dest-city(paris) • Question • Y/N-questions: ?P, P is a proposition • wh-questions: ?x.P(x) • alt-questions: {?P1, …, ?Pn} • ShortAns • individual markers: paris, april, … • yes, no • Q-A relations (adapted from Ginzburg) • resolves(A,Q): A resolves Q • dest-city(paris) resolves ?x.dest-city(x) • relevant(A,Q): A is a relevant answer to Q • not(dest-city(paris)) is relevant to?x.dest-city(x), but does not resolve it

15. sample dialogue plan task: ?x.price(x) < findout(?x.how(x)) findout(?x.dest-city(x)) findout(?x.dept-city(x)) findout(?x.dept-month(x)) raise(?x.dept-day(x)) findout(?return) If return then < findout(?x.ret_month) findout(?x.ret_day) > raise(?x.class(x)) consultDB(?x.price(x))>

16. Dealing with multiple issues • if user asks Q, push Q on ISSUES and load plan for dealing with Q • if users asks Q’ while system is dealing with Q, throw out plan for Q but Q remains on ISSUES • when Q’ resolved, Q topmost on ISSUES will trigger reloading plan for dealing with Q • general rule: if SHARED.COM contains info resolving Q, don’t ask Q • so any resolved questions in plan will be thrown out

17. Information sharing across plans • GoDiS does not keep track of which plan was being executed when propositions were added • so information sharing is determined by question sharing across plans • plan for VISA question: ?need_visa < findout(?x.dest-city(x)) findout(?x.dept-city(x)) findout(?x.citizenship(x)) consultDB(?need_visa) > • shared 2 questions with plan for ?x.price(x)

18. Sample dialogue: multiple tasks & info sharing S> Welcome to the travel agency! U> price information S> (…) Lets see. How do you want to travel? U> by flight S> (…) What city do you want to go to? U> paris S> (…) What city do you want to go from? U> do I need a visa ? S> (…) Lets see. What country are you from? U> sweden S> Okay. Yes, you need a Visa. S> Returning to the issue of price. Lets see. What city do you want to go from?

19. Grounding and feedback(Interactive Communication Management) • feedback types (Allwood, Clark) • action level: contact, perception, understanding, acceptance • polarity: positive, negative • update strategies • optimistic • non-cautious • cautious • pessimistic • feedback and grounding for a dialogue system

20. Some ICM dialogue moves • Interactive Communication Management • feedback • turntaking • sequencing: reflects dialogue structure • icm:Level{*Polarity}{:Content} • icm:und*neg – ”I don’t understand” • icm:und*pos:P – ”To Paris.” • icm:acc*neg:Q – ”Sorry, I can’t answer Q” • icm:acc*pos – ”Okay” • icm:reraise:Q – ”Returning to the issue Q” • icm:loadplan – ”Let’s see…”

21. Question Accommodation • to deal with • user initiative (other than asking questions) • giving more/less/other information than requested • simple information revision • reraising issues • basic idea: • move questions to QUD or ISSUES to adapt to user utterances

22. Sample dialogue: accommodation S: Welcome to the travel agency. U: From London to Paris in April • not relevant to anything on ISSUES (since ISSUES is empty) • look in domain knowledge for a plan with matching questions • the plan for dealing with ?x.price(x) is found; this question depends on • dependent issue accommodation: push on ISSUES ?x.price(x) • load plan for ?x.price(x) • issue accommodation: push on ISSUES ?x.dest-city(x) • integrate answer (requres matching question on ISSUES) • same with ?x.dept-city(x), ?x.dep_month(x) S: Alright, you want to know about price. (…) • proceed to next plan item S: How do you want to travel? • ISSUES=<?x.how(x), ?x.price(x)> (NOTE: sys only deals with price & visa issues)

23. Action-oriented dialogue based on menus • GoDiS-A: for action-oriented dialogue • each plan now associated with an action or a question • add SHARED.ACTIONS: Stack(Action) • ACTIONS has a similar role to ISSUES • New moves: • request(Action) • confirm(Action) [or report(Action, Status)] • adapt accommodation strategies to AOD • sample domain: menu-based dialogue for VCR • (demo during lunch break)

24. Converting menus to plans

25. VCR menu structure fragment • change play status • play, stop etc. • change channel • channel: _ • timer recording • add program • channel:_ • date:_ • start-time:_ • end_time:_ • display added program • delete program • display existing programs • delete program:_ • settings • clock, etc.

26. Plan for VCR>timer recording>add program findout(?x.program_position_to_store(x)), findout(?x.date_to_store(x)), findout(?x.start_time_to_store(x)), findout(?x.stop_time_to_store(x)), dev_do(vcr, 'AddProgram') • (confirmations generated by update rule whenever some action is done, so not needed in plan)

27. Sample menu dialogue(with language change!) S: Welcome to the VCR manager. Let’s see. What can I do for you? U: Add a program today, channel five • request(add_program) -> plan is loaded • issue accommodation to integrate ”channel five” S: What time do you want to start recording? U: Five fifteen …

28. Other architectures • Agent communication protocols • Open Agent Architecture • KQML • Dialogue system toolkits • DARPA Communicator • SOAR, NL-SOAR • Systems (implementing specific dialogue theory) • Verbmobil • VoiceXML • have been compared to TrindiKit (see deliverables or ask)

29. relevant properties • support for information state approach • which datastructures/datatypes are supported? • support for managing control flow • not limited to pipelining • asynchronous processing • modularity • potential for scalability

30. OAA • ”Facilitator” (hub) manages communication between ”agents” (spokes) • facilitator can maintain global information state • what datastructures are available? • asynchronous processing • modular • similar to scriptless version of DARPA Communicator

31. KQML • Knowledge Querying and Manipulation Language • communication protocol between agents

32. DARPA Communicator • Hub-and-spoke architecture • modularity, flexible control, asynchronous processing • hub functions • message routing between servers (spokes) • state maintenance • control flow; script decides which server to call next

33. DARPA communicator cont’d • limited support for information states • ”tokens”: frames with name, index, and list of feature-value pairs • tokens processed by scripts • set of rules determining when to call a server + which arguments (features) to pass • Hub scripts too limited to do real dialogue management • most actual systems have separate dialogue manager • hub mostly used for data routing

34. SOAR • toolkit for modeling cognitive agents • similar to TrindiKit in some respects • keeps single information state • however, only one datastructure whereas TrindiKit has extendible range of (possibly complex) datatypes • update rules • some nice ways of triggering rules which can be considered for TrindiKit • supports ”chunking” of rules • however, may not be too useful in practice

35. VoiceXML • Frame-based dialogue manager • not really an architecture, more like a domain-independent dialogue system • Requires scripting dialogues in detail • Combine with GoDiS? • use GoDiS as VoiceXML server, dynamically building VoiceXML scripts • use VoiceXML specifications converted to GoDiS

36. Verbmobil • not dialogue system per se • rather, dialogue translation system • several local information states, not one global • ”partitioned blackboard” • which datastructures? extendible set? • limited control support