Rough schedule

1. Rough schedule • Multimodal, multi-party dialogue [30 min] • D’Homme, SIRIDUS [10 min] • dialogues with networked devices in a smart house • SRI demo (DM) , (IBL robot (JB)) • negotiative dialogue • menu-based natural command dialogue (SL) • future work on TrindiKit (SL) • Discussion: Dialogue genres [20 min] • reconfigurability (DM) • mixing infoseeking+command dialogue , switching domains dynamically (GoDiS demo) • more on building systems (incrementally) (SL) • GoDiS and VoiceXML (SL): [10 min] • General discussion (everyone): [20 min] • NL generation from infostates • generation from DRSs (JB) • generation of focus (SL) • Logics & proofs of dialogue, e.g.completeness of update rules • Real, practical applications: what are the problems • grammar based vs. statistical approaches (DM)

2. Future work on TrindiKit ESSLLI 2001

3. GUI (Graphical User Interface) • Currently being developed • Purpose • simplifying building of DS • pedagogical purposes • debugging • Gives overview of • information state at various levels of dialogue • update rules • interactions between modules, infostate, and resources • Eventually, build systems using GUI

4. Extended formalism • improved infostate path language • additional constructs for writing update algorithms • inspiration from SOAR, OZ, … • extended typechecking & debugging facilities

5. Adding more readymade modules & interfaces • include more powerful NL interpretation and generation modules • e.g. HPSG • Build hook modules for various useful programs • planners, reasoning engines etc. • Any OAA module can be accessed from TrindiKit, so not limited to Prolog programs • Resource interfaces for • SQL databases, • uPnP-specified devices, • etc.

6. Adding more speech modules • New products come out all the time • Actually using them in a system can be tricky and take lots of time • ”Open Source”; users contribute • modularity promotes reuse • send us a link to your TrindiKit® software! • requires general solutions and explaining how module works

7. Future work on GoDiS ESSLLI 2001

8. Integrate with VoiceXML? • Extended dialogue capabilities: Current • collaborative negotiation • multiple simultaneous plans • information sharing between plans • Extended dialogue capabilities: Future • talking to autonomous robots • noncollaborative negotiation • complex planning tasks • Implement more domains • no limits…

9. Typology of dialogues & dialogue systems

10. Typology of dialogues & dialogue systems • to what extent are dialogue types related by subsumption? To what extent are they orthoginal? • infoseeking < info exchange • instuctional, command < planning • negotiation • meaning / dialogue strategy / domain • collaborative / noncollaborative • argumentative / non-argumentative

11. Typology of dialogues & dialogue systems • Can dialogue types be defined in terms of theories / system features needed to handle them? • infostate contents • moves, rules • Approach: build systems incrementally • succesively increase scope of dialogue phenomena handled by system • reuse of system components over dialogue types

12. Negotiative dialogue some definitions and ideas

13. Problem with current GoDiS • assumes database always returns exactly one post (price info); not generally true • but we want to be able to • talk about several flights, • allowing the user to ask questions about them, • deciding on one of them, and then • getting price information • Requires negotiation

14. Negotiation vs. acceptance • Clark’s ladder: • 1. A attends to B’s utterance • 2. A percieves B’s utterance • 3. A understands B’s utterance (grounding) • 4. A accepts or rejects B’s utterance • Sidner and others sees negotiative dialogue as proposals and acceptance/rejections of proposals • this means that all dialogue is negotiative • all assertions (and questions, instructions etc.) are proposals • But some dialogues are negotiative in another sense, by explicitly containing discussions about different solutions to a problem, and finally deciding on one • Negotiation is not Clark’s level 4

15. Two senses of “negotiation” • Negotiation in Sidner’s sense • A: I want to go to Paris [propose] • B(1): OK [accept] • B(2): Sorry, there are no flights to Paris [reject] • Negotiation in our sense • U: flights to paris on september 13 please [answer] • S: there is one flight at 07:45 and one at 12:00 [propose] • U: what airline is the 12:00 one [ask] • S: the 12:00 flight is an SAS flight [answer] • U: I’ll take the 12:00 flight please [accept]

16. Optimistic approach to acceptance • DPs assume their utterances are accepted (and integrated into SHARED) • If A asks a question with content Q, A will put Q topmost on SHARED.QUD • If addresse indicates rejection, backtrack • using the PRIVATE.TMP field • No need to indicate acceptance explicitly; it is assumed • The alternative is a pessimistic approach • If A asks a question with content Q, A will wait for an acceptance (implicit or explicit) before putting Q on top of QUD

17. Negotiativity • Negotiation is a type of problem-solving (cf. Di Eugenio et. al., Coconut) • Negotiation: DPs discuss several alternative solutions before choosing one of them • Negotiation does not imply conflicting goals • perhaps not 100% correspondence to everyday use of the word “negotiation”, but useful to keep collaborativity as a separate dimension from negotiation • Both AOD and IOD can be negotiative • in a flight information service, the user does not become obliged to fly anywhere; so it’s IOD • but several different flights may be discussed

18. Negotiation tasks • Some factors influencing negotiation • distribution of information between DPs • whether DPs must commit jointly (e.g. Coconut) or one DP can make the comittment (e.g. flight booking) • We’re initially trying to model negotiation in flight booking • sample dialouge • U: flights to paris on september 13 please • S: there is one flight at 07:45 and one at 12:00 • U: what airline is the 12:00 one • S: the 12:00 flight is an SAS flight • U: I’ll take the 12:00 flight please • Sys provides alternatives, User makes the choice • Sys knows timetable, User knows when he wants to travel etc.

19. Degrees of negotiativity • non-negotiative dialogue: only one alternative is discussed • semi-negotiative dialogue: a new alternative can be introduced by altering parameters of the previous alternative, but previous alternatives are not retained • negotiative dialogue: several alternatives can be introduced, and old alternatives are retained and can be returned to

20. Semi-negotiative dialogue • Does not require keeping track of several alternatives • Answers must be revisable; this can be done using reraising of answered questions • Correction of optimistic assumption of acceptance not necessarliy distinguished from revision • Example: Swedish SJ system (Philips): ”Do you want an earlier or later train?”

21. Issues Under Negotiation i negotiative dialogue • IUN is a question e.g. what flight to take • In an activity, some questions are marked as negotiable issues; other questions are assumed to be non-negotiable • Needs a new IS field: SHARED.IUN of type assocset(question,set(answer))

22. Alternatives in negotiation • Alternatives are alternate answers to an IUN • a proposal is the introduction of a new possible answer to IUN • An IUN is resolved when an answer to it is given, i.e. when an alternative is accepted • Alternatives and information about them is optimistically assumed to be accepted • Alternatives are needed whenever database search can return more than one result

23. General and specific information • General information concerns all alternatives, and is collected in an initial information-seeking dialogue (e.g. flights to paris) • e.g. x.dest(x,Paris) • Specific information concerns specific alternatives (e.g. flight f345 leaves at 10:45) • Specific info usually results from a database search whose input is general info; does this motivate separate fields in IS?

24. Example • IUN is lx.sel_flight(x) (“which is the chosen flight”?) • A: flight to paris, december 13 • answer(x.dest(x,paris)) etc.; • B: OK, there’s one flight leaving at 07:45 and one at 12:00 • propose(f1), propose(f2), • answer(dep_time(f1,07:45)), answer(dep_time(f2,12:00)) • A: I’ll take the 07:45 one • answer(sel_flight(X), dep_time(X, 07:45)), • after contextual interpretation: answer(sel_flight(f1))

25. B: OK, there’s one flight leaving at 07:45 and one at 12:00 AGENDA = { findout(?x.sel_flight(x)) } PLAN = findout((?x. ccn(x)) book_ticket PRIVATE = BEL = {flight(f1), dep_time(f1,0745), ... } TMP = (same structure as SHARED) IUN = < ?x.sel_flight(x){f1, f2 }> SHARED = dep_time(f1,0745), dep_time(f2,1200) dest(paris), ... COM = QUD = <> LM = {propose(f1), propose(f2), answer(dep_time(f1,07:40),...}

26. VoiceXML • form-filling paradigm • forms, consiting of slots+values • menus, special case of forms • speech recogntion grammars defined with scope over • slot • form • document • Why is VoiceXML interesting for us? • becoming industry standard • supports plan constructs similar to GoDiS • multiple active grammars allow behaviour reminiscent of question and task accommodation

27. Dialogue capabilities • VoiceXML: if input matches several fields, the first is chosen • GoDiS can ask clarification question • VoiceXML: user initiated subdialogues cannot share information with main dialog, or other subdialogues • the default in GoDiS is that all information is shared between subdialogues • unclear how to implement several simultaneously active plans

28. Architectural issues • information state is global and keeps plan separate from accumulated propositions • VoiceXML based on forms, which can be seen as local information states • VoiceXML mixes dialogue knowledge, domain knowledge, and language knowledge in a single specification • GoDiS keeps them separate, enabling easier reconfiguration and plug-and-play • when implement a new menu-driven system, there is no need to reimplement general dialogue strategies

29. future issues • Open question whether VoiceXML can extend to more complex dialogue, e.g. negotiation • but the information state approach is ideal for complex dialogues • GoDiS is currently being extended to handle collaborative negotiation • Can GoDiS be combined with VoiceXML? • have GoDiS use extended VoiceXML specifications, • or use standard VoiceXML specs in a smarter way

30. generating focus from information states

31. Accommodation • Generality • no need to distinguish requested and non-requested (but relevant) information • single rule for integrating answers • Theoretically motivated concept, with independent justification • Easy to implement, given information state approach • Gives useful side-effects (e.g. task accommodation leads to loading a plan) • supports the idea of accommodation as opposed to “direct interpretation”

32. Extensions • Questions can also be reaccommodated • if the user answers a question which has already been answered: • remove proposition from shared beliefs, and put question back on QUD • Extend to domain where there are many plans containing a question matching a given answer • e.g. menu-based dialogue • Focus intonation and QUD

33. Focus and information states • Focal Question Presupposition (FQP) (based on Ginzburg and others): • An utterance with narrow focus on a constituent presupposes a function/question obtained by abstracting over the focussed constituent • Example: • “Jill likes BILL” [like(jill, bill)] presupposes “Who does Jill like?” [?lx.like(jill, x)]

34. Focal Question Accommodation FQuAcc: interpretation version of FQP “When an utterance occurs which focally presupposes Q, and Q is not topmost on QUD, make Q topmost on QUD” in(SHARED.LM, Move) foc-presupp(Move, Q) ~fst(SHARED.QUD , Q) pre: push(SHARED.QUD, Q) eff:

35. Interpreting utterances with focus • Example: A: Are you FLYING to london? ?(to-city(london)&transport(fly)) presupposes Q’ = ?(lx.to-city(london)&transport(x)), i.e. “how are you getting to london” B1: Yes B2: No, I’m taking a TRAIN ?B3: No [Q’ still on QUD!]

36. Generating utterances with focus • Generation version of FQP: • When generating Q, and there is a Q’ on QUD such that Q’ is an abstraction of Q over constituent C (Q=Q’(C)), put narrow focus on C

37. Generating clarifications with focus (cautious grounding) • Example 1: • U: I want a flight please • S: What city do you want to go to? (Q’) • U: London • S: So you’re flying to LONDON? (Q) [Q=Q’(london); Q’ still on QUD, so put focus on “London”] • Example 2: • U: I want to go to London • S: How do you want to travel? (Q’) • U: A flight please • S: So you’re FLYING to London? (Q)

38. Generating questions with focus II • If • a question is to be asked, and • a parallell question has previously been answerd (there is a parallel information P in SHARED.COM), • put focus on the part that differentiates Q1 and P • Example: • S: What CITY do you want to go to? • U: Paris .... • S: What city do you want to go FROM? • ?x.to-city(x) parallell to ?x.from-city(x)

39. Generating questions with focus II (cont’d) • Extension to focal question presupposition: • A question Q with narrow focus on C presupposes a parallell question Q’’ which differs from Q by having C replaced by some B • Does accommodation apply? • What about propositions? • What, exactly, does “parallell” mean? (cf. Pulman 1998 for a formal account of parallellism for propositions)