Title

1. Title A Joint Intention Based Dialog Architecture Rajah. A Subramanian University of Rochester

2. Need for a Generic Dialog Engine • Current spoken dialog systems • Domain based systems. • Need to explicitly code all possible rules and conditions for prompting, informing etc. • Very difficult to port from one domain to another, breaks easily and cannot be used for complex task-based scenarios. • We propose an architecture that can reason in a general manner and can replace many of the rules and condition that are explicit in current dialog systems. • Handle over-answering, multi-agent interaction, and certain dynamic changes in the environment. • Build Systems that are robust and domain independent.

3. Language Understanding Speech Recognition Dialog Management. Strategy Back-end (various) Speech Synthesis Language Generation Dialog Process We are concerned here in the Dialog Management Strategy.

4. Outline of the talk • Overview • Joint Intentions Theory • Semantics of Communicative Acts • Belief Reasoner • STAPLE interpreter • Room Reservation Demo • Summary, Future & Related work

5. Joint Intention Theory (JIT) - I • Belief, Desire, Intention (BDI) Model – Bratman/Israel/Pollack, Georgeff/Lansky … • Joint Intention Theory (JIT) (Cohen, Levesque). • Uses modal logic (beliefs, goals), dynamic logic of actions (sequence, concurrent actions, etc.), temporal logic (eventually, always). • INTENTION – Choice with commitment. • PGOAL x – Individual persistent goal of an agent. Once adopted, the agent is committed to it until achieved, impossible or irrelevant.

6. Joint Intention Theory (JIT) - II • PWAG x y – Joint goal between x and y. • PGOAL to achieve the goal relative to PWAG. • if already achieved, impossible or irrelevant, then PGOAL to mutually make it known to concerned agents. • Teamwork: JPG x y – Two way PWAG’s/ X has a PWAG towards Y and Y towards X. • Prescribes how to execute actions jointly while being in a shared mental state • Requires starting mutual belief (MB) • Requires MB about task completion, impossibility, or irrelevance • It predicts communication in the event of private beliefs

7. Semantics of Communicative Acts • Formal compositional semantics of commonly used speech acts using JIT constructs • REQUEST and INFORM are the basic CA’s. • The goal of a request (x y a ): y eventually does the action a and also have a PWAG with respect to the requester x to do a. • The goal of an inform (x y p) is to establish MB between the informee (x) and the informed (y) that x believes the informed predicate (p). • The other CA’s such as ASK, AGREE, INFORMREF etc. are built from REQUEST and INFORM. For example, ASK is an REQUEST to INFORMREF. • KNOWREF – To know the referent of the description of a required value.

8. Back Communicative Actions (cont..) • Identified speech acts that can be used to create and discharge teams • Showed how to achieve mutual belief • Explicated defeasible assumptions about communication • STAPLE (Sanjeev Kumar) – Implements the constructs JI theory and interprets PGOAL’s, PWAG’s, Communicative Acts, etc. Can you please help me move the table? Sure REQUEST + AGREE = Joint Commitment

9. STAPLE Architecture (Kumar 2006)

10. STAPLE Optimization • Original version – Java only • Not efficient for unification, that constitutes more than 80% of execution time. • To optimize – combined Java and Prolog, completed as part of RPE project. • Java for threads, stacks, triggers etc. • Prolog for knowledge base, unification etc. • More than 150 times faster for single-agent and 70 times faster for multi-agents.

11. Planning Relevant Questions – Snapshot of a Goal Stack in STAPLE *The PGOAL is actually a bit more complex – the system wants to know the referent of the date that the user wants the room reservation; also the sub-goals are relative to the original goal.

12. STAPLE: Multi-agent Dialog Example Robby (USER) • Room reservation domain • Three Microsoft agent characters play the roles of User, System, & Database agent Peedy (SYSTEM) Merlin (DATABASE AGENT)

13. Multi-Agent Dialog Example - Setup • Robby (USER) • Initial state: - Believes that it wants a room at a particular date & time • Actions: - Cannot perform any actions other than communicative acts • Peedy (SYSTEM) • Initial state: - Believes that the user knows the date & time for reserving the room - Believes that the database agent can query for rooms and hold a room if necessary. • Actions: - Can perform action reserve room that happens to be a named action expression (i.e., a plan) • Merlin (DATABASE AGENT) • Initial state: - No domain related beliefs. • Actions: - Can perform actions to query the database and to hold a room.

14. Handling Over-Answering Agent: Please say the date you want the room User: I would like it on March 20th at Three PM Bel(user, /\ ([ March20 = i(Date, date (Room, Date))), 3PM = i(Time, time (Room, Time)))])). Bel(user, March20 = i(Date, date (Room, Date)))) Bel(user, 3PM = i(Time, time (Room, Time)))). PGOAL Discharged PGOAL Discharged PGOAL (KNOW-REF i (D,date(Room,D))) PGOAL (KNOW-REF i (T,time(Room,T))) PGOAL (KNOW-REF i (S,size(Room,S)))

15. Multi-Agent Interaction System Agent Original JPG to reserve a room USER … (Action Expression Decomposition) PGOAL(system-agent, done(action (getrooms, X,E),…. Bel(system-agent, cando(getrooms, database-agent)) Database Agent

16. Multi-Agent Interaction System Agent Original JPG to reserve a room USER INTEND(system, action(request(system,[system, database], action(getrooms, …… ….. INTEND(system, action(establish_jpg([system, database], done(action(getrooms(……relative to PWAG reserveroom PGOAL(system-agent, done(action (getrooms, X,E),…. REQUEST Database Agent

17. Back Multi-Agent Interaction System Agent Original JPG to reserve a room USER JPG to do getrooms AGREE REQUEST Database Agent

18. Handling Dynamic Changes System Agent Original JPG to reserve a room USER Conversation between user and system to confirm the room INFORM – not able to hold the room JPG to do holdroom JPG to do holdroom becomes impossible INTEND(database, done(action(inform( database,system,always(not(done (action(holdroom(….. Room is TAKEN!!! Database Agent INTEND(database,action(holdroom,database) ..Relative to original PWAG to reserve a room PGOAL(database, mb([system, database], always(not(done(action(holdroom(…..

19. Handling Dynamic Changes System Agent Original JPG to reserve a room USER Check if there are other rooms that are available or inform user that no rooms are available. JPG to do holdroom JPG to do holdroom becomes impossibe Database Agent

20. Back Demo

21. The Point • We get automatic team and task-related communication behavior • If agent believes that somebody else knows an answer then initiates dialogue to find out that answer. • If an agent believes that somebody else can do the action that it wanted done then it establishes a team for that purpose • Dialogue about mutual belief establishment follows automatically. • No mutual observation leads to explicit communication, otherwise explicit communication is absent. • The above behavior can be chained to elicit even more interesting behavior. • Example: A sub-dialogue may get started in the middle of joint action execution to find the truth value of a proposition if it is the precondition for doing a part of the jointly committed action.

22. Future Work • In Progress • Implicit Confirmations • Explicit Clarifications • Still Mulling over… • Partial and weak beliefs • Belief update and revision

23. Related Work • Dialogue & Communication • Collagen: MERL – Rich, Sidner, Lesh • Artimis & Arcol: France Telecom - Sadek, Panaget, Bretier,… • TRIPS: University of Rochester – Allen, Ferguson, … • FIPA, KQML • Teamwork • STEAM & Teamcore: USC – Tambe et al. • Grate: Liverpool, UK - Jennings)

24. Appendix A: STAPLE Program – Multi Agent Interaction and Dynamic Changes • bel(system,can_do(getrooms(Room,List), database)). • bel(system,exists(D,bel(user,equals(D,i(Date,date(Room,Date)))))). plan(reserveroom,4) :- [args: [Room, [Date, Time], E1,E2], achieve_precondition: true, effects: [(roomReserved(Room,[Date,Time]),1.0)], body: { action(roominfo([date(Room, Date), time(Room, Time)]),system,E1)}]. plan(roominfo,1) :- [args: [[date(Room, Date), time(Room, Time)]], effects: [(roomsAvailable(Room, [Date, Time]),1.0)], body: { action(getrooms(Room, [Date, Time]),X, Ev1)}].

25. Appendix B: Belief Reasoner

26. Appendix C: Mutual Belief Reasoner