Dialogue-based approach to intelligent assistance on the web Ana García-Serrano

1. Dialogue-based approach to intelligent assistance on the web Ana García-Serrano Technical University of Madrid, Spain FLINT 2001 University of California, Berkeley August 2001

2. ADVICE (EU project) Http://www.advice.iao.fhg.de

3. The e-commerce solutions (2000) “Internet as a giant marketplace” Web-catalogues: • Mainly focus on the presentation of goods and on sales processes itself. • The interaction is guided by the user • By a click on the article he can get more information about the • product as components, characteristics, quantity,... • From the customer point of view: the search and the selection of products is a difficult task  mainly due to the lack of assistance on the Web.

4. Intelligent assistance to e-commerce 1. Human-like seller: Complementing the services of the current e-commerce systems  perform a dialogue-based interaction (when, why...)  information generation process according to the particular needs of the user (what)  use “expressive” presentations (animation, natural language ...) 2. Advanced user interface (How, which devices, ...) 3. Software engineering  legacy systems (plug-in solution)  interoperability

5. Technologies involved (first step) Quality of information • Knowledge-based engineering • KSM Methodology and tool [Cuena, 98, ...] • multi-paradigm representation and inference models • scalability • Multimodal interfaces • Multimedia (coherent presentations) [Cohen, 00 ...] • Dialogue models [Searle 75, Allen 94, Mittal & Moore, 95... ] • Agent-based design • agent as an abstraction concept key • [Jennings, Sycara, Wooldridge, 98, Wooldridge, Mandani, Rao, Stout,99] • support for distributed and heterogeneous software [Bueno 99] Expressive Com. Interoperability,...

6. Greeting (Smiling Avatar, ...) Interface agent Pictures, GUI ... User Click on alternatives

7. Intelligent Interface agent agent

8. Interaction Intelligent Interface agent agent agent

9. Output: structure obtained from the offer decision tree (S) and the next step (I) Intelligent agent Output Criteria to next step Discourse Maker Input: structure that reflects the intention of the user and the semantic content Interaction Agent • Session Model (static): • Context (referred data) • interaction preferences • Dialogue Manager (dynamic): • State of dialogue • Thread joint (intention) Input User Model

10. Dialogue-based interaction (Dynamic)

11. Communicative acts of the user/system Searle’s Speech acts (Advice project proposal) Representative acts Inform: < t, m, s, c > Patterns: type matter subject content ‘I’m John Smith’ introduction identity user John Smith Saw ‘I want a circular saw’ data identity product product confirmation deny ‘No, .... Type : data, advice, declaration, suggest, explanation, introduction, confirmation Matter: approve, deny purchase, identity ,... Subject : product, user, system, task = ... Content Directive acts (Request, command) Authoritative acts Courtesy acts Null

13. Dialogue-based interaction (Dynamic)

14. Dialogue-based interaction (Dynamic) ... Offers partial solution Show catalogue Show ... l | Ask_product Ask ... confirm l command or Greeting ended confirm(task) Authorize Solve task user for Greeting task greeting deny question inform thank fork ( confirm_|_deny) inform System inform close System provides disannoyance requires explanation thank question explanation System l question grateful question inform confirm l fork System solves Farewell question Offer end session l | farewell or confirm disconnect

15. Dialogue-based interaction (utterance or user cliks on the saw alternative): “I think I need a pendulum-cut saw” • Identification of the comunicative acts: • null, inform(data, identity, product, saw) • inform(data,feature,product,type=pendulum-cut) • Identification of a new state into the (current) question-answer dialogue pattern: “solve”, • Identification of the next interaction step: prune of the configurable offer (decision tree), perform an action ...

16. ACCESORY PRODUCT Customised configuration of the offer ................... U: Well, I want a circular saw. S: What kind of saw do you want to purchase, a pendulum-cover saw or a plunge-cut saw? TOOL :<ask pendulum/plunge> SAW SANDER PLUNGE PENDULUM AT65 AT65 AT65 AT65 AT65 ATF55 ............ AT65 AT65EBplus ATF55 ATF55FS ATF55EB ATF55EBplus ATF55EBFS ATF55EBplusFS AT65FS AT65EBplusFS

17. Generic model formulation of the domain A Knowledge Unit is described by a set of knowledge sub-areas and the tasks is capable to perform

18. Domain model formulation: Tool knowledge Unit (saws) • PATTERN atf55EBplus • DESCRIPTION • type = saw [a], • name = 'atf55EBplus' [m], • saw subtype = plunge [b], • power = '1200' [c], weight = '4800' [d], • saw blade speed <= 4800 [e], • saw blade diameter = '160' [f], • bevel cuts <= 45 [g], cutting depth <= 55 [h], • dust extractor connection = 36 [i], • systainer = 'yes'[j], rail guide = 'no'[k], • electronics = 'yes' [l], • RELEVANCE OF CHARACTERISTICS • a ->50%, b ->50%, c -> 50%, d ->100%, • f ->50%, h ->50%, j -> 50%, k ->50%, • l -> 50%, m -> 100%, c,b -> 50%, • b,f -> 50%, b,h -> 50%, • c,b,j,k,l->100%, b,f,j,k,l->100%, b,h,j,k,l->100%

19. Dialogue-based interaction I think I need a pendulum-cut saw IN product type pendulum inform(data, identity, product, saw) inform(data,feature, product, type=pendulum) Add-data(product type, pendulum) l Request (product) Solve task + Request (product) Add + Request (data) NL GENERATOR Satisfy <formal, closed> Request <data, . 1. prune current offer tree 2. If no solution node then generate a new request of data application , product> What do you need it for? OUT Offer decision tree Interface Agent Interaction Agent Session Model . 1.- Recent Discourse Memory 2.- Context : product tool saw NL INTERPRETER Dialogue Manager Thread Joint stack : Command (req-prod) l 3D Character Discourse Maker . :

20. Interface Agent Dialogue Processing Components streams of speech acts NLG Interaction Agent Intelligent Agent streams of speech acts XML sockets problem solving tasks Interface Manager NLA sockets sockets streams of speech acts XML XML sockets Architecture Overview (current prototype)

21. Some preliminary conclusions • The KSM provided the required methodology, but the current model was operationalized using Ciao-Prolog and Java. • Evaluation (focused on some main aspects of the interaction, interventions for achieving a goal, steps made through an intervention, worthless interventions, etc) • the dialogues for achieving goals are long, in general. • the variability in discourses is too low • The performance of the current prototype is good enough ...

22. Next step... • System should short interactions, trying to interrupt user as less as possible, for which is needed to control reactions (react monitor?) • New “user model” approach (clustering?) • Improve the generic and domain models (update approach?) • The NL interpreter and generator has to be enlarged (corpus-based) • Interoperability: ontology for complete the shop system integration • Multi-user functionality (RMI, to a MAS architecture?)