1 / 39

Stimulating Preference Expression using Suggestions

This presentation discusses the use of suggestions to stimulate preference expression in mixed initiative interactions, focusing on example critiquing and Pareto suggestion strategies.

kmaas
Download Presentation

Stimulating Preference Expression using Suggestions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AAAI Fall Symposium Mixed Initiative Interaction Stimulating Preference Expression using Suggestions Paolo Viappiani1 (speaker), Boi Faltings1 Vincent Schickel Zuber1, Pearl Pu2 Artificial Intelligence Lab (LIA) Human Computer Interaction Group (HCI) École Polytechnique Fédérale Lausanne Switzerland AAAI Fall Symposium 05

  2. Agenda • ExampleCritiquing 2. Stimulate Preference Expression 3. Pareto Suggestion Strategies 4. Evaluation AAAI Fall Symposium 05

  3. How do we usually search for products on the web? AAAI Fall Symposium 05

  4. Sorry there is no solution! AAAI Fall Symposium 05

  5. AAAI Fall Symposium 05

  6. Far too expensive! Go Back AAAI Fall Symposium 05

  7. index.html <HTML> <FORM ACTION=“..”> … </FORM> ..</HTML> <HTML> <TABLE ..> … </TABLE> ..</HTML> query • REPONSE • “sorry there are no results”  have to start from scratch • results not satisfactory  back-button • no checks that the user has really stated all that he wanted! database AAAI Fall Symposium 05

  8. Questions form-filling • Users feel obliged to fill in many of the preferences • Stating preferences without having seen any example isinaccurate • Users don’t know what the attributes mean • Users state means objectives AAAI Fall Symposium 05

  9. Means-objectives • True objective: cheap accommodation • System asks for preferred type • Means objective “I prefer a shared-apartment” [Keeney, Value Focused Thinking] • Means objectives lead to inaccurate decisions (only 25% accurate) AAAI Fall Symposium 05

  10. Example Critiquing • Mixed initiative system • Shows examples of complete solutions • Invites users to construct and revise their preferences through critiques • cooperation between user and the system AAAI Fall Symposium 05

  11. Example Critiquing Initial preferences User revises the preference model by critiquing examples System shows K solutions User picks the final choice AAAI Fall Symposium 05

  12. Example: Isy-travel [2000] AAAI Fall Symposium 05

  13. What to show? • Candidates = the best K given a utility model for current preferences • Suggestions = solutions that stimulate the user in stating his (true) preferences Maximize a combination of all preference orders Our job! AAAI Fall Symposium 05

  14. Suggest Extreme options • [Linden et al. 1997] • Often unreasonable: • Cheapest flight • but leave at 1a.m. • Closest apartment to centre • but no parking • Cheapest student accommodation = 0 $ • but have to work as “au pair” take care the family children • Too many to choose from: two for each attribute! AAAI Fall Symposium 05

  15. User states his preferences only if he expects that it will have some impact on the solutions • Suggestions should be options that could become optimal when an additional preference is stated Optimality definition depends on preference modeling  Pareto Optimality & Dominance Relation AAAI Fall Symposium 05

  16. Dominance relation • A dominates B if A not worse than B wrt to all preferences and A strictly better than B for at least one preference • A dominates B (B dominated by A) if A better wrt all preferences • Pareto-optimal are the non dominated options • Example: • A1={suburbs, downtown} • A2={apartment, house} • A3={expensive, cheap} • There is no HOUSE available in DOWNTOWN AAAI Fall Symposium 05

  17. Downtown > Suburbs • SuburbsHouseCheapdominated by: {DowntownAptExpensive,DowntownAptCheap} • + Cheap > Expensive • SuburbsHouseCheapdominated by: {DowntownAptCheap} • + House > Apartment • SuburbsHouseCheap no longer dominated • we have another Pareto-optimal option • Dominated options can become Pareto-optimal when adding preferences AAAI Fall Symposium 05

  18. There are no houses downtown! Downtown,House,Cheap DowntownHouseExpensive SuburbsHouseCheap DowntownAptCheap SuburbsAptCheap SuburbsHouseExpensive DowntAptExpensive SuburbsApartmentExpensive Pareto Optimal = configuration not possible AAAI Fall Symposium 05

  19. Assumption about the User The user will state his hidden preference on attribute i if The system shows an option that will become Pareto Optimalif that preference is stated AAAI Fall Symposium 05

  20. Suggestions strategies Extrema Probabilistic Counting Attribute To become optimal an option there should exist an attribute on which it is different/extreme than all dominating options. how many options dominate the current one? Maximize the probability that dominance will be broken (through heuristic measure). Pick options showing extreme values. AAAI Fall Symposium 05

  21. You are looking for an accommodation • cheaper than 700chf • Not too far from the university (10 minutes car distance) AAAI Fall Symposium 05

  22. Preferences: PRICE<700, DIST_UNIV<10 SUGGESTIONS RANDOM EXTREME AAAI Fall Symposium 05

  23. Preferences: PRICE<700, DIST_UNIV<10 SUGGESTIONS COUNTING STRATEGY ATTRIBUTE STRATEGY PROBABILISTIC STRATEGY AAAI Fall Symposium 05

  24. AAAI Fall Symposium 05

  25. User Test • 54 users • Interface C: • only shows candidates • Interface CS: • shows candidates+suggestions AAAI Fall Symposium 05

  26. Between groups • Group A: shown interface C • Group B: shown interface CS • tStudent, B>>A 99% confidence AAAI Fall Symposium 05

  27. Within group A • group A • use C • then, use C+S • group A using C + CS  group B using CS AAAI Fall Symposium 05

  28. Work in progress • Online user study monitoring real user looking for an accommodation • New supervised user study measuring real accuracy AAAI Fall Symposium 05

  29. Measuring Decision Accuracy Step 1 Step 2 Step 3 EC interface with: 6 optimal EC interface with: 3 optimal 3 suggestions Full list choice choice choice =? AAAI Fall Symposium 05

  30. User study on Decision Accuracy • 60 subjects • Apartment search • Supervised test AAAI Fall Symposium 05

  31. Conclusions • Standard approaches (form-filling) lead users to state wrong preferences • Mixed-initiative systems help people construct the preference model • Dramatically increases decision accuracy • Suggestions are important AAAI Fall Symposium 05

  32. Attribute filter: motivation • S2 and S3 are both dominated by S1 • If we add new preference • on Location  if North is preferred S2 will be Pareto Optimal • on Transport  if Tramway is preferred to Bus then S2 will be P.O. • S3 will always be dominated!! Preferences: on price (to minimize), on M2 (to maximize) AAAI Fall Symposium 05

  33. Dominance relation and Pareto optimality Penaltytable, 2 preferences P2 s4 9 s5 6 s3 s1 • S1 and S2 are Pareto optimal • S3 is dominated by S1 and S2 • S4 is dominated by S1 • S5 is dominated by S1, S2, S3. 3 s2 3 6 9 P1 AAAI Fall Symposium 05

  34. A new preference is added • New column with penalties • S4 becomes Pareto optimal even if the new penalty (0.6) is worse than for S3 (0.5) and S5 (0.4) The counting filter predict that S4 has better chances to become P.O. when a new preference is added. AAAI Fall Symposium 05

  35. Attribute filter: motivation • S2 and S3 are both dominated by S1 • If we add new preference • on Location  if North is preferred S2 will be Pareto Optimal • on Transport  if Tramway is preferred to Bus then S2 will be P.O. • S3 will always be dominated!! Preferences: on price (to minimize), on M2 (to maximize) AAAI Fall Symposium 05

  36. SUGGESTIONS COUNTING FILTER ATTRIBUTE FILTER PROBABILISTIC FILTER AAAI Fall Symposium 05

  37. Impact of number of preferences 100 sims, 9 preferences, 9 attributes % average fraction of preferences discovered AAAI Fall Symposium 05

  38. Impact of number of attributes 100 sims, 6 preferences, 6/9/12 attributes % of correctly discovered preferences AAAI Fall Symposium 05

  39. Probabilistic filter • Directly estimate probability of becoming P.O. • The bigger the difference on a specific attribute, the more likely the penalties will be different penalty penalty 1 1 domain domain AAAI Fall Symposium 05

More Related