1 / 48

Designing Example Critiquing Interaction

IUI 2004, Madeira, Portugal – Wed Jan 14, 2004. Designing Example Critiquing Interaction. Boi Faltings Pearl Pu Marc Torrens Paolo Viappiani. LIA. HCI. Outline. Introduction Stimulating expression of preferences Guaranteeing optimal solutions Conclusion. Motivation.

maximilian-rankin
Download Presentation

Designing Example Critiquing Interaction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IUI 2004, Madeira, Portugal – Wed Jan 14, 2004 Designing Example Critiquing Interaction Boi Faltings Pearl Pu Marc Torrens Paolo Viappiani LIA HCI

  2. Outline • Introduction • Stimulating expression of preferences • Guaranteeing optimal solutions • Conclusion Designing Example Critiquing Interaction

  3. Motivation • Many real word applications require people to select a most preferred outcome from a large set of possibilities (electronic catalogs) • Users are usually unable to correctly state their preferences up front • People are greatly helped by seeing examples of actual solutions • example critiquing Designing Example Critiquing Interaction

  4. Mixed Initiative Interaction initial preference The user critiques the solutions stating a new preference the systemshows K solutions The user picks the final choice Designing Example Critiquing Interaction

  5. An implementation: reality trade-off between different criteria user critiques existing solutions Designing Example Critiquing Interaction

  6. What to show? • Standard approach • show the best solutions • assumption: user model is complete and accurate • Does not in general stimulate new preferences Designing Example Critiquing Interaction

  7. New approach • Display_set = stimulate_set +optimal_set Designing Example Critiquing Interaction

  8. What to show? • Stimulate set = solutions that • make the user aware of attributes diversity • have high probability to become optimal if new preferences are stated • Optimal set = solutions that • are optimal given the current preferences Designing Example Critiquing Interaction

  9. Outline • Introduction • Stimulating expression of preferences • Guaranteeing optimal solutions • Conclusion Designing Example Critiquing Interaction

  10. Stimulating new preferences • Pareto optimality • general concept • does not involve weights • Dominated solution can become Pareto optimal if new preferences are stated  show solutions that have higher probability of becoming Pareto optimal Designing Example Critiquing Interaction

  11. Dominance relation and Pareto optimality Penaltytable, 2 preferences P2 s4 9 s5 6 s3 s1 • S1 and S2 are Pareto optimal • S3 is dominated by S1 and S2 • S4 is dominated by S1 • S5 is dominated by S1, S2, S3. 3 s2 3 6 9 P1 Designing Example Critiquing Interaction

  12. Pareto Optimal Filters • Estimate the probability that a dominated solution can become Pareto optimal when new preferences are stated • Different Pareto-filters: • counting filter • attribute filter • probabilistic filter Designing Example Critiquing Interaction

  13. Counting filter • We count the number of dominators • S1 and S2 are currently “optimal” • S4 more promising than S3 and S5 Counting Filter: number of Dominators Designing Example Critiquing Interaction

  14. A new preference is added • New column with penalties • S4 becomes Pareto optimal even if the new penalty (0.6) is worse than for S3 (0.5) and S5 (0.4) The counting filter predict that S4 has better chances to become P.O. when a new preference is added. Designing Example Critiquing Interaction

  15. Hasse diagrams Pareto Optimal 1 2 4 1 2 4 3 3 5 User Model={p1,p2, p3} 5 User Model={p1,p2} Adding preferences: Pareto optimal set grows, dominance relation becomes sparse Designing Example Critiquing Interaction

  16. Attribute filter • Solution: n attributes: A1,..,An • D1,..,Dn domains for A1,..,An • a solution is a complete assignment • Preferences modeled as penalty functions defined on attribute domains • Look at the attribute space • if two values are the same, any penalty function defined on these values will be the same Designing Example Critiquing Interaction

  17. Attribute filter: motivation • S2 and S3 are both dominated by S1 • If we add new preference • on Location  if North is preferred S2 will be Pareto Optimal • on Transport  if Tramway is preferred to Bus then S2 will be P.O. • S3 will always be dominated!! Preferences: on price (to minimize), on M2 (to maximize) Designing Example Critiquing Interaction

  18. Attribute Filter • For the new preference, dominated solution s must have lower penalty than all dominant solutions • for discrete domain, attribute values must be different • for continuous domains, consider extreme values Designing Example Critiquing Interaction

  19. Probabilistic filter • Directly estimate probability of becoming P.O. • The bigger the difference on a specific attribute, the more likely the penalties will be different penalty penalty 1 1 domain domain Designing Example Critiquing Interaction

  20. Experiments • Database of actual accommodation offers (room for rent, studios, apartments) • Random datasets • 11 attributes of which • 4 continuous (price, duration, square meters, distance to university) • 7 discrete (kitchen, kitchen type, bathroom, public transportation, ..) Designing Example Critiquing Interaction

  21. Results (accommodation dataset) Designing Example Critiquing Interaction

  22. Results (random dataset) Average fraction of correct predictions number of preferences known Designing Example Critiquing Interaction

  23. Outline • Introduction • Stimulating expression of preferences • Guaranteeing optimal solutions • Conclusion Designing Example Critiquing Interaction

  24. Modelling • True preference model P* (unknown) • P*={p*1, p*2, .., p*k} • st: target solution • Estimated through a model P • P={p1,..,pk} • pi are built-in standard penalty functions • assume limited difference between p and p* • Penalty functions • pi(ak): dk -> R • write pi(s) instead of pi(aj(s)) Designing Example Critiquing Interaction

  25. Selecting displayed solutions • Dominance filters • Utilitarian filters • Egalitarian filters Designing Example Critiquing Interaction

  26. Optimal Set Filters Properties We want.. • To show a limited number of solutions • each filter selects k solutions to display • To ensure that a Pareto-optimal solution in D is Pareto-optimal in S • each filter satisfies this • dominance filter (by definition), Utilitarian and Egalitarian (theorem) Designing Example Critiquing Interaction

  27. Optimal Set Filters Properties • To include target solutions • only if target solution is included the user can choose it! • probability to include the target solution in D depends on filter. Assumption (1-ε)pi ≤ pi* ≤ (1+ε)pi Designing Example Critiquing Interaction

  28. Dominance filter • Display k solutions that are not dominated by another one Designing Example Critiquing Interaction

  29. Dominance filter: target solution • Plot of probability of target solution being included in D, as function of number of preferences • |P|=3,..,12 • K=30, 60 • m=778, 6444 Designing Example Critiquing Interaction

  30. Utilitarian filter • We minimize the un-weighted sum of penalties • Efficiently computed by “branch & bound” Designing Example Critiquing Interaction

  31. Utilitarian filter: probability to find target solution • Does not depend on m, the number of total solutions (proved analytically) • Better than dominator filter Designing Example Critiquing Interaction

  32. Egalitarian filter • Minimize F(s) • In case of equality, use lexicographic order: (0.4, 0.2) preferred to (0.4, 0.4) • Target solution inclusion probability similar to that of the Utilitarian filter. Designing Example Critiquing Interaction

  33. Fraction of PO solutions shown within the best k ones Egalitarian, utilitarian filter Robustness against violated assumption Designing Example Critiquing Interaction

  34. Outline • Introduction • Stimulating expression of preferences • Guaranteeing optimal solutions • Conclusion Designing Example Critiquing Interaction

  35. Conclusion • Optimal and stimulation set • Example critiquing on firmer mathematical ground • Suggestions to system developer • How to compensate an incomplete/inaccurate user model • Experimental evaluation on real and random problems Designing Example Critiquing Interaction

  36. Questions Designing Example Critiquing Interaction

  37. Designing Example Critiquing Interaction

  38. Pareto Filters: conclusions • Counting filter works already fairly well • Attribute filter works very well when only 1 or 2 preferences are missing, but generally probabilistic is the best • Impact of correlation between attributes can affect performance Random Counting Attribute Probabilistic Complexity Designing Example Critiquing Interaction

  39. Attribute filter/2 • Continuous domains: best values are the extremes • assumption: preference functions are monotonic penalty OR 1 domain Designing Example Critiquing Interaction

  40. 1 domain domain ai(o2) ai(o1) θ ai(o2) ai(o1) m 1 1 domain domain θ θ ai(o1) ai(o2) ai(o1) ai(o2) Designing Example Critiquing Interaction

  41. 1 domain domain ai(o) li ai(o) θ li m2 m1 1 1 ai(o) domain domain gi si θ-t ai(o) θ ai(o1)-t θ li θ+t Designing Example Critiquing Interaction

  42. Designing Example Critiquing Interaction

  43. Theorem • Given a set of m solutions S={s1,..,sm} and a set of penalties {p1,..,pd} • Let S’ be the best k solutions according to the utilitarian filter • A solution s in S’ not dominated by any other of S’, is Pareto Optimal in S. Designing Example Critiquing Interaction

  44. Simplified Apartment Domain • A very simple example: • A={Location, Rent, Rooms} • DLocation={Centre, North, South, East, West} • DRent={x|x integer x>0} • DRooms={1,2,3..} Preferences: location should be centre and rent less than 500 Designing Example Critiquing Interaction

  45. Penalty functions • P1:= • if (Location==centre) then 0 else 1 • P2:= • If (Rent > 500) then K*(Rent-500) • Else 0 Designing Example Critiquing Interaction

  46. Electronic catalogues • K attributes: A1,..,Ak • D1,..,Dk domains for A1,..,Ak • a solution is a complete assignment • write aj(s), value of S for attribute j • Solution set S • is a subset of D1 x D2 x D3 x D4 x ... • Preferences modeled as penalty functions defined on attribute domains Designing Example Critiquing Interaction

  47. Counting filter* The Dominator set for a solution s1, is the subset of S of solution that dominates s1. The counting filter orders solutions on the size of the dominator set. Sd(s1) s1 Designing Example Critiquing Interaction

  48. Probabilistic filter • Directly estimate probability of becoming P.O. • The bigger the difference on a specific attribute, the more likely the penalties will be different penalty 1 domain Designing Example Critiquing Interaction

More Related