480 likes | 601 Views
IUI 2004, Madeira, Portugal – Wed Jan 14, 2004. Designing Example Critiquing Interaction. Boi Faltings Pearl Pu Marc Torrens Paolo Viappiani. LIA. HCI. Outline. Introduction Stimulating expression of preferences Guaranteeing optimal solutions Conclusion. Motivation.
E N D
IUI 2004, Madeira, Portugal – Wed Jan 14, 2004 Designing Example Critiquing Interaction Boi Faltings Pearl Pu Marc Torrens Paolo Viappiani LIA HCI
Outline • Introduction • Stimulating expression of preferences • Guaranteeing optimal solutions • Conclusion Designing Example Critiquing Interaction
Motivation • Many real word applications require people to select a most preferred outcome from a large set of possibilities (electronic catalogs) • Users are usually unable to correctly state their preferences up front • People are greatly helped by seeing examples of actual solutions • example critiquing Designing Example Critiquing Interaction
Mixed Initiative Interaction initial preference The user critiques the solutions stating a new preference the systemshows K solutions The user picks the final choice Designing Example Critiquing Interaction
An implementation: reality trade-off between different criteria user critiques existing solutions Designing Example Critiquing Interaction
What to show? • Standard approach • show the best solutions • assumption: user model is complete and accurate • Does not in general stimulate new preferences Designing Example Critiquing Interaction
New approach • Display_set = stimulate_set +optimal_set Designing Example Critiquing Interaction
What to show? • Stimulate set = solutions that • make the user aware of attributes diversity • have high probability to become optimal if new preferences are stated • Optimal set = solutions that • are optimal given the current preferences Designing Example Critiquing Interaction
Outline • Introduction • Stimulating expression of preferences • Guaranteeing optimal solutions • Conclusion Designing Example Critiquing Interaction
Stimulating new preferences • Pareto optimality • general concept • does not involve weights • Dominated solution can become Pareto optimal if new preferences are stated show solutions that have higher probability of becoming Pareto optimal Designing Example Critiquing Interaction
Dominance relation and Pareto optimality Penaltytable, 2 preferences P2 s4 9 s5 6 s3 s1 • S1 and S2 are Pareto optimal • S3 is dominated by S1 and S2 • S4 is dominated by S1 • S5 is dominated by S1, S2, S3. 3 s2 3 6 9 P1 Designing Example Critiquing Interaction
Pareto Optimal Filters • Estimate the probability that a dominated solution can become Pareto optimal when new preferences are stated • Different Pareto-filters: • counting filter • attribute filter • probabilistic filter Designing Example Critiquing Interaction
Counting filter • We count the number of dominators • S1 and S2 are currently “optimal” • S4 more promising than S3 and S5 Counting Filter: number of Dominators Designing Example Critiquing Interaction
A new preference is added • New column with penalties • S4 becomes Pareto optimal even if the new penalty (0.6) is worse than for S3 (0.5) and S5 (0.4) The counting filter predict that S4 has better chances to become P.O. when a new preference is added. Designing Example Critiquing Interaction
Hasse diagrams Pareto Optimal 1 2 4 1 2 4 3 3 5 User Model={p1,p2, p3} 5 User Model={p1,p2} Adding preferences: Pareto optimal set grows, dominance relation becomes sparse Designing Example Critiquing Interaction
Attribute filter • Solution: n attributes: A1,..,An • D1,..,Dn domains for A1,..,An • a solution is a complete assignment • Preferences modeled as penalty functions defined on attribute domains • Look at the attribute space • if two values are the same, any penalty function defined on these values will be the same Designing Example Critiquing Interaction
Attribute filter: motivation • S2 and S3 are both dominated by S1 • If we add new preference • on Location if North is preferred S2 will be Pareto Optimal • on Transport if Tramway is preferred to Bus then S2 will be P.O. • S3 will always be dominated!! Preferences: on price (to minimize), on M2 (to maximize) Designing Example Critiquing Interaction
Attribute Filter • For the new preference, dominated solution s must have lower penalty than all dominant solutions • for discrete domain, attribute values must be different • for continuous domains, consider extreme values Designing Example Critiquing Interaction
Probabilistic filter • Directly estimate probability of becoming P.O. • The bigger the difference on a specific attribute, the more likely the penalties will be different penalty penalty 1 1 domain domain Designing Example Critiquing Interaction
Experiments • Database of actual accommodation offers (room for rent, studios, apartments) • Random datasets • 11 attributes of which • 4 continuous (price, duration, square meters, distance to university) • 7 discrete (kitchen, kitchen type, bathroom, public transportation, ..) Designing Example Critiquing Interaction
Results (accommodation dataset) Designing Example Critiquing Interaction
Results (random dataset) Average fraction of correct predictions number of preferences known Designing Example Critiquing Interaction
Outline • Introduction • Stimulating expression of preferences • Guaranteeing optimal solutions • Conclusion Designing Example Critiquing Interaction
Modelling • True preference model P* (unknown) • P*={p*1, p*2, .., p*k} • st: target solution • Estimated through a model P • P={p1,..,pk} • pi are built-in standard penalty functions • assume limited difference between p and p* • Penalty functions • pi(ak): dk -> R • write pi(s) instead of pi(aj(s)) Designing Example Critiquing Interaction
Selecting displayed solutions • Dominance filters • Utilitarian filters • Egalitarian filters Designing Example Critiquing Interaction
Optimal Set Filters Properties We want.. • To show a limited number of solutions • each filter selects k solutions to display • To ensure that a Pareto-optimal solution in D is Pareto-optimal in S • each filter satisfies this • dominance filter (by definition), Utilitarian and Egalitarian (theorem) Designing Example Critiquing Interaction
Optimal Set Filters Properties • To include target solutions • only if target solution is included the user can choose it! • probability to include the target solution in D depends on filter. Assumption (1-ε)pi ≤ pi* ≤ (1+ε)pi Designing Example Critiquing Interaction
Dominance filter • Display k solutions that are not dominated by another one Designing Example Critiquing Interaction
Dominance filter: target solution • Plot of probability of target solution being included in D, as function of number of preferences • |P|=3,..,12 • K=30, 60 • m=778, 6444 Designing Example Critiquing Interaction
Utilitarian filter • We minimize the un-weighted sum of penalties • Efficiently computed by “branch & bound” Designing Example Critiquing Interaction
Utilitarian filter: probability to find target solution • Does not depend on m, the number of total solutions (proved analytically) • Better than dominator filter Designing Example Critiquing Interaction
Egalitarian filter • Minimize F(s) • In case of equality, use lexicographic order: (0.4, 0.2) preferred to (0.4, 0.4) • Target solution inclusion probability similar to that of the Utilitarian filter. Designing Example Critiquing Interaction
Fraction of PO solutions shown within the best k ones Egalitarian, utilitarian filter Robustness against violated assumption Designing Example Critiquing Interaction
Outline • Introduction • Stimulating expression of preferences • Guaranteeing optimal solutions • Conclusion Designing Example Critiquing Interaction
Conclusion • Optimal and stimulation set • Example critiquing on firmer mathematical ground • Suggestions to system developer • How to compensate an incomplete/inaccurate user model • Experimental evaluation on real and random problems Designing Example Critiquing Interaction
Questions Designing Example Critiquing Interaction
Pareto Filters: conclusions • Counting filter works already fairly well • Attribute filter works very well when only 1 or 2 preferences are missing, but generally probabilistic is the best • Impact of correlation between attributes can affect performance Random Counting Attribute Probabilistic Complexity Designing Example Critiquing Interaction
Attribute filter/2 • Continuous domains: best values are the extremes • assumption: preference functions are monotonic penalty OR 1 domain Designing Example Critiquing Interaction
1 domain domain ai(o2) ai(o1) θ ai(o2) ai(o1) m 1 1 domain domain θ θ ai(o1) ai(o2) ai(o1) ai(o2) Designing Example Critiquing Interaction
1 domain domain ai(o) li ai(o) θ li m2 m1 1 1 ai(o) domain domain gi si θ-t ai(o) θ ai(o1)-t θ li θ+t Designing Example Critiquing Interaction
Theorem • Given a set of m solutions S={s1,..,sm} and a set of penalties {p1,..,pd} • Let S’ be the best k solutions according to the utilitarian filter • A solution s in S’ not dominated by any other of S’, is Pareto Optimal in S. Designing Example Critiquing Interaction
Simplified Apartment Domain • A very simple example: • A={Location, Rent, Rooms} • DLocation={Centre, North, South, East, West} • DRent={x|x integer x>0} • DRooms={1,2,3..} Preferences: location should be centre and rent less than 500 Designing Example Critiquing Interaction
Penalty functions • P1:= • if (Location==centre) then 0 else 1 • P2:= • If (Rent > 500) then K*(Rent-500) • Else 0 Designing Example Critiquing Interaction
Electronic catalogues • K attributes: A1,..,Ak • D1,..,Dk domains for A1,..,Ak • a solution is a complete assignment • write aj(s), value of S for attribute j • Solution set S • is a subset of D1 x D2 x D3 x D4 x ... • Preferences modeled as penalty functions defined on attribute domains Designing Example Critiquing Interaction
Counting filter* The Dominator set for a solution s1, is the subset of S of solution that dominates s1. The counting filter orders solutions on the size of the dominator set. Sd(s1) s1 Designing Example Critiquing Interaction
Probabilistic filter • Directly estimate probability of becoming P.O. • The bigger the difference on a specific attribute, the more likely the penalties will be different penalty 1 domain Designing Example Critiquing Interaction