Informs Annual Meeting 2012, Phoenix, AZ

1. Caveats of Decision Rules for Comparing Alternatives under Incomplete Preference Information Antti Punkka and Ahti Salo Systems Analysis Laboratory Department of Mathematics and Systems Analysis Aalto University School of Science Informs Annual Meeting 2012, Phoenix, AZ

2. Incomplete preference information about additive value functions • Additive value functions elicited through tradeoff statements • E.g. series of pairs ,such that • and describe same preferences • Incomplete preference information1: ”looser statements” • Consistent value functions satisfy the preference statements; they include also functions which describe different preferences (= are not positive affine transformations of each other) • If for all consistent value functions and for some, then alternative dominates1 alternative • We show that many DECISION RULES for comparing non-dominated alternatives can exhibit rank reversals 1 e.g., Kirkwood and Sarin (Man.Sci., 1985), Hazen (Oper.Res., 1986), Salo and Hämäläinen (Oper.Res., 1992)

3. Example • DM compares baskets containing apples and oranges • Linear attribute-specific value functions • Incomplete preference information: and • All additive value functions with are consistent with this information • But so are with • DM asks 2 experts to compare baskets and • Both note that the baskets are non-dominated, because can be positive and negative

4. Experts apply decision rules to obtain decision recommendations • Expert 1normalizes all value functions so that • Expert 2 normalizes all value functions so that (2,1) is the maximax2 solution (1,2) is the maximin2 solution 0.125 0.1 Value Value (2,1) is the weak dominance2,3 solution (1,2) is the maximax2 solution (2,1) is the maximin2 solution (2,1) is the quasi-dominance4 solution with all tolerances in [0.1,0.125) (1,2) is the weak dominance2,3 solution (1,2) is the quasi-dominance4solution... 2 Salo and Hämäläinen (IEEE Transactions on SMC-A, 2001) 3 Park and Kim (EJOR, 1997) 4 Sarabando and Dias (IEEE Transactions on SMC-A, 2009)

5. Where are the attribute weights? • Expert 1 uses : • Bounds on parameter become bounds on attribute weights • E.g. Expert 1: Value Value Expert 1 Expert 2 (1,2) is the domain criterion5 solution (2,1) is the domain criterion5 solution 5 Eiselt and Laporte (EJOR, 1992)

6. Normalization is one possibility to choose representative value functions • All value functions describe the same preferences as • Numerical values and value differences not unique even with complete preference information • For numerical comparisons of values and value differences, one value function needs to be chosen as the representative one • With incomplete preferences, each consistent value function is represented by one of its positive affine transformations

7. Standard procedure for choosing representative value functions • Select and such that for all consistent value functions • Normalize the values of these alternatives: for all representative value functions • Note: Change of normalization implies some, but not necessarily the same positive affine transformation for each value function

8. Revisiting the visualizations • Expert 1: Representative value functions such that . Examined the value difference • Expert 2: Representative value functions such that . Examined the value difference • With ] • With ] • Indeed: , when ; but , when

9. Representative value functions and ordinal comparisons • Representative value functions contain all information about rankings of values and value differences • For any : with positive • It suffices to compute dominance relations using representative value functions • Dominance relations do not depend on what representative value functions are chosen • Examine alternatives’ rankings with all consistent value functions

10. x3 can have ranking 1 x1 can have ranking 4 w1 x2can have ranking 1 x3 can have ranking 3 w2 Ranking intervals Ranking intervals • Rank-order alternatives with all consistent value functions V x1 x2 x3 x4 0.4 0.7 0.6 0.3

11. Robust rankings ”Different weighting would likely yield a better ranking” Ranking intervals for sensitivity analysis: Academic Ranking of World Universities exact weights 20 % interval 30 % interval University incompl. ordinal no information 442nd 10th Ranking

12. Ranking intervals for project portfolio selection – a case study6 revisited DISCARD CONSIDER CHOOSE Ranking intervals without any preference information about the relative values of the three attributes 6 Data and case example from Könnölä et al. (Technological Forecasting & Social Change, 2007)

13. Conclusions • Normalization • Not implied by preference statements • Not needed to compare alternatives, but carried out to associate numerical values with alternatives • Does not affect results with complete preference information • With incomplete information about relative importance of attributes,many value functions that describe different preferences are consistent with stated information • Numerical comparisons of value differences across these value functions do not provide meaningful results • Values and value differences should always be compared with the same value function • Use dominance relations and ranking intervals