- 82 Views
- Uploaded on
- Presentation posted in: General

Rank-Based Sensitivity Analysis of Multiattribute Value Models

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Rank-Based Sensitivity Analysis of Multiattribute Value Models

Antti Punkka and Ahti Salo

Systems Analysis Laboratory

Helsinki University of Technology

P.O. Box 1100, 02015 TKK, Finland

http://www.sal.tkk.fi/

- Provides a complete rank-ordering for the alternatives
- Selection of the best alternative
- Rank-ordering of e.g. universities (Liu and Cheng 2005) or graduate programs (Keeney et al. 2006)
- Prioritization of project proposals or innovation ideas (e.g. Könnölä et al. 2007)

- Methods for global sensitivity analysis on weights and scores
- Focus only on the selection of the best alternative
- Ex post: Sensitivity of the decision recommendation to parameter variation
- Ex ante: Computation of viable decision candidates subject to incomplete information about the parameter values
(e.g., Rios Insua and French 1991, Butler et al. 1997, Mustajoki et al. 2006)

- Focus only on the selection of the best alternative

- Consider the full rank-ordering instead of the most preferred alternative
- How ’sensitive’ is the rank-ordering
- How to compare two rank-orderings? How to communicate differences?

- We compute the attainable rankings for each alternative subject to global variation in weights and scores
- How sensitive is the ranking of an alternative subject to parameter variation?
- Is the ranking of university X sensitive to the attribute weights applied?
- What is the best / worst attainable ranking of project proposal Y?

- Model parameter uncertainty before computation
- Relax complete specification of parameters
- ”Error coefficients” on the statements, e.g. weight ratios
- E.g. Mustajoki et al. (2006)

- Directly elicit and apply incomplete information
- Incompletely defined weight ratios: 2 ≤ w3/ w2 ≤ 3
- Ordinal information about weights: w1≤ w3
- Score intervals: 0.4≤ v1(x12) ≤ 0.6
- E.g., Kirkwood and Sarin (1985),
Salo and Hämäläinen (1992), Liesiö et al. (2007)

- Relax complete specification of parameters
- Set of feasible weights and scores (S)

- Existing output concepts of ex ante sensitivity analysis do not consider the full rank-ordering of alternative set X
- Value intervals focus on 1 alternative at a time
- Dominance relations are essentially pairwise comparisons
- Potential optimality focuses on the ranking 1

- Alternative xk can attain ranking r, if exists feasible parameters such that the number of alternatives with higher value is r-1

ranking 1 is attainable for x3

ranking 4 is attainable for x1

w1

ranking 1 is attainable for x2

ranking 3 is attainable for x3

w2

Attainable rankings

- 2 attributes, 4 alternatives with fixed scores, w1 [0.4, 0.7]

V

x1

x2

x3

x4

0.4

0.7

0.6

0.3

- Application of incomplete information set of feasible weights and scores (S)
- If S is convex, all rankings between the best and the worst attainable rankings are attainable
- Best ranking of xk:
- Worst ranking of xk:

- MILP model to obtain the best / worst ranking of each xk
- V(x) expressed in non-normalized form (linear in w and v)
- # of binary variables = |X| - 1

- Shanghai Jiao Tong University ranks the world universities annually
- Example data from 2007
- http://ed.sjtu.edu.cn/ranking2007.htm
- 508 universities

- Additive model for rank-ordering of the universities

Table adopted from http://ed.sjtu.edu.cn/ranking2007.htm

- How sensitive are the rankings to weight variation?
- What if different weights were applied?
- Relax point estimate weighting
1. Relative intervals around the point estimates

- E.g. =20 %, wi*=0.20:
2. Incomplete ordinal information

- Attributes with wi*=0.20 cannot be less important than those with wi*=0.10
- All weights lower-bounded by 0.02

Unsensitive rankings

”Different weighting would

likely yield a better ranking”

exact weights

20 % interval

30 % interval

University

incompl. ordinal

no information

10th

442nd

Ranking

- A model to compute attainable rankings
- Sufficiently efficient even with hundreds of alternatives and several attributes

- Attainable rankings communicate sensitivity of rank-orderings
- Conceptually easy to understand
- Holistic view of global sensitivity at a glance independently of the # of attributes

- Applicable output in Preference Programming framework
- Additional information leads to fewer attainable rankings

- Connections to project prioritization
- Initial screening of project proposals for e.g. portfolio-level analysis
- Supports identification of ’clear decisions’ (cf. Liesiö et al. 2007)
- ”Select the ones ’surely’ in top 50”
- ”Discard the ones ’surely’ outside top 50”

- Butler, J., Jia, J., Dyer, J. (1997). Simulation Techniques for the Sensitivity Analysis of Multi-Criteria Decision Models. EJOR 103, 531-546.
- Keeney, R.L., See, K.E., von Winterfeldt, D. (2006). Evaluating Academic Programs: With Applications to U.S. Graduate Decision Science Programs. Oper. Res. 54, 813-828.
- Kirkwood, G., Sarin R. (1985). Ranking with Partial Information: A Method and an Application. Oper. Res. 33, 38-48
- Könnölä, T., Brummer, V., Salo A. (2007). Diversity in Foresight: Insights from the Fostering of Innovation Ideas. Technologial Forecasting & Social Change 74, 608-626.
- Liesiö, J., Mild, P., Salo, A., (2007). Preference Programming for Robust Portfolio Modeling and Project Selection. EJOR 181, 1488-1505.
- Liu, N.C., Cheng, Y. (2005). The Academic Ranking of World Universities. Higher Education in Europe 30, 127-136
- Mustajoki, J., Hämäläinen, R.P., Lindstedt, M.R.K. (2006). Using intervals for Global Sensitivity and Worst Case Analyses in Multiattribute Value Trees. EJOR 174, 278-292.
- Rios Insua, D., French, S. (1991). A Framework for Sensitivity Analysis in Discrete Multi-Objective Decision-Making. EJOR 54, 176-190.
- Salo, A., Hämäläinen R.P. (1992). Preference assessment by imprecise ratio statements. Oper. Res. 40, 1053-1061.