An overview of multi-criteria analysis techniques

1. An overview of multi-criteria analysis techniques The main role of the techniques is to deal with the difficulties that human decision-makers have been shown to have in handling large amounts of complex information in a consistent way. MCA techniques can be used to identify a single most preferred option, to rank options, to short-list a limited number of options for subsequent detailed appraisal, or simply to distinguish acceptable from unacceptable possibilities.

2. There are many MCA techniques and their number is still rising. There are several reasons why this is so: • there are many different types of decision which fit the broad circumstances of MCA • the time available to undertake the analysis may vary • the amount or nature of data available to support the analysis may vary • the analytical skills of those supporting the decision may vary, and • the administrative culture and requirements of organizations vary.

3. Criteria for selecting MCA techniques • internal consistency and logical soundness • transparency • ease of use • data requirements not inconsistent with the importance of the issue being considered • realistic time and manpower resource requirements for the analysis process • ability to provide an audit trail, and • software availability, where needed.

4. ADVANTAGES OF MCA OVER INFORMAL JUDGEMENT • it is open and explicit • the choice of objectives and criteria that any decision making group may make are open to analysis and to change if they are felt to be inappropriate • scores and weights, when used, are also explicit and are developed according to established techniques. They can also be cross-referenced to other sources of information on relative values, and amended if necessary

5. ADVANTAGES OF MCA OVER INFORMAL JUDGEMENT • performance measurement can be sub-contracted to experts, so need not necessarily be left in the hands of the decision making body itself • it can provide an important means of communication, within the decision making body and sometimes, later, between that body and the wider community, and • scores and weights are used, it provides an audit trail.

7. SCORING AND WEIGHTING 1.Scoring: the expected consequences of each option are assigned a numerical score on a strength of preference scale for each option for each criterion. More preferred options score higher on the scale, and less preferred options score lower. In practice, scales extending from 0 to 100 are often used, where 0 represents a real or hypothetical least preferred option, and 100 is associated with a real or hypothetical most preferred option. All options considered in the MCA would then fall between 0 and 100. 2. Weighting: numerical weights are assigned to define, for each criterion, the relative valuations of a shift between the top and bottom of the chosen scale.

8. Different types of MCA An important initial consideration in the choice of MCA technique is that of the number of alternatives to be appraised. Some problems, especially in design and engineering, are concerned with outcomes that are infinitely variable. However, most policy decisions, even at fairly low levels, are usually about choices between discrete options, for example, between alternative investment projects, or between alternative types of tax system.

9. Decisions without weights: construction of a performance matrix The stages of a multi-criteria analysis • each row describes one of the options that are being considered • each column corresponds to a criterion, or 'performance dimension', which is considered important to the comparison of the different options, and • the entries in the body of the matrix assess how well each option performs with respect to each of the criteria.

10. Steps in a multi-criteria analysis 1. Establish the decision context. What are the aims of the MCA, and who are the decision makers and other key players? 2. Identify the options. 3. Identify the objectives and criteria that reflect the value associated with the consequences of each option. 4. Describe the expected performance of each option against the criteria. (If the analysis is to include steps 5 and 6, also ‘score’ the options, i.e. assess the value associated with the consequences of each option.) 5. ‘Weighting’. Assign weights for each of the criteria to reflect their relative importance to the decision. 6. Combine the weights and scores for each of the options to derive and overall value. 7. Examine the results. 8. Conduct a sensitivity analysis of the results to changes in scores or weights.

11. Step 1: Establishing the decision context The decision context is the whole panoply of administrative, political and social structures that surround the decision being made. Central to it are the objectives of the decision making body, the administrative and historical context, the set of people who may be affected by the decision and an identification of those responsible for the decision. It is crucial to have a clear understanding of objectives. To what overall ambition is this decision seeking to contribute? MCA is all about multiple conflicting objectives. There are ultimately trade-offs to be made. Nonetheless, in applying MCA it is important to identify a single high level objective, for which there will usually be sub-objectives. To establish objectives (and criteria) we need to establish both who the decision-makers are (in order to establish objectives) and also people who may be affected by the decision. A common component of this step can be to refer to underlying policy statements.

12. Step 2: Identifying options Having established the decision context, the next step is to list the set of options to be considered. It is unlikely, even with a new and unexpected problem, that the decision making group will arrive at the stage of formal MCA structuring without some intuition about options. Often in practice there will be ideas ‘on the books’, sometimes going back many years. Sometimes the problem will be an embarrassment of possibilities and it will be the role of the MCA in the first instance to provide a structured sifting of alternatives to identify a short-list, using basic data and quick procedures. It is sometimes worth carrying out some informal sifting against established legal and similar restrictions. It is not worth considering and putting effort into gathering data about clearly infeasible propositions.

13. Step 2: Identifying options The first visit to step 2 may well not be the last, particularly in problems where there is a paucity of acceptable alternatives. The later steps of the MCA may demonstrate that none of the alternatives is acceptable and can serve to crystallise thoughts as to where the inadequacies lie. At this stage, fresh ideas and creative thought are needed. This will be informed by the MCA. For example, it may encourage a search for new options that combine the strong points of one existing option in some areas with the strong points of another in a different area. The failure to be explicit about objectives, to evaluate options without considering what is to be achieved. to propose that startingwith options is putting the cart before the horse. Options are important only for the value they create by achieving objectives. It might be better to consider objectives first, particularly when the options are not given and have to be developed.