Lihong Zhu 1 , John Holt 2 & Rob Black 3

Incorporating weighting into risk assessment: can this make an overall risk rating more meaningful? Lihong Zhu1, John Holt2 & Rob Black3 Ministry of Agriculture and Forestry, New Zealand Lihong.Zhu@maf.govt.nz Natural Resources Institute, University of Greenwich, UK Department of Law, University of Greenwich, UK

Outlines • Introduction • Why weighting? • How weighting? • What can weighting do for us? • Discussions and Conclusions

Pest risk assessment • Essential stage in PRA: risk assessment • Assessing the likelihood and consequences of pest introduction • Evaluating whether the risk is significant and therefore should be regulated • Criteria including entry, establishment, spread and consequences

Pest risk assessment • Risk factors identified by subdividing each criterion into a series of tangible risk factors, • A risk score attributed to each risk factor • Overall assessment based on a synthesis of the scores: • Simple everage • Weighted average • High & biased weighted everage (Zhu etc 2000, Holt 2005)

Weighting the risk element • But how to achieve a more logical & biologically meaningful overall risk rating? • incorporating weighting i.e. a weighting is given to each risk factor/element • a weighting is a value given to a risk factor according to how important it is perceived to be, or how significant it contribute to the overall risk rating: the larger the value, the more important the factor • Why give weighting? • What weighting systems are adopted? • How to derive weighting?

Table 1. Disadvantage of simple average: A hypothetical example illustrates the impact of simple average of risk factor scores.

Why give weighting? • The risk factors are not all equally important • Weighting should be given to each risk factor to reflect its perceived importance • Those more important should properly contribute more to the final result (i.e. estimated overall risk) than those less important

Weighting systems • How many risk elements being taken into account • 0-2 or 0-3 weighting systems • 0-1 weighting system • Associated combining formulae needed

How to derive weightings 0-1 weighting system: • Weighting is set at 0-1 scale, the higher the weighting, the more important the risk element • Summation of weightings for all risk elements equals 1 • Therefore the overall risk score is normalised and is independent from the number of risk elements

How to derive weightings • Ranking the risk elements • Converting ranks to numerical weightings between 0 and 1 • Effectively, a linear rescaling of the rankings is calculated such that the sum = 1

How to derive weightings • Derived from expert opinion: Delphi technique is a tool to achieve a consented set of rankings from a pool of experts • Derived from multivariate data analysis: principle components analysis (PCA)

Weighting derived from principal components analysis (PCA) • PCA - a data transformation technique, which can reduce the number of variables whilst accounts for the most of total variance • PCA applied to PRA: • High variability in a risk factor means that it has the potential to discriminate the level of risk between cases. Variance is a measure of dispersion of risk scores around the mean of each risk factor (variable) • Some risk factors are correlated

PCA • PCA was performed for data of 264 species and subset data of individual pest groups, 7 risk factors used • Latent vector and percentage variation of principal component (PC) were computed for data of all species (Table 3) • The first 5 PC axes account for almost 90% of the total variation, which means number of principal risk factor can be reduced • Economic impact and host range have the heaviest loadings in PC1 that counts for 38.32% variation, which suggests they are the two most important factors considered while assessing pest risks

Table 3. Loadings & percentage variation explained by each PC

PCA – weightings for various pest categories Weightings of individual risk factors for various pest categories (Table 4) • Risk factors were sorted by weighting rank for all species • Weightings of risk factors were not significantly different because of the small number of variables • Weightings for some pest categories (nematode, bacteria, phytoplasma and mite) were considered unacceptable because of insufficient case of PRA data

Table 4. Weightings for various pest taxa

Is it possible to have a generalised weighting pattern? Rank correlation of weightings for various pest categoriesexamined by Spearman test, it revealed that • No any pair of pest categories are significantly correlated on weighting rank, which means it is difficult to find a general pattern of weighting that suits all the pest categories • Insecta and Fungi have the most similar weighting rank; correlations exists between Insecta and nematodes; Nematodes and Bacteria have the most contrary weighting rank

What can weighting do? Summary assessment scheme • Summary assessment – a quick qualitative risk assessment for immediate action or with limited information: low, medium, or high risk? Scheme comprised (Figure 1): • Key risk factors identified producing most of the risk • Weightings derived showing the relative importance of each factor • Correlations analysed showing the relationships of key factors

Figure 1. Correlations & PCA derived weightings in a summary risk assessment

Does summary scheme work? • Summary risk rating derived from weighted averaging (7 factors involved), in comparison with averaged overall risk and weighed averaged risk from detailed risk assessment (45 risk factors involved) (Figure 2) • Does this summary scheme work (Table 5 & Figure 3)?

Does summary scheme work?

Figure 3. Does summary scheme work?

Discussions • Incorporating weighting into PRA • Historical data of pest introductions and invasions • Previous PRA cases • Expert opinion • Previous data do not necessarily apply to new situations; however, these can provide at least a starting point for new pests.

Conclusions Weightings can be derived for individual risk elements/factors by applying statistical techniques or assigning by expert judgement • identify the more important risk elements • filter out the factors that are low contributors to the overall assessment while retaining the important ones • Without compromising the rigor we striving for in risk assessment • It is difficult to develop a generic weighting pattern for different pest categories • A quick summary scheme can be developed, it gives a quick and precautious idea of risk rating

Acknowledgement Ministry of Agriculture and Forestry Higher Education Funding Council for England through University of Greenwich

Lihong Zhu 1 , John Holt 2 & Rob Black 3