Use of Microbial Risk Assessment in Decision-Making

1. Note the 2 ‘l’s ! Use of Microbial Risk Assessment in Decision-Making Slide show on: www.risk-modelling.com/firstmicrobial.htm David Vose Consultancy 24400 Les Lèches Dordogne France www.risk-modelling.com Email David Vose's secretary David Vose

2. Introduction • Applying CODEX guidelines in reality • Difficulties • Other ways of thinking • Experience with microbial modelling • Some survey results • The Dutch experience • Some US experience • Modelling challenges • Comparison of some complete models • Reviewing a model in context David Vose Consultancy Ltd www.risk-modelling.com

3. Codex Alimentarius CommissionFAO/WHO (1995) Microbial risk assessment is a scientifically-based process consisting of four steps: • Hazard Identification The identification of known or potential health affects associated with a particular agent; • Exposure Assessment The qualitative and/or quantitative evaluation of the degree of intake likely to occur; • Hazard Characterization The quantitative and/or qualitative evaluation of the nature of the adverse effects associated with biological, chemical and physical agents that may be present in food… For biological agents… a dose-response assessment should be performed if the data is available; • Risk Characterization Integration of Hazard Identification, Hazard Characterization and Exposure Assessment into an estimation of the adverse effects likely to occur in a given population, including attendant uncertainties. David Vose Consultancy Ltd www.risk-modelling.com

4. OIE experience • OIE produced guidelines for animal import risk assessments (for the management of disease spread) • Now in its second edition • Guidelines were offered as a way to help member (including developing) countries understand how to perform a r.a. • First Ed. guidelines were used too literally, both by analysts and lawyers, and found to be often impractical or irrelevant to the risk question • Lesson: keep guidelines non-specific, encourage understanding rather than prescribing a formulaic approach • Popular interpretation of CODEX guidelines suffer similarly David Vose Consultancy Ltd www.risk-modelling.com

5. Society for Risk Analysis Principles for Risk Analysis (1)Risk analysis uses observations about what we know to make predictions about what we don’t know. Risk analysis is a fundamentally science-based process that strives to reflect the realities of Nature in order to provide useful information for decisions about managing risks. Risk analysis seeks to inform, not to dictate, the complex and difficult choices among possible measures to mitigate risks... David Vose Consultancy Ltd www.risk-modelling.com

6. Society for Risk Analysis Principles for Risk Analysis (2) Risk analysis seeks to integrate knowledge about the fundamental physical, biological, social, cultural, and economic processes that determine human, environmental, and technological responses to a diverse set of circumstances. Because decisions about risks are usually needed when knowledge is incomplete, risk analysts rely on informed judgment and on models reflecting plausible interpretations of the realities of Nature. We do this with a commitment to assess and disclose the basis of our judgments and the uncertainties in our knowledge. David Vose Consultancy Ltd www.risk-modelling.com

7. Current modelling • Microbial QRA is a developing science • We’re making a lot of progress, but it is still in infancy • Mostly producing ‘farm-to-fork’ • Models the whole system but very poorly • Not designed to model any decision question well • Often relies on poor data, surrogates, and guesses • Almost never is a decision question posed beforehand • Assessors have probably over-sold QRA’s usefulness • Managers have expected too much David Vose Consultancy Ltd www.risk-modelling.com

8. F2F Achilles’ Heels • Very little data available, system being modelled is hugely complex! • Uncertainty, variability, inter-individual variablility • Take too long to complete, too easy to make mistakes • F2F considers only pathogen on the food source • E.g. not E.coli produced during life of animal, appearing in water, vegetables, farmers’ exposure • Predictive microbiology still unreliable • Broth data doesn’t translate well to food (usually overestimate, but some data – Tamplin, USDA – shows lag period can be shorter, e.g. E.coli in ground beef, Listeria in processed hams) • Models often not based on physical/biological ideas, so we don’t learn • Attenuation may not be death, and ignores reactivation of bacteria • D-R models inadequate • Don’t describe variability observed • P(ill|dose, infected) = P(ill|infected)? • Feeding trial data don’t match epi data – can hugely underestimate the risk • Little cost-benefit analysis effort made • Including actions affecting several risk issues • Requires enormous resources – impractical for many countries David Vose Consultancy Ltd www.risk-modelling.com

9. Dutch observations on past QRAHavelaar, Jansen (2002) The lessons learnt from risk analysis experiences: • Risk management has not always been an integral part of risk analysis so far; • Risk managers should be trained to understand risk assessment, and risk assessors should be trained to explain their work; • Available data are often of limited use for risk assessment and communication of data needs between risk assessors, food scientists and risk managers is a critical issue; • The risk manager questions usually require rapid results, whereas (farm-to-fork) risk assessment projects require several years to complete. Solving this conflict requires open communication; • Uncertainty is often large. David Vose Consultancy Ltd www.risk-modelling.com

10. Our surveyInternet based, voluntary participation, 39 valid responses David Vose Consultancy Ltd www.risk-modelling.com

11. David Vose Consultancy Ltd www.risk-modelling.com

12. David Vose Consultancy Ltd www.risk-modelling.com

13. David Vose Consultancy Ltd www.risk-modelling.com

14. Completion times of some farm-to-fork QRAs Final report Final report Draft report Being revised Draft report Final report David Vose Consultancy Ltd www.risk-modelling.com

15. Salmonella dose-responseEpi and feeding trial comparison Review by Amir Fazil in FAO/WHO (2001) D-R mathematical models review by Haas (2002) David Vose Consultancy Ltd www.risk-modelling.com

16. USDA-FSIS-FDA Salmonella Enteritidis “Although the goal was to make the model comprehensive, it has some important limitations. It is a static model and does not incorporate possible changes in SE over time as either host, environment or agent factor change. For many variables, data were limited or nonexistent. Some obvious sources of contamination, such as food handlers, restaurant environment, or other possible sites of contamination on or in the egg (such as the yolk), were not included. And, as complex as the model is, it still represents a simplistic view of the entire farm-to-table continuum. Finally, the model does not yet separate our uncertainty from the inherent variability of the system. Much more work is needed to address this, and all other, limitations.” David Vose Consultancy Ltd www.risk-modelling.com

17. USDA-FSIS-FDA Salmonella Enteritidis • Original model impetus was to evaluate effect of refrigeration temp from laying to retail on food safety • Empirically must have little affect since it only deals with a few days in the life of an egg • No cost-benefit attached • Now being redone to focus on level of performance required for shell, and liquid egg pasteurisation • i.e. much more decision focused David Vose Consultancy Ltd www.risk-modelling.com

18. FDA Listeria risk assessment • No specific decision questions attached • Attempted to look at relative importance of a large list of Listeria-carrying foods • Given the data available, perhaps the only method possible to estimate which food types contribute the greatest risk • So a good QRA application David Vose Consultancy Ltd www.risk-modelling.com

19. Remedies: focusing on decisions • Consider what is known about the risk problem, and data available immediately or within acceptable time frame • Use epi data as much as possible • Collect more epi data (e.g. Japan, Denmark) • Consider what analysis could be done with this knowledge • i.e. a risk-based reasoned argument for evaluating particular actions • Estimate the possible magnitude of benefit for a risk action • Note that it may not be possible to evaluate all actions • Perform a cost-benefit analysis on these actions • Perform a Value of Information analysis • Determines whether it is worth collecting more data before making a decision • Consider strategy to validate whether predicted improvement occurs • Train data producers to supply maximally useful data • E.g. microbiologists taken more than one cfu from a plate • More inter-agency unity • E.g. Farm (APHIS) Þ Slaughter (FSIS) Þ Retail (FDA) David Vose Consultancy Ltd www.risk-modelling.com

20. Make it as simple as possible: example • Risk: Human illness from SE in eggs • A shell-egg selection system proposed that will reduce by 30% the number of contaminated eggs going to market • Currently, 30,000 people a year suffer from SE from eggs • What will be the reduction in cases if the new system is implemented? Reduction in cases = 30%*30,000 = 9,000 people/year No need for models of D-R, bacterial growth, handling, etc. Vulnerability to assumptions smaller than from using F2F model David Vose Consultancy Ltd www.risk-modelling.com

21. An example of the way forwardHavelaar, Jansen 2002 Campylobacter Risk Management and Assessment Dutch proposal The main objectives of the project are to advise on the effectiveness and efficiency of measures aimed at reducing campylobacteriosis in the Dutch population. The two key questions are: • What are the most important routes (quantifiable?)? • Which (sets of) measures can be taken to reduce the exposure to Campylobacter, what is their expected efficiency and societal support? David Vose Consultancy Ltd www.risk-modelling.com

22. The way forward – cont. The target of the assessment is not limited to estimating the possible reduction in disease incidence but to evaluate both costs and benefits of possible interventions and to access their acceptance by stakeholders. Interventions with low social support will require more effort to uphold, which increases their costs and reduces their efficacy. David Vose Consultancy Ltd www.risk-modelling.com

23. Danish Vet Service Salmonella QRA“A Bayesian Approach to Quantify the Contribution of Animal-food Sources to Human Salmonellosis” - Hald, Vose, Koupeev (2002) Estimated number of cases of human salmonellosis in Denmark in 1999 according to source Model ranks food sources by risk. Easily updateable with each year’s data. Bayesian update improves estimate and checks validity of assumptions. David Vose Consultancy Ltd www.risk-modelling.com

24. Fluoroquinolone-resistant Campylobacter risk assessment Model: Contaminated carcasses after slaughter plant * probability = affected people David Vose Consultancy Ltd www.risk-modelling.com

25. Broiler house Transport Slaughterhouse model Slaughter house Hanging Example of Farm-to-Fork model Campylobacter in poultry Draft report 2001 Institute of Food Safety and Toxicology Division of Microbiological Safety Danish Veterinay and Food Administration Scalding Defeathering Evisceration Washing Chilling Export Chicken parts Whole chickens Chilled Frozen Further Import Processing Retail Catering Cross contamination Consumer model Heat treatment Consumer Cross contamination Behaves the same way as CVM model if prevalence is reduced Heat treatment Dose response Risk estimation David Vose Consultancy Ltd www.risk-modelling.com

26. Reviewing a risk assessment • Risk assessment should be decision focused • It is not appropriate to review a risk assessment independently from the question(s) the assessment is addressing • Eg because a point is moot if the decision is insensitive to the argument • It uses science but is not itself scientific research • So we have to go with the best we’ve got David Vose Consultancy Ltd www.risk-modelling.com

27. Finally - risk assessors should gain hands-on experience to ensure their models reflect the real world David Vose Consultancy Ltd www.risk-modelling.com

28. References • Haas, C.N., (2002), Conditional Dose-Response Relationships for Micro-organisms: Development and Applications. Risk Analysis 22 (3): 455-464. • Havelaar, H. and J. Jansen, (2002), Practical Experience in the Netherlands with quantitative microbiological risk assessment and its use in food safety policy. Draft paper, RIVM, Bilthoven, The Netherlands. • Hope, B.K., et al. , (2002), An overview of the Salmonella Enteritidis Risk Assessment for Shell Eggs and Egg Products. Risk Analysis 22 (3): 455-464. • Joint FAO/WHO Expert Consultation on the Application of Risk Analysis to Food Standards Issues (Joint FAO/WHO, 1995). • Joint FAO/WHO Expert Consultation on Risk Assessment of Microbiological Hazards in Foods: Risk characterization of Salmonella spp. in eggs and broiler chickens and Listeria monocytogenes in ready-to-eat foods. (2001), FAO headquarters, Rome. • Teunis, P.F.M. and A.H.Havelaar, (2001), The Beta-Poisson Dose-Response Model Is Not a Single-Hit Model. Risk Analysis 20 (4): 513-520. David Vose Consultancy Ltd www.risk-modelling.com