Rapid Risk Assessment (and/or Rapid Risk Analysis) Are they possible?

1. Rapid Risk Assessment(and/or Rapid Risk Analysis)Are they possible? Villie Flari, Food and Environment Research Agency, UK Villie.flari@fera.gsi.gov.uk JIFSAN / Fera annual meeting, June 2011

2. Trigger for this talk… • … part of my work when at CFSAN, FDA (2007 – 2009) • “Rapid Risk Assessment Framework” project: a tool to apply with CFSAN emergency response procedures • Ambitious and quite challenging project: aimed to identify whether there is room for improving the existent strategies, employed at the time by CFSAN/FDA during emergency events, and if yes to suggest and justify alternative approaches • Review FDA/CFSAN processes to assess public health threats during emergency events. • Identify areas of opportunities for implementing risk assessment thought approaches with the view to improving the current FDA/CFSAN process. • Prepare a framework document to describe the recommended process. • Develop templates for assessing information, characterizing unknown factors and for communicating assessment results effectively to risk managers and/or risk communicators. Project orphan..

3. Are any changes to current schemes and approaches truly needed? • What is the outcome we want to estimate – which is our aim? • Preventive approaches and emergency responses for ensuring food safety and protecting human health • Overall, not that bad • Apart form regulatory processes a number of initiatives and support systems nationally and internationally developed over the last 30 years - currently in place (e.g. RASFF) 2008 2008-9 2009 2010 2011

4. Food safety challenges - constantly change… Dynamic Emerging sciences & technologies Source: Food Safety Risk Analysis: a guide for national food safety authorities, (FAO Food and Nutrition Paper 87, 2006)

5. Emerging risks – emerging sciences / technologies Genetically Modified Organisms ? Nano-sized materials ? ? Unknown, thus far, emerging risk(s) Synthetic Biology ? Tactical and strategic problems

6. Designing rapid risk assessments – are they possible? • What is the outcome we want to estimate – which is our aim? • What do we mean precisely when we term a document as a “risk assessment” or a “rapid risk assessment”? • How do we define the term “risk”? • Safety assessment? • Risk assessment? • A collection of (background + epi) information?

7. Possible scenarios

8. Looking into historical events & following events “in vivo” Historical events • “Melamine in pet food” - March to May 2007 Following events “in vivo” • “Salmonella saintpaul in fresh produce” – May to August 2008 • “Salmonella typhimurium in peanut butter” - November 2008 to January 2009

9. What did we learn? Incomplete knowledge Limited time Many dimensions

10. Lines of investigation of emergency events: relevant information needs RefDoc 11 Recall International Domestic TRACEFORWARD General info Specific to product FOOD SUPPLY CHAIN Production Processing Plants Market share (specific to firm (s) Packing Firm’s product Trace back SOP Historical Data (Firm) Distributors Kill steps FIRMS’ PRACTICES Compliance Background / rationale Distributors RISK ASSESSMENT Shelf life of product Best before date LOT Trace back 2nd level Trace back 3rd level Most illnesses > 10/01 DISTRIBUTION TRACEFORWARD Trace back 1st level Questionnaires INVESTIGATION FINDINGS Illness onset range OUTBREAK INVESTIGATION PRODUCTS MANUFACTURED Number of Ill Deaths related to outbreak Source Vehicle (s) FACTS Databases Inspection reports EPIDEMIOLOGY Illnesses Outside USA Complaints INVESTIGATIVE LABORATORY FINDINGS Case control Study results BACKGROUND INFORMATION CLINICAL DATA FDA data Epi curve Raw data International samples FIRM data STATE Data

11. Problem i.e. Risk Management question (s) See RefDoc1 and 2 for extended information Assess problemagainst criteria for Class I Recall1 See RefDoc 2 for extended information Yes Risk Management decision Problem classified As Class I Recall? No Define RA endpoint (s) See RefDoc 8 for extended information Yes Continue investigation Is Risk Assessment required by RM? No Introduce RA as line of investigation See RefDoc 8 for extended information Define all lines of investigation See RefDoc11 for extended information Repeat for each line of investigation Monitor & Organize Data See RefDoc12 for extended information Evaluate data Accept, Reject, Withhold data See RefDoc 14 for extended information Uncertainty analysis Decision making process See RefDoc 3 & 16 for extended information Form hypothesis See RefDoc 14 for extended information Test hypothesis See RefDoc 14 for extended information Consider all lines of investigation Data considered enough for accepting hypothesis? See RefDoc 14 for extended information Do not accept hypothesis No Risk Communication Yes Hypothesis accepted Need to redefine problem? Risk Management options Yes No

12. Risk Assessment’s value: • Bringing all the information together – connecting all these lines of investigation in order to reveal patterns, hypotheses. • Gather information, direct research for the future. RefDoc 11 TRACEFORWARD Risk Assessment would allow one to be able to answer questions regarding “how risky it is to consume peanut butter products”, e.g. “if I eat 200 crackers from 200 different packs which was the probability to have been contaminated by Salmonella typhimurium?” FOOD SUPPLY CHAIN Risk Assessment would allow to address questions regarding the effect of risk factors if certain mitigation methods would be followed; for example “If the temperature during a kill step is always above the threshold of 350 F what would be the decrease in probable pathogen concentration in a cookie?” FIRMS’ PRACTICES RISK ASSESSMENT Background / rationale Risk Assessment approach can be employed to ask information about the denominator of epidemiological and investigative laboratory results: e.g. “During the “Salmonella typhimurium in the peanut butter” case how many samples have they been put under test?” Usually, a lot of attention is paid on the positive microbiological samples, but not on the total of samples tested. The denominator could give us information on the actual risk posed by the particular microbiological hazard. Usually a presence/absence of the microbiological hazard is given. Colony Forming Units (CFUs) information is not given. As a result, Dose Response information is missing. Requesting CFUs could help risk assessments in the future as well. INVESTIGATION FINDINGS INVESTIGATIVE LABORATORY FINDINGS EPIDEMIOLOGY

13. Is this what we need? • Partially yes, but ….

14. Risk assessments / analyses in emergency responses: why, how? The STEC outbreak reported from Germany is noteworthy considering its magnitude: 276 HUS cases and two deaths due to HUS reported in just a few weeks. The majority of cases reported are adult women. Usually, about 15% of children with STEC infection present with HUS, with this proportion being much lower among adults. This means that several hundred STEC cases with diarrhoea are likely to be occurring in the current outbreak. The isolated outbreak strain STEC O104:H4 is very rare. Prior to the current outbreak, only one case has been documented in literature, and this case was a woman in Korea in 2005. The case-control study in Hamburg revealed that raw tomatoes, fresh cucumber and leafy salad are the likelyvehicles of infection. Samples of fresh cucumbers taken in Hamburg tested positive for STEC, however, the exact time and place of contamination remains unclear. It is unclear whether the results from Hamburg can be extrapolated to the whole of Germany. Furthermore, it cannot be excluded that an alternative food item is the vehicle of infection. The definite source of the infection remains to be confirmed. There is currently no indication that either the source of the outbreak or the vehicle of infection has been distributed outside of Germany. Likely 47% sure? 71% sure? 83% sure?

15. The question then becomes… • Whether approaches and thought processes we develop for and apply in innovative Risk Assessments (or Analyses…), can help us overcome difficulties such as those we encountered recently in the food safety field. • Chemical: Can we identify the true hazard quickly? • Microbiological: Can we identify the true vehicle(s)/source(s) of an outbreak quickly? • Can we foresee possible future challenges relevant to new sciences and technologies that are being developed? • Can we produce reproducible, transparent, fully defensible assessments / analyses that will withstand media pressure, public criticism?

16. What did we learn? The work on this project highlighted the significance a number of aspects that appear to be inherent in an emergency investigations: • Unknown factors, and in particular the value of embracing uncertainty- embracing uncertainties by recognizing them, classifying them, characterizing them can only be beneficial for all involved in the investigation, the scientists, the decision makers and the public. • The art of eliciting expert judgment: subjective information plays a major part in the science on which rapid decisions are made. How do we ensure that we obtain the most informative points of view, and how do we ensure that we capture experts’ uncertainty? How do we combine different opinions, if this is deemed necessary? • Following a problem – solving approach: this type of work may be quite different from simply following a process. Asking “what is the problem, and how the problem could be defined?” is not merely a philosophical question; instead, it is the most important question to be settled in the start of an investigation. Incomplete knowledge Limited time

17. Uncertainty in risk assessments, risk analyses in emergency events: why, how? • What is the desired outcome – which is our aim? • Rapid Risk Assessment • Risk is the uncertainty of the outcome • Outcome(s)  Risk Assessment endpoint(s) • Which are the chances of my child dying if s/he consumes x amount of contaminated food? • Which is the proportion of Denmark population who may suffer from severe adverse affects (i.e. >10 days of hospitalisation) if they consume x amount of contaminated food? • Uncertainty of outcome(s) • Probability distributions - Quantitative (Hard data and/or expert judgment) • + Qualitative (acquired via expert judgment) information ANDY HART

18. What did we learn? The work on this project highlighted the significance a number of aspects that appear to be inherent in an emergency investigations: • Unknown factors, and in particular the value of embracing uncertainty- embracing uncertainties by recognizing them, classifying them, characterizing them can only be beneficial for all involved in the investigation, the scientists, the decision makers and the public. • The art of eliciting expert judgment: subjective information plays a major part in the science on which rapid decisions are made. How do we ensure that we obtain the most informative points of view, and how do we ensure that we capture experts’ uncertainty? How do we combine different opinions, if this is deemed necessary? • Following a problem – solving approach: this type of work may be quite different from simply following a process. Asking “what is the problem, and how the problem could be defined?” is not merely a philosophical question; instead, it is the most important question to be settled in the start of an investigation. Incomplete knowledge Limited time

19. Expert judgment in risk assessments in emergency responses: why, how? • Who is to be involved? • Best expertise possible • Lists of experts available • Calibration of experts only in retrospect (time-consuming) • Fit for purpose method • Qualitative information? • Quantitative information? • Both? • Uncertainty of experts to be captured • Trained risk analysts • Table top exercises ROGER COOKE

20. What did we learn? The work on this project highlighted the significance a number of aspects that appear to be inherent in an emergency investigations: • Unknown factors, and in particular the value of embracing uncertainty- embracing uncertainties by recognizing them, classifying them, characterizing them can only be beneficial for all involved in the investigation, the scientists, the decision makers and the public. • The art of eliciting expert judgment: subjective information plays a major part in the science on which rapid decisions are made. How do we ensure that we obtain the most informative points of view, and how do we ensure that we capture experts’ uncertainty? How do we combine different opinions, if this is deemed necessary? • Following a problem – solving approach: this type of work may be quite different from simply following a process. Asking “what is the problem, and how the problem could be defined?” is not merely a philosophical question; instead, it is the most important question to be settled in the start of an investigation. Incomplete knowledge Limited time

21. Problem i.e. Risk Management question (s) See RefDoc1 and 2 for extended information Assess problemagainst criteria for Class I Recall1 See RefDoc 2 for extended information Yes Risk Management decision Problem classified As Class I Recall? No Define RA endpoint (s) See RefDoc 8 for extended information Yes Continue investigation Is Risk Assessment required by RM? No Introduce RA as line of investigation See RefDoc 8 for extended information Define all lines of investigation See RefDoc11 for extended information Repeat for each line of investigation Monitor & Organize Data See RefDoc12 for extended information Evaluate data Accept, Reject, Withhold data See RefDoc 14 for extended information Uncertainty analysis Decision making process See RefDoc 3 & 16 for extended information Form hypothesis See RefDoc 14 for extended information Test hypothesis See RefDoc 14 for extended information Consider all lines of investigation Data considered enough for accepting hypothesis? See RefDoc 14 for extended information Do not accept hypothesis No Risk Communication Yes Hypothesis accepted Need to redefine problem? Risk Management options Yes No

22. Defining the problem - question RefDoc 1 2: Phrase question (s) of investigation 1: Report trigger (s) Investigators, Analysts Decision Makers, Risk Managers Template flow chart Questions considered unambiguous by investigators, analysts? No Yes Background / rationale Assessment of problem, i.e. risk management question (s), begins

23. It is best to define theproblem (i.e. risk management question (s)) precisely to ensure that answers provided fit the purpose. • “To identify the food vehicle responsible for causing an outbreak of Salmonella typhimurium infections with the following PulseNet strains1:JPXX01.1818, JPXX01.1825, JPXX01.0459”. • Problem (i.e. risk management question (s)) is dynamic; thus, it may need to be redefined in subsequent phases of the investigation. • “To a) identify distribution patterns of Peanut Corporation of America products, b) elicit expert opinion on manufacturing practices and c) review epidemiological data associated with Salmonella typhimurium, particularly strains JPXX01.1818, JPXX01.1825, JPXX01.0459 in collaboration with CDC with the view to prioritizing products according to the risk they may impose to consumers”. • Advantages: • Diminishing ambiguity in provided answers. • Allowing investigators, analysts who participate in the emergency case to get familiar with important information of the investigation irrespective of their role and/or expertise and have the opportunity to look at the big picture of the investigation. RefDoc 1 Template flow chart Background / rationale

24. What did we learn? • Following a problem – solving approach: this type of work may be quite different from simply following a process. Asking “what is the problem, and how the problem could be defined?” is not merely a philosophical question; instead, it is the most important question to be settled in the start of an investigation • “Identifying, defining and then respecting the natural shape of the problem does not come naturally to agencies, since these are based on an organizational structure that was formed years ago and most of the approaches they follow are process-based” – Sparrow, 2008 • Perhaps, such an approach becomes even more challenged during an investigation of an emergency event as the organization is under extra pressure to deliver results and facilitate relevant decision making. People Perceptions Biases

25. Following a problem-solving approach “Solving a problem simply means representing it so as to make the solution transparent” Herbert A. Simon: The sciences of the artificial, 1996

26. Building a conceptual model of the problem: example based on “Melamine in pet food” emergency event RefDoc 15 Chemistry findings: Melamine present in contaminated wheat gluten 03/25/2007 Hypothesis: Melamine responsible for illnesses and deaths Melamine Toxicology findings: Melamine not toxic for mammals, in particular for the recorded adverse effects 03/27/2007 Example Pathology findings: Acute/subacute kidney failure in cats and dogs 03/15/2007 Chemistry findings: Further compounds identified in contaminated wheat gluten ureidomelamine, urea, ammeline, ammelide, cyanuric acid 03/30/2007 Ammeline Hypothesis: Other compound(s) responsible for illnesses and deaths Urea Ammelide Hypothesis: More than one compounds responsible in association with other agents for illnesses and deaths Cyanuric Acid Toxicology findings: There may be some additional links between melamine and uric acid 04/03/2007 Ureidomelamine Toxicology findings: No compound, of those identified in contaminated wheat gluten is considered as toxic to mammals 04/04/2007 Pathology findings: Oral toxicology experiments indicate formation of crystals due to combination of chemicals, e.g. melamine+ammeline+ammelide+ cyanuric acid 04/24/2007 Melamine Ammeline Ammelide Cyanuric Acid

27. Decision makingmodels RefDoc 16 ECRT decides that: Hypothesis Is accepted ECRT COLLABORATES WITH EXPERTS TO DEFINE OPTIONS OF INTERVENTION Intervention A Intervention B Intervention C … ECRT COLLABORATES WITH EXPERTS AND RISK MANAGERS TO DEFINE STAKEHOLDERS, CRITERIA Stakeholders – Criteria - Units of measure for criteria Default method to handle information ECRT COLLABORATES WITH EXPERTS TO DISPLAY SENSITIVITY ANALYSIS, GATHER INFORMATION What if scenario What if scenario What if scenario What if scenario What if scenario What if scenario What if scenario What if scenario What if scenario What if scenario Aggregation of criteria in a single function in order to be able to “compare” and prioritize between alternative scenarios. This could be performed via a number of Decision Making tools. ECRT COLLABORATES WITH EXPERTS, ANALYSTS, AND DECISION MAKERS TO APPLY DECISION MAKING TOOLS

28. Challenges in ameliorating risk assessments / risk analyses: Strategic versus tactical • Similarities • Encompass uncertainties • Define problem in question precisely • Employ the best experts • Provide support tools for transparent, reproducible, fully defensible assessments • Further challenges when dealing with emergency events • All of the above become even more crucial when dealing with emergency investigations!

29. Thank you! • CFSAN / FDA colleagues • Sherri Dennis • Elisa Elliott • Jack Guzewich • David Hattan • Faye Fenstein • Karl Klonz • Sherri McGarry QUESTIONS ? Villie.flari@fera.gsi.gov.uk JIFSAN / Fera annual meeting, June 2011