rethinking uncertainty challenges for science and society

1. rethinking uncertaintychallenges for science and society Jeroen van der Sluijs Copernicus Institute for Sustainable Development and Innovation Utrecht University

2. Complex environmental risks Typical characteristics (Funtowicz & Ravetz): • Decisions will need to be made before conclusive scientific evidence is available; • Decision stakes are high: potential error costs of wrong decisions can be huge • Values are in dispute • Knowledge base is mixture of knowledge and ignorance: • large (partly irreducible) uncertainties, knowledge gaps, and imperfect understanding; • Assessment dominated by models, scenarios, and assumptions • Many (hidden) value loadings in problem frames, indicators, assumptions Coping with uncertainty is essential

3. Problematic definitions of uncertainty Example 1: Walker et al. 2003 “We adopt a general definition of uncertainty as being any departure from the unachievable ideal of complete determinism” Might make more sense to talk about unreality or uncomplexity as being any departure from or reduction of the inherent complexity of systems outside the controlled environment of the laboratory.

4. Definition from HarmoniCa uncertainty guidance document • A person is uncertain if s/he lacks confidence about the specific outcomes of an event or action. Reasons for this lack of confidence might include a judgement of the information as incomplete, blurred, inaccurate or potentially false or might reflect intrinsic limits to the deterministic predictability of complex systems or of stochastic processes.Similarly, a person is certain if s/he is confident about the outcome of an event. It is possible that a person feels certain but has misjudged the situation (i.e. s/he is wrong).

5. The definition above defines uncertainty as a property (state of confidence) of the decision. Alternatively uncertainty can be defined asa property (state of perfection) of the total body of knowledgeor information that is available at the moment of judgement. Uncertainty is then seen as an expression of the various forms of imperfection of the available information and depends on the state-of-the-art of scientific knowledge on the problem at the moment that the decision needs to be made (assuming that the decision maker has access to the state-of-the-art knowledge).

6. Challenges • Increase society’s capacity to manage and surmount uncertainties surrounding knowledge production and use in designing and implementing precautionary (or should I say: responsible) policies and sustainable development • New epistemology that does not see uncertainty as deviation from deterministic ideal, nor as imperfect knowledge, nor as low quality • Need for a new (multidimensional) definition of uncertainty (maybe even a new word)

7. Insights on uncertainty • Omitting uncertainty management can lead to scandals, crisis and loss of trust in science and institutions • More research tends not to reduce uncertainty • Usually reveals unforeseen complexities • Meets irreducible uncertainty (intrinsic or practically) • High quality  low uncertainty • Quality relates to fitness for function (robustness, PP) • In many complex problems unquantifiable uncertainties dominate the quantifiable uncertainty • Shift in focus needed from reducing uncertainty towards systematic ways to explicitly cope with uncertainty and quality -> knowledge quality assessment

8. Uncertainty as a “monster” • A monster is a phenomenonthat at the same moment fits into two categories that were considered to be mutually excluding (Smits, 2002; Douglas 1966)

9. Cultural categories that we thought to be mutually exclusive and that now tend to get increasingly mixed up: • knowledge – ignorance • objective – subjective • facts – values • prediction – speculation • science – policy

10. Responses to monsters Different degrees of tolerance towards the abnormal: • monster-exorcism (expulsion) • monster-adaptation (transformation) • monster-embracement (acceptance) • monster-assimilation (rethinking)

11. Footnote:compare to Lakatos (1976) preserving mathematical models against apparent refutations • Surrender (throw the model away and start all again), • Monster barring, • Monster adjustment • Lemma incorporation.

12. monster-exorcism • Uncertainty causes discomfort • Reduce uncertainties! • Strong believe in “objective science”: “the puzzle can be solved” Example: • “We are confident that the uncertainties can be reduced by further research” (IPCC 1990)

13. But…. • For each head science chops off of the uncertainty monster, several new monster heads tend to pop up (unforeseen complexities) • 1994 IGBP dropped objective to reduce uncertainty: ”full predictability of the earth system is almost certainly unattainable”

14. Former chairman IPCC on objective to reduce uncertainties: • "We cannot be certain that this can be achieved easily and we do know it will take time. Since a fundamentally chaotic climate system is predictable only to a certain degree, our research achievements will always remain uncertain. Exploring the significance and characteristics of this uncertainty is a fundamental challenge to the scientific community." (Bolin, 1994)

15. Monster adaptation • Fit the uncertainty monster back in the categories: purification • Quantify uncertainty, subjective probability & Bayesian • Tendency to build system models based on “objective science” and externalise the subjective parts and uncertainties into scenario’sand storylines • Boundary work

16. IPCC 10 years after “we are confident that the uncertainties can be reduced…”

17. Monster adaptation meets its limits • Different models fed with the same scenarios produce very different results • “Integrated Assessment Modeling of Global Climate Change: Transparent Rational Tool for Policy Making or Opaque Screen Hiding Value-laden Assumptions?”(Steve Schneider)

18. Monster Embracement • Uncertainty is welcomed: an appreciated property of life fascination about the unfathomable complexity of our living planet Gaia room for spirituality and wonder as counterweight to the engineering worldview of “managing the biosphere” • Plea for a humble science • Holism; Inclusive Science --------------------- Or: • Uncertainty is welcomed because it fits well in other political agenda’s • (strategic) Denial of realness of environmental risks by emphasizing all those uncertainties • Manufacturing uncertainty

19. Monster Assimilation • Rethink the categories by which the knowledge base is judged • Create a place for monsters in the science policy interface • Post Normal Science; Reflexive science; Complex systems research

20. Uncertainty has multiple dimensions • Technical (inexactness) • Methodological (unreliability) • Societal (limited social robustness) • Epistemological (ignorance)

21. Inexactness Intrinsic uncertainty: • Variability / heterogeneity Technical limitations: • Resolution error • Aggregation error • Unclear definitions

22. Unreliability Methodological limitations Limited internal strength in: • Use of proxies • Empirical basis • Methodological rigour • Validation Bias in knowledge production • Motivational bias (interests, incentives) • Disciplinary bias • Cultural bias • Choice of (modelling) approach • Subjective judgement Future scope

23. Limited social robustness Limited external strength in: • Bias / Value ladenness • Insufficient exploration of rival problem framings • Management of dissent • Extended peer acceptance / stakeholder involvement • Transparency • Access & availability • Intelligibility Strategic/selective knowledge use

24. Ignorance Epistemological limitations • Limited theoretical understanding • System indeterminacy • Open-endedness • Chaotic behavior • Intrinsic unknowability • Active ignorance • Model fixes for reasons understood • Limited domains of applicability of functional relations • Numerical error • Surprise A • Passive ignorance • Bugs (software error, hardware error, typos) • Model fixes for reasons not understood • Surprise B

26. RIVM/MNP Uncertainty Guidance Systematic reflection on uncertainty and quality in: • Problem framing • Involvement of stakeholders • Selection of indicators • Appraisal of knowledge base • Mapping and assessment of relevant uncertainties • Reporting of uncertainty information

27. Mini-Checklist QuickscanQuestionnaire Further GuidanceAdviceHints & Implications Quickscan Hints & Actions List Tool Catalogue for Uncertainty Assessment Detailed Guidance RIVM-MNPUncertaintyGuidance Reminder listInvokes ReflectionPortal to QS Advice on Quantitative + Qualitative tools for UA Downloads: www.nusap.net

29. “Wisdom is to know, that you do not know” (Socrates)