A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making

1. A Systems Approach to Establishing Scientific Integrity in Evidence Based Policy Making Prof. Dr. Wijnand J. Swart Centre for Plant Health Management (CePHMa) Faculty of Natural and Agricultural Sciences University of the Free State Bloemfontein South Africa swartwj.sci@ufs.ac.za

2. EVIDENCE in Policy Context • Sound, credible and robust evidence, whether : • quantitative • qualitative • statistical • economic • attitudinal / behavioural • anecdotal • social • opinion based • or review based ..... ......is an essential and necessary part of the enabling environment for formulating policies that are coherent and effective in terms of their outcomes.

3. There are policies that: …and use it WELL… Use GOOD data & information…. Use POOR data & information…. …and use it POORLY… Source: DEFRA, UK The Policy Making Process • Techniques, analyses and judgements used to evaluate and formulate data and information into knowledge / evidence for making effective policies are critical.

4. What is ‘Sound Evidence?’ • Concept of “sound and credible evidence” is very complex. • Dependant on inter alia: • types • sources • generating techniques • context • ‘understanding’ ...... of data / information / knowledge

5. ‘Sound Evidence’: Context & Understanding TRUTH? • Dataare facts (e.g. numbers, names, symbols) and have little value in themselves. • Information relates to description, definition, or perspective (what, who, when, where). • Knowledge comprises strategy, practice, method, or approach (how). • Wisdom embodies principle, insight, morals, or archetypes (why). • Absolute Truth ? WISDOM understanding principles KNOWLEDGE CONTEXT INDEPENDENCE understanding patterns INFORMATION understanding relations DATA UNDERSTANDING

6. Knowledge vs Science • Science is organized knowledge. Herbert Spencer • Science: a knowledge of principles and causes.(Webster's Revised Unabridged Dictionary) • Science is a way of thinking much more than it is a body of facts. Carl Sagan • “Good Science” could be defined as those practices which contribute most to advances in understanding. Sindermann: The Joy of Science 1985. • Science is to see what everyone else has seen but think what no one else has thought. Albert Szent-Gyorgyi

7. Science and Policy Making • Creative and innovative contributions of scientists and policy makers and the trust engendered in them by the public, to whom they are accountable is of paramount importance. • Follows therefore that fostering an environment that promotes research or scientific integrity is an integral part of that accountability and the pursuit of new knowledge.

8. ‘Scientific Integrity’ • “Integrity” by definition: • “honesty” • “a state of being entire or whole” • Perspectives of ‘scientific integrity’ : • Ethical issues relating to misconduct (fraud) or manipulation, suppression, or distortion of facts. 2. Striving towards “wholeness” or excellence in the search for knowledge.

9. Research Integrity • Essential for maintaining scientific excellence and for keeping the public’s trust. • Research integrity characterizes : • Institutional integrity: • creating an environment that promotes responsible conduct and high levels of integrity • embracing standards of excellence, trustworthiness, and lawfulness • Individual integrity: Scientist’s commitment to intellectual honesty and personal responsibility and is an aspect of moral character and experience. A good scientist must: • communicate well • obtain research grants • excel in teaching and mentoring • engage in ethical decision making • use knowledge ‘wisely’ to plan and execute research.

10. Institutional Integrity:Politics vs Science • Politicization of science as old as science itself e.g Galileo's theory that the Earth revolves around the sun perceived as a challenge to the authority of the Catholic church. • Political interference threatens the integrity of government science and policy making all over the world. • Manipulation, suppression, and distortion of government science misinforms public and leads to poor policy decisions. • Especially rife in developing countries, e.g. the assertion of the South African president Thabo Mbeki that AIDS is not caused by HIV flew in the face of decades of research and threatened to undermine proper treatment of the disease.

11. Bush Administration's Misuse of Science • On February 18, 2004, 62 pre-eminent scientists AND researchers released a statement titled Restoring Scientific Integrity in Policy Making in the USA. • Scientists charged the Bush administration with widespread and unprecedented “manipulation of the process through which science enters into its decisions.” • Scientists accused Bush administration of : • Epidemic altering and concealing of scientific information by senior officials in various federal agencies. • Active censorship of scientific information that the administration considered threatening to its own philosophies • Restriction of the ability of government-supported scientists to freely communicate scientific ideas related to "sensitive" issues .

12. Integrity of Research Institutions • The organizational structure and processes that typify the mission and activities of a research institution can either promote or detract from the responsible conduct of research. • These process are in part determined by the external environment and are influenced by the dynamics between and among organizational members. • Any element or part of an organization can be viewed as a system in and of itself. • External conditions influence the inputs into an organization, affect the reception of outputs from an organization’s activities, and directly affect an organization’s internal operations.

13. Open systems model of internal environmental elements of a research organization showing: • Inputs / resources for organizational functions • Structures and processes that define an organization’s operation • Outputs / outcomes of activities carried out by individual scientists, research groups or teams, and other research-related programs. Source: National Academy of Sciences - http://www.nap.edu

14. Interrelatedness between research organizations and the various external influences that have an impact on integrity in research. • Systems and subsystems of the external-task environment are embedded within the general sociocultural, political, and economic environment. • Relationships also exist between and among the elements within the external environment. Source: National Academy of Sciences - http://www.nap.edu

15. Holistic Knowledge / Evidence • Rather than focus on the ethical or moral aspects of scientific integrity, focus here is on the process ofgenerating data and information and integrating it into soundknowledge (sound evidence) for decision-making. • “Integrity” by definition: • “honesty” • “a state of being entire or whole” • “Integrate” by definition: • “to combine parts into a whole” • “A whole.. is more than the sum of its parts.” Jan Christian Smuts, Holism and Evolution 1926

16. ‘Sound Evidence’: A Holistic View • A collection of data is not information. • A collection of information is not knowledge. • A collection of knowledge is not wisdom. • Information, knowledge, and wisdom are more than simply collections • Each concept represents more than the sum of its parts and has a synergy of its own.

17. A “WHOLE” AS A SYSTEM • “ A system is defined as a set of interacting units with relationships among them. The properties (or behaviour) of the system as a whole emerge out of the interaction of the components comprising the system.” • The interactions of the parts become more relevant to understanding the system than understanding the parts. • This definition of a system implies something beyond cause and effect.

18. The Ultimate System • In truth only one system, the “Universe" • All systems are sub-systems of a larger system. ? universe galaxy solar system world nation state community COMPLEXITY person organ cell molecule atom particle NUMBER OF SUB-SYSTEMS

19. Systems Thinking and Policy • Science is a way of thinking much more than it is a body of facts. Carl Sagan • “Systems thinking” offers a conceptual framework or model for ‘thinking differently’. • Systems thinking has permeated many scientific fields including: education, business management, human development, sociology, psychology, agriculture, ecology and biology, earth sciences.

20. Hard vs Soft Systems • Adopting a systemic perspective on solving policy problems therefore appear to offer a useful way of correcting these deficiencies. • In 1960s, a ‘hard’(quantitative) systemsapproach was touted as the policy science. • However, hopes not realized for variety of reasons; its comprehensive modelling too information-intensive and mathematical. • The ‘soft’ (qualitative) systems approach of systems thinking has increasingly been used since the 1990’s as a paradigm in policy planning and implementation. • Soft systems methods stress the self-organizing and adaptive capacities of appropriately designed systems.

21. Soft systems methods Subjective (interpretive) philosophy Systems + sociological theory base Flexible methodology Organizational problem-solving focus Creative / intuitive Analyst is facilitator Participative Organizational learning outcomes Several ambiguous outcomes Hard systems methods Objective philosophy Computer science + systems theory Rigid method Data, process, database technical focus Scientifically analytical Analyst is expert Analyst dominated Computer design outcomes One ‘correct’ solution Hard vs Soft Systems

22. Agro-ecosystem A hard systems view of a farming system; a biological network suitable for mathematical solution. A soft systems view of a farming system; an arena for gaining experience and increased understanding. Source: Robinson, B. 2003. 11th Australian Agronomy Conference)

23. Soft Systems Methodology (SSM) • Can be used both for general problem solving and in the management of change. • Used in the analysis of complex situations where there are divergent views about the definition of the problem — "soft problems" or policy options (e.g. How to improve health services delivery; How to manage disaster planning). • At the heart of SSM is a comparison between the world as it is, and some models of the world as it might be.  • Out of this comparison comes a better understanding of the world ("research"), and some ideas for improvement ("action").

24. SSM for Problem Solving • ‘Classic form’ of SSM consists of seven steps: • Problem unstructured by researchers • Problem situation expressed to capture “rich picture” • Create root definitions of relevant systems (i.e. social, political & environmental) • Making and testing conceptual models based upon world views • Comparing conceptual models with reality • Identifying feasible and desirable changes • Acting to improve the problem situation • Differences between models and reality become the basis for planning and policy making process. reality understanding and improvement conceptual models

25. Multi-agent systems (MAS) • Policy increasingly has to address topics that have to do with disequilibrium, dynamics, and locality. • The overwhelming complexity of biophysical and socio-economic constraints that increasingly characterize rural areas in developing countries necessitates the development of more sophisticated tools to support policy making in these areas. • Multi-agent systems (MAS) are a relatively new field in computer science that have been proposed as a modelling approach for establishing even higher levels of scientific integrity in the generation and evaluation of evidence for making policies. • Analogous to artificial intelligence.

26. MAS for Policy Making • Multi-agent models might be the preferred choice when heterogeneity and interactions of agents and environments are significant and policy responses cannot be aggregated linearly. • MAS can thus complement bio-economic simulation models which cannot fully capture the heterogeneity in biophysical and socio-economic constraints and the interactions between them. • There are several policy questions in the context of agricultural development of rural areas where MAS simulations may generate useful information for decision making on public investments in R&D and the targeting of policy interventions. • Examples of such policy questions: • Should funds be spent on crop breeding for stress resistance or in research for improved crop management? • Should micro-finance be promoted or should agricultural inputs be subsidized?

27. Agents • “An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors” • Agents may be persons, farms, markets, computer programmes or anything that is reactive, autonomous, and goal-oriented. • Agents may have the ability to communicate with other agents, learning, mobility, and flexibility. May even have personality and show emotions! AGENT EFFECTOR SENSOR OUTPUT INPUT SYSTEM ENVIRONMENT

28. Agent Flexibility • An intelligent agent is capable of flexible autonomous action. • FLEXIBLE: • Reactive: A reactive system is one that maintains an ongoing interaction with its environment, and responds to changes that occur in it (in time for the response to be useful) • Pro-active: Generating and attempting to achieve goals; not driven solely by events; taking the initiative i.e. goal directed behavior; recognizing opportunities • Social: Ability to interact with other agents via some kind of agent-communication language, and perhaps even cooperate with others.

29. PROBLEM SOLVING REACTIVE PROACTIVE Identifying a problem Definition of the system ANALYSIS ANALYSIS Understanding the problem Understanding the system POLICY POLICY MANAGEMENT MANIPULATION Maintaining the system “Improving” problem

30. BIBLIOGRAPHY • Balmann, A. 2000. Modeling land use with multi-agent systems: perspectives for the analysis of agricultural policies. • Berger, T. and Ringler, C. 2002. Trade-offs, efficiency gains and technical change – Modeling water management and land use within a multiple-agent framework, Quarterly Journal of International Agriculture41:119–144. • Berger, T., Schreinemachers, p., and Woelcke, J., 2006. Multi-agent simulation for the targeting of development policies in less-favored areas. Agricultural Syatems 88:28-43. • Checkland, P.  1981 Systems thinking, systems practice.  Chichester: Wiley.  • Checkland, P., and Holwell, S. 1998  Information, systems, and information systems: making sense of the field.  Chichester, UK: Wiley.   • Checkland, P.  and Scholes, J.  1991 Soft systems methodology in action.  Chichester: Wiley.  • Harrison MI. 1994. Diagnosing Organizations: Methods, Models, and Processes, 2nd ed. Thousand Oaks, CA: Sage. • Schreinemachers, P. Berger, T. and Aune, J.B., 2007. Simulating soil fertility and poverty dynamics in Uganda: A bio-economic multi-agent systems approach. Ecological Economics 64:387-401 • Union of Concerned Scientists. 2004. Scientific integrity in policy making: An investigation into the Bush administration's misuse of science. Cambridge (Massachusetts): Union of Concerned Scientists; 49 pp. • Woolridge, M. 2002. An Introduction to Multiagent Systems by John Wiley & Sons (Chichester, England).

31. THANK YOU!