Program Science: Definition, Components and Issues

1. Program Science:Definition, Components and Issues James Blanchard, MD, MPH, PhD Director, Centre for Global Public Health University of Manitoba

2. Presentation Overview • Introduction to the “program science” concept • Discuss briefly the components of program science • Provide a few illustrative examples • Note: HIV/STI prevention lens

3. Brief History… leading to Rome • SOA and JB in N. Karnataka (2007): • Avahan (BMGF) supported large scale targeted intervention program • “How are strategic decisions being made, and why?” • “How are issues of heterogeneity between contexts being handled?” • “How does the project go to scale, and how is it managed?” • “What lessons are transferable to other contexts?” • Chicago, 2008, National meeting of the STD Control Directors: • Introduce the Program Science concept, and examine a range of issues • Philadelphia, 2008 (APHA): • Symposium on Program Science • London, 2009 (ISSTDR): • Meeting with scientific colleagues and NIH  proposed meeting • New Delhi, Bethesda, etc…: • Meetings with academic colleagues and program / policy leaders

4. Some Conceptual Roots • “Evidence-based Decision Making” • Decision makers actively using “evidence” in policy and program development • “Knowledge Translation” • “… the process of supporting the uptake of health research…”1 • “Implementation Science” • “…the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices… to improve the quality and effectiveness of health services and care.”2 1. Canadian Institutes of Health Research 2. Eccles and Mittman. Implementation Science;1:2006.

5. Some Potential Pitfalls in Knowledge Translation in Public Health • Knowledge production and synthesis often addresses a single component in a single sphere of knowledge: • Protective efficacy of a vaccine • Effectiveness of a specific behaviour change method • “Evidence traps” for public health practice: • Focus on single interventions (“magic bullets”) • Ignore the epidemiological context • Ignore the social, cultural, political, economic context

6. “Knowledge Translation Bias” • Inappropriate prioritization of interventions for which evidence has been generated and disseminated, often without regard to context “We have some evidence in support of this intervention, so let’s do a lot of it”

7. Program Science: Two Aspects • A research domain or enterprise (“program” as adjective): • Theoretical base • Methodologies • An interactive process (“program” as noun): • Dynamic interaction between scientists, program and policy leaders, and implementers/practitioners

8. Intervention vs. Program • Public Health Intervention • Specific technological or behavioural modality • Particular target group(s) • Focus on effectiveness, fidelity, coverage • Public Health Program • Multiple components (interventions) • Resource allocation between components, and sharing across components • Emphasis on optimizing population level impact

9. “Program Science” for HIV/STI Prevention:A Component Model Spheres of Knowledge Spheres of Practice Intended Outcomes • Choose: • The best strategy… • The right populations… • The right time… • Epidemiology • Transmission dynamics • Policy analysis • Health systems research Strategic Planning Policy Development Program Implementation • Do: • The right things… • The right way… • Efficacy / effectiveness • Operations research Program Management • Surveillance • Monitoring/evaluation • Operations research • Health systems research • Ensure: • Appropriate scale… • Efficiency… • Change when needed…

10. Program Science Components – Interactive Model

11. Illustrative Issues • Strategic planning • Rubrics for classifying epidemics and tailoring prevention strategies • Program implementation • Assessing the effectiveness and generalizability of single and combined interventions • Program management • Methods for scaling up and sustaining effective interventions

12. Strategic Planning <> Epidemiology and Transmission Dynamics • Focused on providing science-based guidance for resource allocation for prevention: • The best strategic approaches • The right population focus • The right timing of interventions • The challenges: • Epidemic heterogeneity • Understanding transmission dynamics and classifying epidemics appropriately

13. HIV Epidemic Remains Highly Heterogeneous, At the Global Level

14. Rubrics for HIV Epidemic Classification and Strategic Response • “Prevalence-based” classification • “Modes of Transmission” method for “Know Your Epidemic, Know Your Response” • “Epidemic typology” approach

15. “Prevalence Based” (numerical proxy) Approach • Based on HIV prevalence threshold rules: • <5% in all “high risk groups”  “Low Level” • >5% in at least one “high risk group”  “Concentrated” • >1% in the general population (usually antenatal women)  “Generalized” • Prescribed program strategies depending on epidemic typology

16. “Know Your Epidemic…” and “Modes of Transmission” Approach • A recent initiative focused on “modes of transmission” studies to understand “where incident HIV infections are occurring” • Standardized method for allocating incident infections to behaviourally defined sub-groups • Focus prevention strategy to “where new HIV infections are occurring”

17. “Epidemic Typology” Approach • Focuses on classifying epidemics based on transmission dynamics • Depends on an understanding of sexual structure and other transmission networks, in addition to epidemiological parameters • Considers epidemic potential and trajectory, not just current status • Focuses on assessing the relative contribution of identifiable high risk groups and networks to the overall epidemic in all groups

18. Uganda “Modes of Transmission” Study

19. Uganda “Know Your Epidemic, Know Your Response” Analysis

20. “Modes of Transmission” Analysis – India

21. “Modes of Transmission” –3 Districts in India

22. Same Epidemic, Different Prevention Strategies? Epidemic typology

23. Program Science Challenges • Development of methods for classifying epidemics • Development of methods for collecting key information: • Estimating the size of key populations • Measuring and describing sexual behaviours and networks • Linking transmission dynamics so prevention priorities and resource allocation

24. Issue – Assessing the effectiveness of interventions • More than 30 randomized controlled trials with HIV incidence as an end point: • Behavioural (partner reduction, condom use) • Technological (microbicides) • Biological (male circumcision) • Only 4 had “positive” results (i.e. reduced HIV incidence): • 3 individual RCTs on male circumcision • 1 randomized community trial on enhanced STD control – 2 subsequent trials contradicted the results • Possible explanations… • Intervention has no efficacy • Intervention couldn’t be implemented effectively, or at sufficient scale • Wrong intervention for that epidemic at that time

25. “Research driven” approach to intervention design and assessment Theoretical Basis Intervention Design Demonstrate efficacy / effectiveness Implement and Scale Up With “Fidelity”

26. Constraints of “research driven” models • Complexity and heterogeneity of transmission dynamics: • Designing and assessing the right intervention for a particular context • Secular trends and “co-interventions”: • Isolating a particular intervention or intervention package • Cost • Ethics

27. Differing Research Paradigms:“GRIP” to “GROP”1 • “GRIP” – Getting Research Into Policy • “GROP” – Getting Research Out of Practice 1. Parkhurst J, Weller I, Kemp J. Getting research into policy, or out of practice, in HIV? Lancet 2010:375;1414-5.

28. Pathways of activities and evidence flow into, and out of, policy and practice1 GRIP Evidence continuum From… Clear, “reliable” evidence, direct cause-effect To…. Evidence gaps, unclear causality, complex interventions, multiple interacting factors Implementation Policy Formulation Programme evaluation Activity flow Evidence flow 4. Outcome evaluation Implementation Designing interventions 1. Development of hypotheses and research questions 2, 3 Operational research and process evaluation GROP 1. Parkhurst et al. Lancet 2010.

29. Four Challenges within the GROP Model (Parkhurst et al.) • Better and more explicit hypothesis development: • Reviewing existing knowledge and explaining assumptions • Improved operational research: • Learning how complex interventions are implemented for scaling up what works in particular contexts • Increasing the use of process evaluation: • Investigate causal mechanisms by which an intervention works for particular groups in given settings • Proper outcome evaluation

30. Issues in Program Management and Scale Up • Contrasting conceptual approaches to scaling up • Management processes and models for scaling up

31. Conceptual Approaches to Scaling Up…(from David Peters et al.)

32. Management Models and Processes – Example, Targeted Interventions • Management by review and audit: • Established guidelines and norms • Generic episodic training programs • Monitor programs based on “effort based” targets • Periodic performance reviews and audits • “Red light, Orange light, Green light” performance assessments and funding decisions • Management through supportive supervision: • Established intervention package, with local adaptation • Ongoing training, technical assistance, knowledge sharing among implementers • Monitor based on “outcome based” targets • Supportive supervision with active joint problem solving • Participatory performance assessments

33. Issues in Program Evaluation • Expanding methodological base: • Complex interventions • Non-randomized methods • Understanding internal and external validity: • How does context influence the evaluation method and interpretation? • Specifying the level of evidence required for different types of interventions: • E.g. Prevention technologies vs. implementation models

34. Other Issues • Capacity building for “program science” • Integrating knowledge / science domains and improving knowledge translation processes

35. Final thoughts…. (from Michael Gibbons1) • “Socially robust knowledge” • “… can only be produced by much more sprawling socio/scientific constituencies.” • “… is superior to reliable knowledge both because it has been subjected to more intensive testing and retesting in many more contexts…. and also because of its malleability and connective capability.” • “…. is the product of an intensive (and continuous) interaction between data and other results, between people and environments, between applications and implications.”

36. Thank You