Applicability and Transferability Role of review authors

1. Applicability and TransferabilityRole of review authors Belinda Burford Rebecca Armstrong Emma Tavender

2. Some definitions • applicability and transferability are two very important and related concepts (often used interchangeably) • Two sides of the one coin: • Applicability: whether or not it would be feasible to implement this intervention in a particular setting • OR “Is it possible?” • Transferability: whether we can expect to see similar results after implementing this intervention in a particular setting • OR “Even if it’s possible, will it work?”

3. Some challenges for complex reviews • Heterogeneity, heterogeneity, heterogeneity! • Population • Interactions between intervention, population characteristics and individual preferences/attitudes • Groups rather than individuals • Intervention • Multiple components • ‘active’ components?; interactions between components? • Fidelity/integrity difficult to measure • Comparisons • Poorly described • Outcomes • Proxy/surrogate outcomes • Setting • Important! Often poorly described

4. Image from: http://www.harlem-school.com/10TH/math_pdf/math.html

5. Organising the information What are some approaches you have taken in your reviews to help the reader consider applicability/transferability?

6. What do end-users need from our reviews? • Who are your end-users? • Differing needs • What do they need to know1: • Could it work in my jurisdiction? • Will it work? • What would it take to make it work? • Is it worth it? • Many resources for end-users for assessing applicability of systematic reviews: • SUPPORT tools to support policy-making • Applicability/transferability checklist2 • Many, many more… 1Lavis et al. Use of research to inform public policymaking Lancet 2004; 364: 1615–21 Lancet 2005 2 Wang et al 2005. Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1):76-83.

7. Starting from the end-user perspective: Applicability 1 Questions from: Wang et al 2005. Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1):76-83. 2 Table from: Burford et al. Forthcoming

8. Starting from the end-user perspective: Transferability 1 Questions from: Wang et al 2005. Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1):76-83. 2 Table from: Burford et al. Forthcoming

9. Organising the information

10. Approaches for dealing with this in your review • Extrinsic Approach • Give context-dependent guidance for applying the review findings • 1-2 paragraphs in the discussion • Implementation tables • Intrinsic Approach • Build into the review planned subgroup analyses • Good to do both

11. EXAMPLE: Interventions for preventing obesity in children1 • Data extraction • “implementation factors” based on “practice-relevant tool” • Extrinsic approach • Described these across studies in results section and summarised in the discussion • Intrinsic approach • Meta analyses to explore heterogeneity 1 Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.

12. Extrinsic Approach - Results • Effectiveness • Maintenance / Sustainability of effects • Equity • PROGRESS-Plus • Harm-adverse/unintended effects • Implementation • Intervention design and theoretical basis • Process evaluation (including: intensity/dose, acceptability, barriers/facilitators) • Resources needed (including: who delivered, funding, hours of implementation, other?) • Strategies to address disadvantage/diversity Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.

13. Extrinsic approach: Discussion Applicability of the evidence This review included studies from high income countries as well as lower-middle- and upper-middle-income countries, with five studies conducted in countries within the latter two groupings (Thailand, Brazil, Chile and Mexico). This means that, while predominantly conducted within high-income settings, the findings from this review may be generalisable to a number of settings. A total of nineteen studies specifically reported incorporating strategies to target socio-economic and/or cultural diversity or disadvantage. One such study was conducted outside of the high-income country setting, in Chile, an upper-middle-income country. Of the remaining eighteen studies, seven studies conducted in the USA were of interventions targeting African American children and their communities and another two studies targeted Native American communities. Other studies targeted participants of low socio-economic status, or were implemented in areas of social disadvantage. By far the most common setting for interventions included in this review were schools (43 studies). Other interventions were (or included) home-based (14 studies), community-based (six studies), or were set in a health service (two studies) or care setting (two studies). Eleven studies incorporated interventions across multiple settings. Most interventions took a combined dietary and physical activity approach to obesity prevention (31 studies). As a single strategy, targeting physical activity alone was more popular (17 studies) than targeting diet alone (seven studies). The predominant theoretical basis for interventions in this review was behaviour change theory. Other theories represented include environmental change strategies, the socio-ecological framework, social learning theory, health promotion theory, transtheoretical models, and youth development and resiliency based approaches. The theoretical basis for interventions was explicitly reported in approximately half of the included studies. Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.

14. Intrinsic approach: Synthesis • Meta analyses to explore heterogeneity: • Age group of participants • Setting: • Education only • Education plus other settings • Non-education • Intervention type: • PA • Diet • PA/diet combined • Intervention duration: • <=12 months • > 12 months Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.

15. The next update? • Intrinsic • meta-regression: investigate multiple factors simultaneously (continuous and categorical) for their possible influence on the size of the intervention effect • Caution: • Generally not considered when <10 studies in meta-analysis • Characteristics should be pre-specified (supported by scientific rationale) [ours will be post-hoc] • Select a small number of characteristics • Careful interpretation (Ch 9 of Cochrane Handbook) • Extrinsic • Implementation tables

16. Using qualitative synthesis to explore heterogeneity of complex interventions • 2 reviews: • Qualitative review on HIV/AIDS patients perspectives on adherence • Cochrane Review of intervention trials to improve adherence to therapy in HIV/AIDS patients • Tabulated whether the interventions in the CR corresponded with the patients’ perspectives on how to help them Candy et al. BMC Medical Research Methodology 2011, 11:124

17. Exploring temporal study design: Handwashingbehaviour in LMIC Vindigni et al. Systematic review: handwashing behaviour in low- to-middle-income countries: outcome measures and behaviour maintenance. Tropical Medicine and International Health. 2011;16(4):466-477

18. Handwashing: Temporal study design • Community settings (n=16 studies) • Formative research: average = 4.8 months • Intervention: average = 17.2 months • Evaluation: average = 9.6 months • Schools (n=4 studies) • Formative research: 1 study • Intervention: average = 4.8 months • Evaluation: average = 5 months • Health care settings (n=7 studies) • Formative research: 1 study • Intervention: average = 2.5 months • Evaluation: average = 3.2 months Vindigni et al. Systematic review: handwashing behaviour in low-tomiddle-income countries: outcome measures and behaviour maintenance. Tropical Medicine and International Health. 2011;16(4):466-477

19. Example from the EPOC Group Are there different amounts of relevant evidence? Eg. Specialist outreach clinics in primary care and rural hospital settings

20. Specialist Outreach review