Realist Synthesis 101

Realist Synthesis 101 Dr Andrew Booth

Illustrating a Realist Review approach • For Which Schools Do School Performance League Tables Work? • How Do School Performance League Tables Work? • Do they work in a similar way to Hospital Performance League Tables? • What about Individual Surgeon Performance League Tables?

Developing Programme Theory • Programme Theory #1: Proponents of performance league tables believe their publication stimulates competition, • Programme Theory #2: and that, as each provider adopts “best practice,” the quality of services will improve.6 • Supporting Data: Some published evidence seems to support this. New York probably has the best established system for providing the public with information… Soon after publication of performance league tables based on providers, the risk adjusted mortality for bypass surgery declined, leading some to conclude that this was a direct result.7 • Counter Argument: One alternative explanation being that, once providers know their data will be used for comparative purposes, they may resort to “creative reporting.”89 • Programme Theory #3: Performance league tables may also improve patients' choice, and proponents argue that this is necessary for an efficient market economy by encouraging consumers to seek out high ranking providers.9

Developing Propositional Statements (IF-THENs) • IF Performance Tables are published THEN: • Staff at Well-performing schools (Context1) will change their reasoning/actions because (Mechanisms1) (e.g. Maintain competitive advantage)…..with the result (Outcome1) that…… • Staff at Average-performing schools (Context2) will change their reasoning/actions because (Mechanisms2) (e.g. Identification of good practice)…..with the result (Outcome1) that……. • Staff at Poor-performing schools (Context3) will change their reasoning/actions because (Mechanisms3) (e.g. Shame)……with the result (Outcome1) that…….. • CMO configurations where Mechanisms = changes that occur in people's reasoning and actions

Outcomes (intended and unintended) Mechanisms Mechanisms Programme intervention Realist synthesis Context Altered context Context triggers and Interacts with

Steps for conducting a realist synthesis • Formulate review question • Scope the primary literature • Decide on the scope of the review • Identify the programme components • Map the programme theory(s) • Assess effectiveness of components, considering context and mechanisms • Develop propositional statements • Identify patterns in propositional statements across cases (demi-regularities) • Search sources for theories related to the propositions • Produce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)

Formulating the review question

Formulating the review question • Realist questions aim to identify diverse literature on the intervention of interest, in order to evaluate what works for whom in what circumstances • They are relatively open ended, aiming to capture a broad spectrum of relevant primary research in the first instance • They aim to identify the relationships between Intervention, context, mechanisms and outcomes

Our realist questions • What approaches to community engagement are most effective in promoting peer support, to which people and in what circumstances? • (2) How does community-based peer support impact on understanding of existing health information and use of health information and health services to improve health and reduce health inequalities?

Preliminary theory for COPES • Community engagement aiming to promote peer support • increases health literacy • improves health • reduces health inequalities

Assumptions • Communities initiate and foster peer support • The process of providing community-based peer support influences the ability to develop health literacy • Health literacy creates a sense of empowerment, which is associated with a reduction in health inequalities • Community-based peer support raises awareness of issues with literacy and empowerment Peer support Community Health literacy

Scoping the literature • For the initial scoping, do a comprehensive search to ‘map the territory’ • Abstract sift: Identify and categorise abstracts by programmetheory, type of intervention

Scoping the literature for key concepts • Community engagement: Very small number of articles documenting ‘working collaboratively with and through’ community engagement for peer support • There were variations in community-based peer support in terms of • Amount of ‘emotional, appraisal and informational assistance’ • Degree of ‘created social network member’ • Degree of ‘experiential knowledge of a specific behaviour or stressor’ • Definitions of ‘Similar characteristics’ • The term health literacy was not used in peer support articles • so included all articles on health education/health promotion relating to ‘Understanding and use of health information and health services’ • Reduction of health inequalities – or any explicit discussion of health inequalities – was absent

Identifying sources of information in real life settings Health librarian Qualitative researchers Systematic reviewers CBPR researchers Propose a model Practitioners Community organizations Peer supporters Expert patients Test the emerging theory Identify what works Explore gaps Compare definitions Define concepts Network with participants Scope the literature Research knowledge Community knowledge

Steps for conducting a realist synthesis • Formulate review question • Scope the primary sources of information • Decide on the scope of the review • Map the programme theory(s) • Identify programme components • Assess effectiveness of components, considering context and mechanisms • Develop propositional statements • Identify patterns in propositional statements across cases (demi-regularities) • Search sources for theories related to the propositions • Produce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory)

Defining key concepts: community engagement and peer support • Community engagement is ‘the process of working collaboratively with and through groups of people affiliated by geographic proximity, special interest, or similar situations to address issues affecting the well-being of those people…It often involves partnerships and coalitions that help mobilize resources and influence systems, change relationships among partners, and serve as catalysts for changing policies, programs, and practices (CDC, 1997) • Peer support within a healthcare context is “...the provision of emotional, appraisal and informational assistance by a created social network member who possesses experiential knowledge of a specific behaviour or stressor and similar characteristics as the target population.” (Dennis, 2003). Emotional support includes expressions of caring, empathy, and encouragement, and is seen to enhance self-esteem. Appraisal support involves encouraging persistence and optimism for resolving problems, affirmation of a peer’s feelings, and reassurance that frustrations can be handled. Informational support is providing advice, suggestions, alternative actions, feedback and information relevant to the issue with which the peer is dealing.

Defining key concepts: health literacy • Health literacy – ‘the personal, cognitive and social skills which determine the ability of individuals to gain access to, understand, and use information to promote and maintain good health. These include such outcomes as improved knowledge and understanding of health determinants, and changed attitudes and motivations in relation to health behaviour, as well as improved self-efficacy in relation to defined tasks. Typically these are outcomes related to health education activities…’ (Nutbeam, 2001) • From a number of definitions for health literacy – we chose one that related to the notion of empowerment because it was relevant to health inequalities

(Nutbeam, 2009)

Health literacy model proposed by Sorensen et al (2012) • A year into our review, a conceptual analysis of the definitions was conducted, and a unifying model was produced (Sorensen et al, 2011). This model focuses on the capabilities of accessing, understanding, appraising and applying health information.

Me Team World Decide on the scope of the review World • Things to consider: • Amount of literature: Comprehensive versus sampling? • Diversity of literature: Consensus on programme theory? Variations on theory by what, who, when, setting • Access to other sources of information • Interests of those who commissioned the review • What was the original problem that needs research evidence? • International, national, regional, local? • What will the findings be used for? • Who is the audience? (this is sometimes wider than just commissioners) • Time and money • Experience

Setting the parametres via sampling • Programme theory was not used for sampling because • Theory was only explicitly cited in 28 instances • Studies combined elements from the Heath Belief Model, Theory of Planned Behaviour, social cognitive and social learning theories, the social support literature, social comparison theory, social network approaches, and empowerment models • Studies used both individual behaviour change and community-based theories, including community organization, diffusion of innovation, and social marketing • Domains for all the individual behaviour change theories are included in the Capabilities Opportunity Motivation Behaviour theory (COM-B, Michie et al 2011) • Pragmatic sampling on location: the most common United Kingdom health conditions in the peer support literature were reviewed first

Mapping literature topics to UK settings

Summary of searching, sifting and sampling

Identify the programme components • Programmescategorisedby Stage • Recruitment of peers • Training and supporting peers • Implementing peer support - what was delivered • Outcomes • Within each Stage, components were identified in the literature and through workshops with community members • Components put together in CMO configurations by the topic leads

Mapping programme theories • The theories and assumptions were documented using Context Mechanism Outcome (CMO) configurations for each health topic

Dealing with diverse theories • Some topics used a single theory, for example Diffusion of Innovation • Other topics did not explicitly state how theory informed design, but contained implicit assumptions about what would work • Concepts from both stated/implicit theories identified via CMOs • Dimensions of concepts for one topic used to question concepts in another topic area • For example, ‘recognising someone as a peer’ was an important mechanism. We asked ‘what does recognising someone as a peer mean across different health topics? • The dimensions we discovered included age, ethnicity, experience with the problem or condition • We also used Advisory Network as a source of propositional statements

Steps for conducting a realist synthesis • Formulate review question • Scope the primary literature • Decide on the scope of the review • Identify the programme components • Map the programme theory(s) • Assess effectiveness of components, considering context and mechanisms • Develop propositional statements • Identify patterns in propositional statements across cases (demi-regularities) • Search sources for theories related to the propositions • Produce a set of summary statements on what works for whom, in what circumstances and at what point in time (mid-range theory) Iterative

Turning CMOs into propositional statements: analysis of the literature

Searching for related theories • CMOs and propositional statements (IF-THENs) considered in terms of higher order concepts, for example

Mid-range theory • ‘Theories that lie between the minor but necessary working hypotheses that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behaviour, social organization and social change.’ (Merton, 1967: 39)

Mid-range theory • Mid-range theories relate empirical data to higher order constructs, and are ‘sufficiently abstract to deal with different spheres of social behaviour and social structure, so that they transcend sheer description or empirical generalisation’ (Merton, 1967: 68). • Middle range theory involves abstraction, of course, but…close enough to observed data to be incorporated in propositions that permit empirical testing.”

References Dennis, C.-L. (2003). Peer support within a health care context: a concept analysis. International journal of nursing studies, 40(3), 321–32. Kincheloe, J. and McLaren. Rethinking theory and research. (2005) In NK Denzin & YS Lincoln (eds) Sage Handbook of Qualitative Research. Merton R. On Theoretical Sociology. Five Essays, Old and New. New York: The Free Press, 1967 Nutbeam, D. (2008). The evolving concept of health literacy. Social science & medicine (1982), 67(12), 2072–8. Pawson, R. (2013) the science of evaluation: A realist manifesto. London: Sage. Sørensen, K., Van den Broucke, S., Fullam, J., Doyle, G., Pelikan, J., Slonska, Z., & Brand, H. (2012). Health literacy and public health: a systematic review and integration of definitions and models. BMC public health, 12(1), 80. Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013). RAMESES publication standards: realist syntheses. BMC medicine, 11(1), 21. doi:10.1186/1741-7015-11-21

Acknowledgements • Funded by the National Institute of Health Research Public Health programme PHR - 09/3008/04: Community-based peer support: A realist synthesis and model for promoting health literacy (COPES)

Realist Synthesis 101