Involving Consumers in Developing and Implementing Systematic Reviews

1. Involving Consumers in Developing and Implementing Systematic Reviews Carol Sakala, PhD, MSPH Director of Programs Childbirth Connection Standards for Systematic Reviews of Clinical Effectiveness Research Institute of Medicine Workshop January 14, 2010

2. Background: Pathbreaking 1989 Pregnancy & Childbirth Resources • Effective Care in Pregnancy and Childbirth (Chalmers, Enkin, Keirse) “probably the most important book in obstetrics to appear this century and its value to the profession will be profound and long lasting” British Journal of Obstetrics and Gynecology 1990;97:967-73. • A Guide to Effective Care in Pregnancy and Childbirth (Enkin, Keirse, Chalmers • Oxford Database of Perinatal Trials (Chalmers) Inspired establishment of Cochrane Collaboration to carry out similar work in all fields of health and medicine

3. Background: Response to Lack of Uptake in U.S. Decade Later Childbirth Connection (then known as Maternity Center Association) begins long-term national program to promote evidence-based maternity care, 1999- • at core: translate best evidence, especially systematic reviews, and selectively commission and carry them out • audiences: health professionals, childbearing women, policy makers, journalists • continues work beginning in 1918 to improve maternity care quality focusing on needs and interests of childbearing women and families Mission: improve the quality of maternity care through research, education, advocacy and policy

4. Role of Consumer in SR Process Complement professionals and add value as team members in all phases • prioritizing topics and questions • planning reviews/developing protocols • participating on author teams that develop reviews • commenting on draft protocols and reviews alongside peer referees • accessing completed reviews • participating in evidence-informed decision making • contributing to other translational activities • periodically updating reviews

5. Who Should Be Considered a Consumer? Limited role for randomly selected member of the public Potential for substantive contributions from those who • are experienced with the condition, and perhaps intervention, in question • have been trained and supported to participate in clinical effectiveness activities • are experienced in participating in clinical effectiveness activities • are active in consumer or patient support and advocacy organizations • are on the professional staff of organizations devoted to research, education, advocacy and policy on behalf of consumers

6. Who Should Be Considered a Consumer? Diverse complementary perspectives and contributions suggest: avoid single “consumer representative” when possible (just as we benefit from diverse professional perspectives) Consumers drawn to participate in clinical effectiveness activities often sophisticated, self-selecting for skills and experience • resourceful users of Pub Med and other online resources • in active communication with others who share interests • basic or greater critical appraisal skills • continuous environmental scanning

7. Lessons from Existing Models Cochrane Pregnancy and Childbirth Group Consumer Panel to referee draft protocols, systematic reviews and review updates (12 years’ experience) Consumers often identify issues that professionals may not raise, e.g., • outcomes: harms, longer-term effects, psychosocial impacts, experience of undergoing intervention • background sections: more precise and informative; sound evidence • research priorities: prevention, less invasive interventions, validity of accepted but poorly assessed indications, whether or when to use interventions • language: clear, sensitive • outsider insights: role of co-interventions, limitations of RCTs

8. Lessons from Existing Models Consumer Panel evaluation: review authors generally valued consumer contributions and review group editors were very enthusiastic Consumers often value opportunity to be at table, participate diligently • Review group editor to independent evaluator “I think I can speak for all [the] editors on this because it’s come up at every single one of our editorial meetings, that the [consumer] input in general is fantastic. It’s the highest quality we get. It’s certainly the most detailed and thoughtful.” • Review author to independent evaluator “It was very long when it came…. It took me awhile to get over the heart-sink. I mean, it was enormously valuable.”

9. Lessons from Existing Models Factors contributing to success of Consumer Panel model • leadership: strong support from administrative office and review group editors • established structure for consumer involvement, e.g., procedures, forms, informational documents, regional coordinators • paid coordinator • coordinator with strong advocacy and interpersonal skills • coordinator consumer-professional liaison role • accessible collated feedback with summary of key points helps busy professionals

10. Lessons from Existing Models Consumer member of Cochrane Review author team • with 3 experienced, sensitive, respected reviewers and researchers • division of labor: study assessment, data abstraction/analysis; relevance to low-income settings; methodology; consumer concerns • resulted in strong, gratifying, highly accessed review • with change in funding opportunities, Consumer Panel coordinator received support to participate in series of reviews, published this month and incorporating many of the common consumer concerns

11. Training and Support for Consumer Participation Just as professional participants draw on education, experience and institutional support, so are training and support beneficial for consumers Consumers likely to be more effective when they • are conversant in research language and concepts • have fundamental critical appraisal skills • understand systematic review procedures • have confidence to engage respectfully with professionals

12. Training and Support for Consumer Participation Support Project LEAD model from National Breast Cancer Coalition Based on current involvement in systematic reviews, guidelines (Guidelines International Network) and performance measures (National Quality Forum) Train and support consumers to work in a specific clinical domain (e.g., breast cancer) across continuum of clinical effectiveness activities (e.g., primary research, SRs, guidelines, performance measures, decision tools) • content and quality depend on upstream sources • basic biology applies across continuum • key personnel/relationships, organizations and agencies involved across continuum

13. Move Toward Standardized Consumer Role Standardizing consumer role in systematic reviews would foster consistent involvement and assist those with limited experience Begin with general principles, e.g., • involving consumers in all phases of systematic reviews • achieving a balance of perspectives and experiences • avoiding and minimizing conflicts of interest • endorsing but not requiring sources of training and support • compensating consumers for costs of participation • acknowledging participation of those involved • providing those involved with final products Encourage evaluation (evidence!) and refinement

14. Welcome Standardized Grading Scheme Would reduce confusion for consumers and professionals, especially if consistent with evolving international standard, such as GRADE Would help align all stakeholders to foster national quality improvement aims Would provide basis for national educational programs • important educational role for advocacy and professional organizations • aids consumer interest in knowing all or most benefits/harms • provides support for developers of consumer educational materials Consider also a standardized format “Plain Language Summary”

15. Other Consumer Viewpoint Recommendations Address strong consumer interest in understanding harms, a shortcoming of many SRs, through focused attention to harms in report, including use of most appropriate study designs Give special attention to transparency and conflict of interest issues to foster integrity of results Develop publicly available bibliographic databases of SRs, by clinical area; make good use of available SRs (e.g., estimated 2,000-3,000 for maternity care alone, but dispersed, so missed opportunities) Support access to good sources of information about consumer views, preferences, experiences (ex. national Listening to Mothers surveys) Recognize interrelationships across continuum of clinical effectiveness activities and foster coherent integration across the full continuum