Health Care Stories are good for you

1. Health Care Stories are good for you Sharon Manson Singer, EvidenceNetwork.ca Steve Buist, Hamilton Spectator Jennifer Verma, CHSRF

2. Overview • Introduce EvidenceNetwork.ca • Talk about the Hierarchy of Evidence • What makes a good health story? A bad one? • Top ten questions to ask of health experts about their research • Where to go for data • How to assess the quality of data • Last word to you the audience

3. What does EvidenceNetwork.ca do? • EvidenceNetwork.ca links journalists with health policy experts to provide access to credible, evidence-based information.

4. What is EvidenceNetwork.ca? • EvidenceNetwork.ca is a non-partisan, web-based project funded by the Canadian • Institutes of Health Research and the Manitoba Health Research Council to make the • latest evidence on controversial health policy issues available to the media.

5. Why do we do it? • The Canadian Health Accord is scheduled for renegotiation in 2014. Canadians will have to make decisions about many complex health policy issues, including; • • Aging population impact • • Rising drug costs • • Health care accessibility • • Private sector financing/delivery • • User fees • • Sustainability of the healthcare system • EvidenceNetwork.ca is committed to working with the media to build a healthy dialogue around Canadian healthcare.

8. Hierarchy of Evidence – Top Tier • Systematic Review and meta-analysis • Use upper tier studies in a synthesis of research findings • Strongest evidence – only as good as the underlying studies Type of Study Expected Results

9. Hierarchy of Evidence – Upper Tier • Randomized experiments • Natural experiments • Well designed with sufficient sample size • High quality source of exogenous variation generating comparison group • Well designed pre and post measures • Analytical techniques are appropriate Type of Study How good is it?

10. Hierarchy of Evidence –Middle Tier • Some control in the assignment of treatment • Correlational studies • Limited source of exogenous variables or some control of selection or process • Well designed pre and post measures • Appropriate data with large sample • Reasonable approach to estimating counterfactuals Type of Study How good is it?

11. Hierarchy of Evidence – Lower Tier • Studies without a comparison group • Participant Satisfaction • Expert Opinions • Credible case selection with explicit causal logic model • Quality outcome measures • Collect feedback from participants on quality of intervention • Respected individuals or organizations with explicit rationale for opinion Type of Study How good is it?

12. Hierarchy of Evidence – Lower Tier • Exploratory case studies • Less credible or explicit case selection criteria, theory of change or outcome measure(s) Type of Study How good is it?

13. Contact Us • www.EvidenceNetwork.ca • Sharon Manson Singer, PhD • smansonsinger@gmail.com

14. Examples of bad science, dubious science or no science Steve Buist, Investigations Editor The Hamilton Spectator

15. Examples of bad science, dubious science or no science Luigi Di Bella and the “miracle” cure for cancer (Pulling for the underdog . . .)

16. Examples of bad science, dubious science or no science Autism and the MMR vaccine (Science is a lot easier if you just make it up . . .)

17. Examples of bad science, dubious science or no science Multiple sclerosis and “liberation therapy” (The underdog tale, with a modern-day social media twist . . .)

18. Examples of bad science, dubious science or no science Climate change and the human touch (What makes for a balanced story . . .)

19. 10 questions to consider when writing about science

20. 10 questions to consider when writing about science 1. Who is conducting the science?

21. 10 questions to consider when writing about science Who is conducting the science? Who is paying for the research?

22. 10 questions to consider when writing about science Who is conducting the science? Who is paying for the research? Who is paying the researcher?

23. 10 questions to consider when writing about science Who is conducting the science? Who is paying for the research? Who is paying the researcher? Where are the results being published?

24. 10 questions to consider when writing about science Who is conducting the science? Who is paying for the research? Who is paying the researcher? Where are the results being published? What was the population being tested?

25. 10 questions to consider when writing about science 6. What was the sample size?

26. 10 questions to consider when writing about science 6. What was the sample size? 7. How significant are the results?

27. 10 questions to consider when writing about science 6. What was the sample size? 7. How significant are the results? 8. What do other people think, and do those people have their own conflicts of interest?

28. 10 questions to consider when writing about science 9. How do these results fit into the context of what’s already known?

29. 10 questions to consider when writing about science 9. How do these results fit into the context of what’s already known? 10. Are there opposing viewpoints and how much weight should those viewpoints be given?

30. Making measures meaningful:Finding and interpreting healthcare data Jenn Verma, Director, Collaboration for Innovation and Improvement

31. mythbusters USING EVIDENCE TO DEBUNK COMMON MISCONCEPTIONS IN CANADIAN HEALTHCARE

32. Data, data…everywhere

33. THE LATEST RESEARCH SHOWS THAT WE REALLY SHOULD DO SOMETHING WITH ALL THIS RESEARCH

34. In a review of World Health Organization (WHO) and World Bank recommendations on five topics (contracting, healthcare financing, HHR, tuberculosis control and tobacco control): • 2/8 publications cited systematic reviews; • 5/14 WHO and 2/7 World Bank recommendations were consistent with both the direction and nature of effect claims from systematic reviews. Hoffman SJ, Lavis JN, Bennett S. 2009. The use of research evidence in two international organizations’ recommendations about health systems. Healthcare Policy 9(1): 66-86.

35. Adapted with permission from Health Quality Ontario (2011)

36. Adapted with permission from the Health Council of Canada (2011)

38. Health Indicators provide a Dashboard for Health and Healthcare They can let you know that things are running smoothly. They can alert you to problems that may need attention.

39. Interpreting Data… • rising BMI (Body Mass Index) doesn’t explain the root cause of weight gain. • In 2009, Canadians received 121 CT scans per 1000 people. There were also 8 MRI units per million population (vs. 12 MRI units per million as the OECD average). OECD (2011) reports Canada is “lagging behind,” but there is no agreed-upon benchmark. • In 2009, Canada had 2.4 physicians per 1000 population (vs. 3.1 OECD avg), but… • We have more physicians than ever before – Is this about supply or distribution and deployment? • We also have more nurses per 1000 people (9.4 in Canada vs. 8.4 OECD avg)

40. Interpreting Data • Comparing apples-to-apples? • Age standardization • Risk adjustment • Measuring intangibles • e.g., quality of life • Composite indicators

42. Lavis J. et al. 2009. SUPPORT Tools for evidence-informed health Policymaking (STP). Health Research Policy & Systems 7(Suppl 1). http://www.health-policysystems.com/content/7/S1/I1

43. “Numbers can’t ‘talk’ but they can tell you as much as your human sources can. But just like with human sources, you have to ask” (Niles, 2007). Niles R. 2007. Statistics every writer should know: A simple guide to understanding basic statistics, for journalists and other writers who might not know math. http://nilesonline.com/stats/

44. Useful Links • Support Tools for Policy Making http://www.chsrf.ca/PublicationsAndResources/ResearchReports/Support_Tools_for_Policy-Making.aspx • Mythbustershttp://www.chsrf.ca/PublicationsAndResources/Mythbusters.aspx What If? http://www.chsrf.ca/Programs/HealthcareFinancingInnovationAndTransformation/WhatIf.aspx CHSRF’sQuality of Healthcare in Canada: A Chartbook(2010) CIHI’sMaking Sense of Health Indicators (2011) HCC’sA Citizen’s Guide to Health Indicators (2011) CIHI’sMaking Sense of Health Rankings (2008) OECD’s Health Data 2011: How Does Canada Compare? (2011)