Presenting statistical results to nonstatistical audiences

1. Presenting statistical results to nonstatistical audiences Jane E. Miller, PhD

2. Overview • Academic and nonstatistical audiences • Defined • Interests and background • Adapting description of methods • Adapting presentation of results

3. Why adapt material? • A survey by Sorian and Baugh of government policy makers showed that they • Want to know how the findings relate to issues • Don’t want to wade through a formal research paper • Complaints about many research reports • “too long, dense, or detailed” • “too theoretical, technical, or jargony”

4. Example: Policy analysts and consultants • Policy analysts must explain results of their models to experts in government or nonprofit agencies. • Economic consultants have to communicate results of their models to corporations or community development agencies. • Those experts are principally interested in • How to interpret and apply the findings. • Reassurance that you know the correct statistical methods.

5. How to adapt material • Familiarize yourself with your audience’s interests and likely applications of your study findings. • Present your analyses to match issues of concern to them. • Don’tmake them translate statistical results to fit their interests.

6. What statistics courses teach • Statistics courses emphasize • understanding statistical assumptions • estimating models • interpreting statistical tests • assessing coefficients and model fit. • Students expected to demonstrate mastery by • working with equations written in statistical notation • identifying the numbers for formal hypothesis testing

7. What academic papers look like • Detailed review of the literature • Comprehensive data and methods section • Statistical tables • Jargon and equations used as shorthand • A real mismatch with many applied audiences

8. Example: Study of family and county level factors associated with SCHIP disenrollment • SCHIP = State Children’s Health Insurance Program • Health insurance for children in low- to moderate-income families who lack other health insurance • Collaborative effort of • Rutgers University’s Center for State Health Policy • New Jersey Department of Human Services • Project applied discrete time hazards models in a multilevel (hierarchical linear model [HLM]) framework

9. Academic audiences for SCHIP study • Northwestern/University of Chicago Joint Center for Poverty Research (JCPR) • Funding agency • Policy oriented • Rutgers Institute for Health, Health Care Policy and Aging Research and Bloustein School of Planning and Public Policy • Both policy-oriented research units • University of Pennsylvania • Academic but not policy oriented • Health Services Research • Journal with research emphasis

10. Applied audiences for SCHIP study • New Jersey Department of Human Services • Raised policy question • Provided data • Client for deliverable • US Department of Health and Human Services • Funding source

11. Which would you rather have?

12. Which would your client rather have?

13. Chances of disenrollment by race, SCHIP plan, and county physician racial composition

14. Adapting results for nonstatisticians • Increase prominence of the substantive question. • Reduce emphasis on technical details of data and methods. • Rephrase jargon and statistical concepts into colloquial language. • Avoid equations or Greek symbols. • Minimize use of formal citations. • Translate results to show how they apply to real-world issues of interest to that audience.

15. Writing style and organization • Write a clear, well-organized narrative • What questions did you address? • What answers did you find? • How can the findings be applied? • Use standard expository writing guidelines • Good introduction • Present numbers as evidence • Explain what question each is intended to answer • Good summary of findings and what they mean • See article in Chance and podcast on presenting numbers as evidence.

16. How to write about technical stuff • Explaining why your methods are needed • Especially if using multivariate models • Showing how key variables are measured • Interpreting numeric values (coefficients) • Reporting statistical significance • Adapting tables and charts

17. Acronyms and statistical vocabulary • Even with a quantitatively sophisticated audience, don’t assume that people will know the statistical vocabulary used in other fields. • Define the term you use, then mention synonyms. • If you use acronyms, spell them out at first usage. • “HEDIS”(Health Plan Employer Data and Information Set) • “HLM”(hierarchical linear model) • Avoid acronyms if they are not familiar to the field or are used only once or twice.

18. Why your methods are needed • Explain what your model did that couldn’t have been answered with simpler techniques. • Incorporate the specific concepts you study. Poor: “We use logistic regression and a discrete-time hazards specification to assess relative hazards of SCHIP disenrollment, with plan level as our key independent variable.” Better: “Because chances of disenrollment from the State Children’s Health Insurance Program (SCHIP) vary by the amount of time enrolled, our analyses correct for differences in duration of enrollment across families when estimating the patterns for different income levels.”

19. Application of a method to your topic • Replacetechnical terms with familiar names. • Show how that method applies to your research question and data. • Poor: “The data structure can be formulated as a two-level hierarchical linear model, with families (the level-1 unit of analysis) nested within counties (the level-2 unit of analysis).”

20. Better presentation of methods: Tailored to the audience Better [for a nonstatistical but academic audience]: “The data have a hierarchical (or multilevel) structure, with families clustered within counties.” Better [for a lay audience]: “To disentangle the contributions of families’ and counties’ characteristics to the problem of program disenrollment, we used models that incorporated information at both levels.”

21. Measurement of key variables • To report an unfamiliar type of statistic, embed the definition in your explanation. Poor: “The sensitivity of the new screening test for diabetes is 0.90.” Better: “The new screening test had a sensitivity of 0.90, correctly identifying 90% of diabetics.”

22. Adapting tables and charts • Create small tables or charts • Divide up large complex tables into smaller parts • Focus each on one fact or pattern • Use simple, familiar formats • Replace standard errors and test statistics with • p-values • Symbols such as asterisks or daggers • Formatting such as color, italics, or bold

23. OK, but… Statistics are weighted to population level using weights provided with the NHANES III (US DHHS 1997). Differences across racial/ethnic origin groups were statistically significant for all variables shown (p < 0.01).

24. Low birthweight by race/ethnicity From second row of preceding table. p < 0.05

25. Minority racial groups have lower SES From bottom three rows of table. All p < 0.05

26. Interpreting OLS coefficients • Emphasize direction and size of the association • Name the specific variables involved • Incorporate units of measurement • Use colloquial language “OLS” = ordinary least squares regression

27. Examples of interpreting βs Poor: “Age and weight were correlated.” Poor version number2: “Beta was 10.7.” Better: “For each additional year of mother’s age at the time of her child’s birth, birth weight increased by 10.7 grams.”

28. Coefficients from logit models • Replace log-odds (logit coeffs) with odds ratios. • Can be described in terms of simple multiples. • Don’t need to use the term “odds ratio” at all! Poor: “The log-hazard of disenrollment for one-child families was 0.316.” Better: “Families with only one child enrolled in the program were about 1.4 times as likely as larger families to disenroll.”

29. Wording for statistical significance • State the conclusion of the statistical test, not the raw numbers or calculations. Poor: “The log-relative hazard for SCHIP plans C and D was 0.826 with a standard error of 0.142. Because the beta was more than 2.56 times the standard error, we conclude that the effect is statistically significant at p < 0.01.” Better: “Families in SCHIP plans C and D were roughly 2.3 times as likely to disenroll as those in plan B. A difference that large is unlikely to occur by random chance alone.”

30. Wording for LACK of statistical significance • “The difference between the disenrollment rates for Plans C and D could easily have occurred by chance alone.”

31. Summary • Get to know your audience before you write. • What questions do they want answered? • How familiar are they with statistics? • Avoid statistical language. • Report direction and size of associations in plain English. • Mention conclusions of inferential statistics, notthe raw numbers or calculations. • Use charts or simple tables to convey shape and size of numeric patterns visually.

32. Suggested resources • Chapter 20 in Miller, J. E. 2013. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. • Miller, J.E. 2006. “How to Communicate Statistical Findings: An Expository Writing Approach.” Chance. 19(4):43-49. • Nelson, D. E., R. C. Brownson, P. L. Remington, and C. Parvanta, editors. 2002. Communicating Public Health Information Effectively: A Guide for Practitioners. Washington DC: American Public Health Association. • Sorian, R., and T. Baugh. 2002. “Power of Information: Closing the Gap between Research and Policy.” Health Affairs 21 (2): 264–73.

33. Suggested online resources • Podcasts on • Reporting one number • Comparing two numbers or series of numbers • Creating effective tables and charts • Interpreting multivariate coefficients • Designing slides for a speech

34. Suggested practice exercises • Study guide to The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. • Questions #1 through 3 in the problem set for chapter 20 • Suggested course extensions for chapter 20 • “Reviewing” exercises #1 through 5 • “Writing” exercises #1, 2, 3, 6, 7 and 9 • “Revising” exercises #2 and 4

35. Contact information Jane E. Miller, PhD jmiller@ifh.rutgers.edu Online materials available at http://press.uchicago.edu/books/miller/multivariate/index.html