Quantitative Content Analysis

1. Quantitative Content Analysis Wendy Anderson, MD MS UCSF CTSI Training in Clinical Research EPI 240: Qualitative Research Methods February 5, 2013

2. Objectives • Define quantitative content analysis • Identify when quantitative content analysis is a helpful method to employ • Practice the steps of developing a codebook from study data

3. Activities • Brief presentation and discussion: • Definition, examples of use, process • Working with data: • Identify possible categories in a sample of survey data (patient concerns) • Discuss how you might create a system to categorize them • Conclusion and take home points

4. Content Analysis • A systematic investigation aimed at describing the content of data • Can be qualitative or quantitative • Quantitative content analysis aims to reliably code the presence or absence or frequency of occurrence of an element of content • Can be used to test association between qualitative data (e.g. communication) and predictors or outcomes • Can be applied to a number of data sources: survey responses, recorded communication, visual data, etc.

5. Coding Schemes

6. Survey Data: Patient Concerns Anderson WG, Winters K, Auerbach AD. Patient concerns at hospital admission. Arch Intern Med. 2011 Aug 8;171(15):1399-400. • What are hospitalized patients most concerned about at admission? • Surveyed 109 patients admitted to medicine svc • “Please list all of the problems and concerns you want to talk with the doctor about today” • Quantitative content analysis: • Define categories of concerns • Identify most frequent categories • Illustrate whether these were relevant to hospital care

7. Recorded Communication: Empathy Mjaalandet al. Physicians' responses to patients' expressions of negative emotions in hospital consultations: a video-based observational study. Patient EducCouns. 2011 Sep;84(3):332-7. • How do physicians respond when patients express emotion in outpatient encounters? • 96 video-taped consultations • Verona Coding Definitions of Emotional Sequences • Identify patients' expression of negative emotions • Code physicians' subsequent responses • Identified 163 expressions of emotion • Only 22 (13%) of physician responses provided space for patients to discuss emotion further • No difference in physician response by gender, age • Surgeons more likely to give space-reducing responses

8. Recorded Communication: Satisfaction Eideet al. Physician communication in different phases of a consultation at an oncology outpatient clinic related to patient satisfaction. Patient EducCouns. 2003 Nov;51(3):259-66 • Does content of the different phases of an oncology consultation predict patient satisfaction with the visit? • Audio-recorded 36 cancer patients’ consultations with oncologists • RoterInteraction Analysis System (RIAS) to describe communication • Surveyed patients to assess satisfaction • Physician focus on psychosocial exchange predicted lower patient satisfaction

9. Recorded Communication: ICU Stapleton RD, et al. Clinician statements and family satisfaction with family conferences in the intensive care unit. Crit Care Med 2006;34:1679-85. • Is provision of emotional support associated with higher family satisfaction in ICU meetings? • Audio-recorded 51 ICU family meetings • Coded clinicians statements of emotional support • Linear regression: association statements and satisfaction • 3 types of statements associated with higher satisfaction: non-abandonment, ensuring comfort, supporting family decision

10. Effect of an Intervention: SCOPE Study Tulsky JA, et al. Enhancing communication between oncologists and patients with a computer-based training program: a randomized trial. Ann Intern Med 2011;155:593-601. • Can a computerized intervention increase oncologists’ expressions of empathy? • RCT of tailored CD-ROM vs. lecture • Audio-recorded post-intervention patient encounters • Coded % of patient expressions of negative emotion to which oncologists responded with empathy • CD-ROM oncologists more likely to express empathy; higher patient trust

11. Code Book Creation and Testing • Identify key variables: from qualitative analysis and/or literature review • Create definitions for when each variable will be tested: example, when to code, when not to • Test and refine codebook iteratively as you apply it to samples of your data • Assess inter-coder reliability: • Qualitatively, quantitatively (double code 20% of data) • Apply final codebook to data

12. Codebook Structure: Example 1

13. Codebook Structure: Example 2

14. Reliability: Inter-rater Agreement • Percent agreement – does not adjust for chance • Cohen’s Kappa score – • Adjusted for the proportion of cases in which the raters would agree by chance • Weighted vs. unweighted; Bi-rater vs. mult-irater • Kappa interpretation by convention < 0 No agreement 0.0 — 0.20 Slight agreement 0.21 — 0.40 Fair agreement 0.41 — 0.60 Moderate agreement 0.61 — 0.80 Substantial agreement 0.81 — 1.00 Almost perfect agreement

15. Kappa: Example Calculate Kappa: on-line: http://www.vassarstats.net/kappa.html (or can use statistical package) Kappa= 0.76 (substantial agreement) Provider speech turns in encounter = 1907 Both raters coded = 21 Coded only by Rater #1 = 6 Coded only by Rater #2 = 7