1 / 20

Jennifer L. Hay, PhD Task 4, Cognitive Interviews May 24, 2010 National Cancer Institute

PRO-CTCAE Face-To-Face Meeting #2 Advancing the Science of Adverse Symptom Monitoring in Cancer Treatment Trials. Jennifer L. Hay, PhD Task 4, Cognitive Interviews May 24, 2010 National Cancer Institute Rockville, MD. Key personnel –Task 4. Memorial Sloan-Kettering Cancer Center

hisano
Download Presentation

Jennifer L. Hay, PhD Task 4, Cognitive Interviews May 24, 2010 National Cancer Institute

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PRO-CTCAE Face-To-Face Meeting #2Advancing the Science of Adverse Symptom Monitoring in Cancer Treatment Trials Jennifer L. Hay, PhD Task 4, Cognitive Interviews May 24, 2010 National Cancer Institute Rockville, MD

  2. Key personnel –Task 4 • Memorial Sloan-Kettering Cancer Center • Jennifer Hay, Ethan Basch, Thomas Atkinson • M.D. Anderson • Tito Mendoza • Dana Farber Cancer Institute • Josh Gagne, VishViswanath, Lauren Becker • Duke University • Amy Abernethy, Lori Hudson • University of Tokyo • Yujiro Kuroda • National Cancer Institute • Bryce Reeve, Sonja Stringer, Gordon Willis • Interviewers • Mike Fruscione, Sabrina Jhawar, Dawn Lavene, Mary Shaw (MSKCC), Ariceli Gonzalez (M.D. Anderson), Jordan Lordata, Krista Rowe (Duke), Josh Gagne (DFCI)

  3. Cognitive Interviewing • Involves the manner in which targeted audience members understand, mentally process, and respond to the materials clinicians/researchers present to them, with particular attention to potential breakdowns in this process (Willis 2005). • Formulating a response to a survey question involves multiple cognitive steps (Cognitive Aspects of Survey Methodology; CASM approach, Tourangeau, 1984): • Comprehension: Involves participants’ beliefs about the question intent and meanings of specific terms used in question and response options. • Memory: Participants must identify the kinds of information that must be recalled and a strategy to maximize recall in order to retrieve the relevant information from memory. • Judgment: Participants must exert more/less motivation to answer the question accurately, more/less affected by social desirability. • Response processes: Participants must expend cognitive effort to match their response onto the categories presented.

  4. Strategies • Think-aloud protocols (Cacioppo, et al., JCCP, 1997) • ‘Questerviews’ (Adamson et al., J Health Serv Res Policy, 2004) • Cognitive interviewing procedures (Willis, 2005) All integrate qualitative strategies with quantitative strategies to inform survey development

  5. Task 4 Overall Aim:To evaluate patient comprehension of the draft PRO-CTCAE items and the extent to which the items correspond to the concepts of interest. Five critical questions for analysis: • What is the most appropriate wording of the introductory instructions for the PROs? • How comprehensible is the response scale for the items? • Do patients interpret and comprehend symptom terms – such as fatigue, sadness - in the same way? • What are the best verbal descriptors to use to anchor the response categories? • What constitutes the best wording for questions regarding interference with daily activities and symptom frequency?

  6. In the last seven days, what was the severity of your pain at its WORST?None/Mild/Moderate/Severe/Very Severe Standard Probes • What does “pain” mean to you as it is used in this question? • What did you need to recall in order to answer this question? • How easy or difficult was it to think back over the last seven days? Conditional/emergent probes • Probes that are triggered by participants’ behavior that can be developed prior or during the interview • Ex: You seem a little unsure of your answer here – if so, can you tell me why?

  7. Recruitment Strategy • We conducted two rounds of cognitive interviews in 107 patients; currently in the field (MSKCC) for round 3; 20 patients anticipated. • Eligible patients > age 18, English-fluent, receiving cancer chemotherapy/ radiotherapy for curative or palliative intent. • We prioritized recruitment of patients with lower educational attainment (> 25% hs or less), and recruitment of those with diverse disease sites, age, and racial/ethnic subgroup. • Interviews consist of: evaluation of about 25 PROs; retrospective probing; collection of basic demographic information (>1 hour). • Recruitment sites include the following medical centers and satellites: • Memorial Sloan-Kettering Cancer Center (MSKCC) • Dana Farber Cancer Institute (DFCI) • MD Anderson • Duke University Medical Center

  8. Interviewer training and supervision • Step 1: Task 4 collaborative formulation of cognitive interviewing scripts, interview summary forms, and macros (used by DFCI to descriptively summarize the results for each round) • Step 2: Josh Gagne (qualitative methodologist, DFCI) conducted on-site day long training at each site for all interviewers. • Qualitative methods, best practices for clinical interviewing, study rationale, presentation of the scripts, role plays, common problems/solutions. • Step 3: At each site, close collaborations between site PIs and interviewers with interviewers met with site PIs regularly (ie, MSK biweekly meetings to discuss recruitment rates, level of success in recruiting across disease sites and education levels. • Step 4: Bi-weekly Task 4 meetings to discuss study-wide interviewer issues, recruitment rates, etc. • Step 5: After Round 1, Josh Gagne conducted telecon “refresher” training for all interviewers. • Steps 3 and 4 were instituted again for Round 2 interviews.

  9. Number of interviews and issues addressed for each round by institution (n=107 completed; n=20 anticipated for Round 3)

  10. Data Analysis Procedure for Rounds 1 and 2 Interviews at MSKCC Interviews at MD Anderson Interviews at Duke Interviews at DFCI Summary forms for each interview Summary forms for each interview Summary forms for each interview Summary forms for each interview Comprehensive compilation of findings by Josh Gagne (DFCI) Executive Summary highlighting salient, frequent findings and problems. (Hay, Atkinson, Gagne) Presentation to Task 4 group via conference call Items deemed comprehensible Items requiring adjustment in verbiage (proceed to next round) Items not receiving adequate endorsement (<25% endorsement, proceed to next round)

  11. Number of interviews and issues addressed for each round by institution

  12. Round 1 Summary (n=60 interviews) • Comprehension of the PRO dimension terms (severity, frequency, interference) and their corresponding response scales was very high. • Participants tended to disregard (forget or override) the 7-day time frame Solution: Time frame added to every PRO, every dimension (severity, frequency, interference) • Participants requested a “not applicable” category for many PROs (sexual symptoms, radiation burns) Solution: This category was added to relevant PROs • For interference items, the term “interfere with daily activities” often prompted consideration of daytime interference only; rather than a 24-hour time period Solution: The PRO was expanded to include “interfere with daily or usual activities” • Comprehension of symptom terms was very high with over 85% of PROs generating no or few problems with comprehension.

  13. Examples of Round 1 symptom term issues (raised by >3 participants)

  14. Number of interviews anticipated and issues addressed for each round by institution

  15. Round 2 Summary (n=47 interviews) • While most participants evaluated their symptom severity using worst or average severity, their approach varied by whether the symptom was perceived to comprise discrete, remembered incidents (eg, vomiting) or global, more chronic conditions (eg., aching joints). • There was significant overlap between severity and frequency, less so with interference and duration. • Participants used a 24 hour time frame for interference when the symptom dictated it (eg, blurred vision could interfere with nighttime bathroom trips). • Frequency was as likely to be evaluated by number of days they have had a symptom as number of incidents of the symptoms.

  16. Example Round 2 symptom term issues (raised by >3 participants)

  17. Demographics (Rounds 1 and 2. n=107)

  18. Number of interviews anticipated and issues addressed for each round by institution

  19. Additional Deliverables • Final Round 3 Executive Summary: Early June 2010. • Final PRO assessment battery: Early-Mid June 2010. • Task 4 Poster presentation: ASCO, June 2010. • Task 4 team preparation of methods/findings manuscript: Summer 2010.

  20. Many thanks to our collaborators, interviewers, Laura Sit, and patient participants!

More Related