1 / 45

Secondary Aims Using Data Arising from a SMART

Secondary Aims Using Data Arising from a SMART. Module 6—Day 2 Getting SMART About Developing Individualized Adaptive Health Interventions Methods Work, Chicago, Illinois, June 11-12 Daniel Almirall & Susan A. Murphy. Secondary Aims, Outline. Discuss what is a “more deeply-tailored ATS”

zaria
Download Presentation

Secondary Aims Using Data Arising from a SMART

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Secondary Aims Using Data Arising from a SMART Module 6—Day 2 Getting SMART About Developing Individualized Adaptive Health Interventions Methods Work, Chicago, Illinois, June 11-12 Daniel Almirall & Susan A. Murphy

  2. Secondary Aims, Outline • Discuss what is a “more deeply-tailored ATS” • Review of auxiliary data typical of SMARTs • Q-Learning: Using backward-induction logic • S(a): Analysis of time-varying tailoring variables of second-stage treatment • There is one step to this: Step (i) • S(b): Analysis of baseline tailoring variables of first-stage (assuming an optimal second-stage treatment) • There are two steps to this: Steps (ii) and (iii)

  3. What is a more deeply-tailored ATS? • To understand this, we first review what are the “embedded ATSs” within the ADHD SMART study • Recall there are 4 SMART-design embedded ATSs.

  4. Recall the Prototypical SMART Design: ADHD SMART Example Continue Medication Responders Medication Increase Medication Dose R Non-Responders Add Behavioral Intervention R Continue Behavioral Intervention Responders Behavioral Intervention Increase Behavioral Intervention R Non-Responders Add Medication

  5. These are the 2 embedded ATS that employ the “add other treatment” tactic Continue Medication Responders Medication Increase Medication Dose R Non-Responders Add Behavioral Intervention R Continue Behavioral Intervention Responders Behavioral Intervention Increase Behavioral Intervention R Non-Responders Add Medication

  6. These are the 2 embedded ATSs that employ the “intensify initial treatment” tactic Continue Medication Responders Medication Increase Medication Dose R Non-Responders Add Behavioral Intervention R Continue Behavioral Intervention Responders Behavioral Intervention Increase Behavioral Intervention R Non-Responders Add Medication

  7. So why consider a more deeply-tailored ATS? • First, it may be that some participants (e.g., those who have used MED in the past) may benefit more from MED vs BMOD. • Second, certain types of non-responders (e.g., those who do not adhere to initial treatment) may benefit more from INTENSIFY vs ADD.

  8. What is a more deeply-tailored ATS? • It may look like this… • If the child used MED in prior year, then begin with MED; otherwise, begin with BMOD. • If the child is non-responsive and non-adherent to either first-line treatment, then AUGMENT with the other treatment option. • If the child is non-responsive but adherent to either first-line treatment, then it is better to INTENSIFY first-line treatment. • If the child is responsive to first-line treatment, then CONTINUE first-line treatment.

  9. Secondary Aims, Outline • Discuss what is a “more deeply-tailored ATS” • Review of auxiliary data typical of SMARTs • Q-Learning: Using backward-induction logic • S(a): Analysis of time-varying tailoring variables of second-stage treatment • There is one step to this: Step (i) • S(b): Analysis of baseline tailoring variables of first-stage (assuming an optimal second-stage treatment) • There are two steps to this: Steps (ii) and (iii)

  10. Other Measures Collected in a SMART O1 A1 O2 / R Status A2 Y Continue Medication Responders Medication Increase Medication Dose R Non-Responders Add Behavioral Intervention R Continue Behavioral Intervention Responders Behavioral Intervention Increase Behavioral Intervention R Non-Responders O1 = Demog., Pre-txt Medication Hx, Pre-txt ADHD scores, Pre-txt school performance, ODD Dx, … Add Medication O2 = Month of non-response, adherence to first-stage txt, …

  11. How are O1 and O2 used? • We can use the auxiliary data O1 to help decide which is best MED v BMOD for certain individuals • We can use the auxiliary data O1 and O2 to help decide which is best INTENSIFY v ADD for certain individuals

  12. Secondary Aims, Outline • Discuss what is a “more deeply-tailored ATS” • Review of auxiliary data typical of SMARTs • Q-Learning: Using backward-induction logic • S(a): Analysis of time-varying tailoring variables of second-stage treatment • There is one step to this: Step (i) • S(b): Analysis of baseline tailoring variables of first-stage (assuming an optimal second-stage treatment) • There are two steps to this: Steps (ii) and (iii)

  13. Using Q-Learning to develop a more richly-individualized ATS • Q-Learning is an extension of regression to sequential treatments. • Q-Learning results in a proposal for an adaptive treatment strategy with greater individualization. • A subsequent trial would evaluate the proposed adaptive treatment strategy versus usual care.

  14. Three Steps in Q-Learning Regression Work backwards (as you would for a project time-line) Do a regression to learn how to more deeply- tailor second-stage treatment using O1 and O2 (in ADHD SMART, this is only with non-responders) Assign each non-responder the value Ŷi , an estimate of the outcome under the second-line treatment that yields best outcome for that person i. Responders get their observed Yi. Using Ŷi do a regression to learn how to more deeply-tailor first-stage treatment using O1

  15. Step (i) of Q-Learning: Second-stage treatment tailoring? Continue Medication Responders Medication Increase Medication Dose R Step (i)a. Non-Responders Add Behavioral Intervention Adherence to initial MED R Continue Behavioral Intervention Responders Behavioral Intervention Increase Behavioral Intervention R Step (i)b. Non-Responders Add Medication Adherence to initial BMOD O1 A1 O2 / R Status A2 Y

  16. Step (i) of Q-Learning: Second-stage treatment tailoring? To accomplish Step (i) of Q-Learning, you may fit a regression model such as this: Y = b1 + b2 o11c + b3 o12c + b4 o13c + b5 o14c + b6 o21c + b7 a1 + b8 o22 + b9 a2*a1 + b10 a2*o22 + b11 a2 + error From such a model we would learn if, for example, o22 (adherence to first stage txt) is a strong moderator of the effect of a2.

  17. Step (i) of Q-Learning: Learn best second-stage treatment for non-responders INT –ADD ≈ -1.3 to -2.1 Among non-adherers to either first-stage treatment, better to add txt. This analysis is with simulated data.

  18. Step (i) of Q-Learning: Learn best second-stage treatment for non-responders Among adherers to MED, better to intensify MED. INT –ADD ≈ +1.02 This analysis is with simulated data.

  19. Step (i) of Q-Learning: Learn best second-stage treatment for non-responders Among adherers to BMOD, better to intensify but not by much (NS). INT –ADD ≈ +0.25 This analysis is with simulated data.

  20. Three Steps in Q-Learning Regression Work backwards (as you would for a project time-line) Do a regression to learn how to more deeply- tailor second-stage treatment using O1 and O2 (in ADHD SMART, this is only with non-responders) Assign each non-responder the value Ŷi , an estimate of the outcome under the second-line treatment that yields best outcome for that person i. Responders get their observed Yi. Using Ŷi do a regression to learn how to more deeply-tailor first-stage treatment using O1

  21. Step (ii) of Q-Learning: Graphically For patients who would initially begin MED and don’t respond, we assign them their optimal outcome under the treatment that is best for them based on adherence or non-adherence. For those who would begin on BMOD and don’t respond, assign them their optimal outcome based on adherence or non-adherence. This analysis is with simulated data.

  22. Three Steps in Q-Learning Regression Work backwards (as you would for a project time-line) Do a regression to learn how to more deeply- tailor second-stage treatment using O1 and O2 (in ADHD SMART, this is only with non-responders) Assign each non-responder the value Ŷi , an estimate of the outcome under the second-line treatment that yields best outcome for that person i. Responders get their observed Yi. Using Ŷi do a regression to learn how to more deeply-tailor first-stage treatment using O1

  23. Step (iii) of Q-Learning: First-stage treatment tailoring? To accomplish Step (iii) of Q-Learning, you may fit a regression model such as this: Ŷ = b1 + b2 o11c + b3 o12c + b4 o14c + b5 o13 + b6 o13*a1 + b7 a1 + error From such a model we would learn if, for example, o13 (prior MED) is a strong moderator of the effect of a1 such that it would make a good tailoring variable.

  24. Step (iii) Q-Learning: Learn optimal first-treatment for all given optimal future txt Among kids using MED in prior year, it is better to start with MED. -0.50 = BMOD – MED This analysis is with simulated data.

  25. Q-Learning Step (iii): Learn optimal first-treatment for all given optimal future txt Among kids not using MED in prior year, it is better to start with BMOD +0.62 = BMOD – MED This analysis is with simulated data.

  26. SAS software to perform Q-Learning • We next show you how to do • Step (i) using regression • Steps (ii) and (iii) using a SAS add-on known as PROC QLEARN • We will use the two example regression models shown previously.

  27. SAS code for Step (i) of Q-Learning: Second-stage treatment tailoring * use only non-responders; data dat10; set dat1; if R=0; run; * comparison of add vs intensify given first line txt and adherence; proc genmod data = dat10; model y = o11c o12c o13c o14c a1 o21c o22 a2 a2*a1 a2*o22; * effect of add vs intensify given first-line = MED x ADH status; estimate 'INT vs ADD for NR MED ADH' a2 2 a2*a1 -2 a2*o22 2 ; estimate 'INT vs ADD for NR MED Non-ADH' a2 2 a2*a1 -2 a2*o22 0 ; * effect of add vs intensify given first-line = BMOD x ADH status; estimate 'INT vs ADD for NR BMOD ADH' a2 2 a2*a1 2 a2*o22 2 ; estimate 'INT vs ADD for NR BMOD Non-ADH' a2 2 a2*a1 2 a2*o22 0 ; run; Contrast Estimate Results 95% Conf Limits Label Estimate Lower Upper P-value INT vs ADD for NR MED ADH 1.0240 0.4131 1.6350 <.0001 INT vs ADD for NR MED Non-ADH -1.3412 -1.9896 -0.6927 <.0001 INT vs ADD for NR BMOD ADH 0.2503 -0.3950 0.8956 0.4471 INT vs ADD for NR BMOD Non-ADH -2.1149 -2.7050 -1.5248 <.0001 This analysis is with simulated data.

  28. Try it yourself in SAS • Go to the file: sas_code_modules_4_5_and_6_ADHD.doc • Copy the SAS code on Page 11 • Paste into SAS Enhanced Editor window • Press F8 or click the Submit button (the little running guy)

  29. SAS code for Step (i) of Q-Learning: Second-stage treatment tailoring * use only non-responders; data dat10; set dat1; if R=0; run; * comparison of add vs intensify given first line txt and adherence; proc genmod data = dat10; model y = o11c o12c o13c o14c a1 o21c o22 a2 a2*a1 a2*o22; * effect of add vs intensify given first-line = MED x ADH status; estimate 'INT vs ADD for NR MED ADH' a2 2 a2*a1 -2 a2*o22 2 ; estimate 'INT vs ADD for NR MED Non-ADH' a2 2 a2*a1 -2 a2*o22 0 ; * effect of add vs intensify given first-line = BMOD x ADH status; estimate 'INT vs ADD for NR BMOD ADH' a2 2 a2*a1 2 a2*o22 2 ; estimate 'INT vs ADD for NR BMOD Non-ADH' a2 2 a2*a1 2 a2*o22 0 ; run; Contrast Estimate Results 95% Conf Limits Label Estimate Lower Upper P-value INT vs ADD for NR MED ADH 1.0240 0.4131 1.6350 <.0001 INT vs ADD for NR MED Non-ADH -1.3412 -1.9896 -0.6927 <.0001 INT vs ADD for NR BMOD ADH 0.2503 -0.3950 0.8956 0.4471 INT vs ADD for NR BMOD Non-ADH -2.1149 -2.7050 -1.5248 <.0001 This analysis is with simulated data.

  30. SAS add-on: PROC QLEARN • What do we provide PROC QLEARN? • Data set with O1, A1, R, O2, A2, Y • The second stage regression model • Y ~ O1, A1, O2, A2 • Specify among who to do this regression (e.g., non-responders in ADHD SMART) • The first stage regression model • Ŷ ~ O1, A1

  31. SAS add-on: PROC QLEARN • What does PROC QLEARN do? • It implements Step (i): Second-stage Regression • Provides us with second-stage regression parameter estimates • It implements Step (ii): Obtains Ŷ • It assigns non-responders the outcome under the best second-stage treatment based on Step (i). • It assigns responders their observed outcome. • It implements Step (iii): First-stage Regression • Provides first-stage regression parameter estimates • Provides appropriate confidence intervals for the first-stage regression parameter estimates

  32. SAS add-on: PROC QLEARN What does the syntax look like? PROC QLEARN <options for input> ; MAIN1 variables ; TAILOR1 variables ; MAIN2 variables ; TAILOR2 variables ; RESPONSE variable ; STG1TRT variable ; STG2TRT variable ; STG2SAMPLE variable ; ALPHA value ; RUN;

  33. SAS add-on: PROC QLEARN Model Specification PROC QLEARN <options for input> ; MAIN1 variables ; TAILOR1 variables ; MAIN2 variables ; TAILOR2 variables ; RESPONSE variable ; STG1TRT variable ; STG2TRT variable ; ... RUN; STAGE2 REGRESSION RESULTS ARE OUTPUT LIKE THIS: INT + MAIN2 + TAILOR2 + TAILOR2*STG2TRT + STG2TRT STAGE1 REGRESSION RESULTS ARE OUTPUT LIKE THIS: INT + MAIN1 + TAILOR1 + TAILOR1*STG1TRT + STG1TRT

  34. Recall our second-stage regression model data dat10; set dat1; if R=0; run; * use only non-responders; proc genmod data = dat10; model y = o11c o12c o13c o14c a1 o21c o22 a2 a2*a1 a2*o22; run; This is how that same model is specified in PROC QLEARN data dat11; set dat1; S = 1-R; run; * use only non-responders; proc qlearn data=dat11; ...[next slide] main2 o11c o12c o13c o14c o21c; tailor2 a1 o22; stg2sample s; response y; stg2trt a2; ...[next slide] run; Y = b1 + b2 o11c + b3 o12c + b4 o13c + b5 o14c + b6 o21c + b7 a1 + b8 o22 + b9 a1*a2 + b10 o22*a2 + b11 a2 + error

  35. PROC QLEARN Full Example, ADHD Data data dat11; set dat1; S = 1-R; run; proc qlearn data=dat11 contrasts1=contrasts1 deriveci; main1 o11c o12c o14c; tailor1 o13; main2 o11c o12c o13c o14c o21c; tailor2 a1 o22; stg2sample s; response y; stg1trt a1; stg2trt a2; run; This requests contrasts of interest (in GENMOD they are call “estimates”. See next slide for details. This requests confidence intervals by Laber and Murphy (2012) This will ask SAS to fit the following two regressions: Stage 2: Y = b1 + b2 o11c + b3 o12c + b4 o13c + b5 o14c + b6 o21c + b7 a1 + b8 o22+ b9 a1*a2 + b10 o22*a2 + b11 a2 + error (fit only with non-responders, S=1) Stage 1: Ŷ = b1 + b2 o11c + b3 o12c + b4 o14c + b5 o13 + b6 o13*a1 + b7 a1 + error

  36. PROC QLEARN: Contrast Matrix data contrasts1; input M1 M2 M3 M4 M5 M6 M7; *no. of mtrx cols = no. of stage 1 parameters ; * the cols correspond to the parameters in stage 1 model: ; * b1 + b2 o11c + b3 o12c + b4 o14c + b5 o13 + b6 a1 o13 + b7 a1 ; * each row corresponds to a different linear comb of the b's. some linear ; * combos can be used to obtain mean outcomes for different children under ; * initial BMOD vs initial MED. whereas other linear combos can be used to ; * compare mean outcomes between MED vs BMOD for different children. ; datalines; 1 0 0 0 1 1 1 1 0 0 0 1 -1 -1 0 0 0 0 0 2 2 1 0 0 0 0 0 1 1 0 0 0 0 0 -1 0 0 0 0 0 0 2 ; * taking into account optimal future decisions (to intensify vs augment) ; * contrast 1 = mean outcome under BMOD for children w/yes med in prior year; * contrast 2 = mean outcome under MED for children w/yes med in prior year ; * contrast 3 = mean diff (BMOD - MED) for children w/yes med in prior year ; * contrast 4 = mean outcome under BMOD for children w/no med in prior year ; * contrast 5 = mean outcome under MED for children w/no med in prior year ; * contrast 6 = mean diff (BMOD - MED) for children w/no med in prior year ; run;

  37. Try it yourself in SAS • Go to the file: sas_code_modules_4_5_and_6_ADHD.doc • Copy the SAS code on Page 14 • This code defines the contrast matrix and runs PROC QLEARN • Paste into SAS Enhanced Editor window • Press F8 or click the Submit button (the little running guy)

  38. PROC QLEARN: Results PROC QLEARN -- Q-Learning and Adaptive Confidence Intervals Number of observations in dataset: 150 Number of observations in stage 1: 150 Number of observations in stage 2: 99 First Stage Regression Result -------------------------------------------------------------- Parameter Confidence Interval Variable Estimates Upper Lower intercept 3.4575 3.7119 3.2135 o11c -0.4407 -0.0777 -0.7903 o12c -0.3366 -0.1552 -0.5061 o14c 0.5650 1.0026 0.1586 o13 -0.0418 0.3439 -0.4235 o13 :a1 -0.5610 -0.2836 -0.8292 a1 0.3104 0.4992 0.0993 Parameter Confidence Interval Contrasts Estimates Upper Lower Contrast 1 3.1651 3.6676 2.6437 Contrast 2 3.6663 4.0535 3.3291 Contrast 3 -0.5012 -0.0032 -1.0399 Contrast 4 3.7679 4.0726 3.4328 Contrast 5 3.1471 3.4739 2.8384 Contrast 6 0.6208 0.9984 0.1986

  39. What did we learn with Q-learning? • Adaptive Treatment Strategy Proposal • If the child used MED in prior year, then begin with MED; otherwise, begin with BMOD. • If the child is non-responsive and non-adherent to either first-line treatment, then AUGMENT with the other treatment option. • If the child is non-responsive but adherent to either first-line treatment, then it is better to INTENSIFY first-line treatment. • If the child is responsive to first-line treatment, then CONTINUE first-line treatment. This Q-learning analysis was done with simulated/altered data.

  40. What did we learn with Q-learning? • Adaptive Treatment Strategy Proposal • The mean Y, school performance, under the more deeply individualized ATS obtained via Q-learning is estimated to be 3.72. • As expected, this is larger than the value of the ATS which started with BMOD and augmented with MED for non-responders (mean = 3.51) • Recall (BMOD, MED) was the ATS with the largest mean among the 4 embedded ATSs. This Q-learning analysis was done with simulated/altered data.

  41. Citations to Technical Reports • Nahum-Shani, I., Qian, M., Almirall, D., Pelham, W. E., Gnagy, B., Fabiano, G., Waxmonsky, J., et al. (2010). Q-Learning: A data analysis method for constructing adaptive interventions . Technical Report, The Methodology Center, Penn State University. • Nahum-Shani, I., Qian, M., Almirall, D., Pelham, W. E., Gnagy, B., Fabiano, G., Waxmonsky, J., Yu, J., & Murphy, S. (2010). Experimental design and primary data analysis for informing sequential decision making processes. Technical Report, The Methodology Center, Penn State University.

  42.  Practicum • Autism Exercises: As before, we will go through the Autism Starter File to continue practicing/working through these primary data analyses using the Autism data set.

  43.  Extra Slides: For Statistical Aficionados! In the next two slides, we actually do Steps (ii) and (iii) of Q-Learning by hand. This is what the PROC QLEARN software automatically does. The issue, however, is that the standard errors here are incorrect. PROC QLEARN calculates the appropriate standard errors.

  44. SAS code for Step (ii) of Q-Learning: Assign optimal outcome based on Step (i) data dat11; set dat1; * first, everyone gets their observed outcome; yhat = y; * second, re-assign the outcome for non-responders; if R=0 then yhat = 3.0039 - 0.2462*o11c - 0.2961*o12c + 0.0391*o13c + 0.4868*o14c + 0.0758*a1 - 0.0097*o21c - 0.0980*o22 + abs(-0.8640*a2 - 0.1934*a1*a2 + 1.1826*o22*a2) ; run; proc means data=dat11; var y yhat; run; The MEANS Procedure Variable N Mean Std Dev ------------------------------------------------- Y 150 2.9533333 1.2814456 yhat 150 3.4107823 0.9385790 ------------------------------------------------- This analysis is with simulated data.

  45. SAS code for Step (iii) of Q-Learning: First-stage treatment tailoring * Step (iii) regression using the new outcome; proc genmod data=dat11; model yhat = o11c o12c o14c o13 a1*o13 a1; estimate 'BMOD vs MED given MED prior yr ' a1*o13 2 a1 2; estimate 'BMOD vs MED given NO MED prior yr' a1*o13 0 a1 2; run; * medication in the year prior appears to be a tailoring variable ; * however, statistical inferences (p-values, confidence intervals) ; * should not be based on this output. ; Contrast Estimate Results Naïve 95% Conf Limits Label Estimate Lower Upper P-value BMOD vs MED given MED prior yr -0.5012 -0.9423 -0.0601 0.0259 BMOD vs MED given MED prior yr 0.6208 0.3266 0.9150 <0.001 This analysis is with simulated data.

More Related