Methods for Estimating the Decision Rules in Dynamic Treatment Regimes

1. Methods for Estimating the Decision Rules in Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan IBC/ASC: July, 2004

2. Dynamic Treatment Regimes

3. Dynamic Treatment Regimes are individually tailored treatments, with treatment type and dosage changing with ongoing subject information. Mimic Clinical Practice. • Brooner et al. (2002) Treatment of Opioid Addiction • Breslin et al. (1999) Treatment of Alcohol Addiction • Prokaska et al. (2001) Treatment of Tobacco Addiction • Rush et al. (2003) Treatment of Depression

4. EXAMPLE: Treatment of alcohol dependency. Primary outcome is a summary of heavy drinking scores over time.

7. Examples of sequential multiple assignment randomized trials: • CATIE (2001) Treatment of Psychosis in Alzheimer’s Patients • CATIE (2001) Treatment of Psychosis in Schizophrenia • STAR*D (2003) Treatment of Depression • Thall et al. (2000) Treatment of Prostate Cancer

8. k Decisions Observations made prior to jth decision Action at jth decision Primary Outcome: for a known function f

9. A dynamic treatment regime is a vector of decision rules, one per decision If the regime is implemented then

10. Methods for Estimating Decision Rules

11. Three Methods for Estimating Decision Rules • Q-Learning (Watkins, 1989) • ---regression • A-Learning (Murphy, Robins, 2003) • ---regression on a mean zero space. • Weighting (Murphy, van der Laan & Robins, 2002) • ---weighted mean

12. One decision only! Data: is randomized with probability

13. Goal Choose to maximize:

14. Q-Learning Minimize

15. A-Learning Minimize

16. Weighting

17. Discussion

18. Discussion • Consistencyof Parameterization • ---problems for Q-Learning • Model Space • ---bias • ---variance

19. Q-Learning Minimize

20. Minimize

21. Discussion • Consistencyof Parameterization • ---problems for Q-Learning • Model Space • ---bias • ---variance

22. Points to keep in mind • The sequential multiple assignment randomized trial is a trial for developing powerful dynamic treatment regimes; it is not a confirmatory trial. • Focus on MSE recognizing that due to the high dimensionality of X, the model parameterization is likely incorrect.

23. Goal Given a restricted set of functional forms for the decision rules, say , find

24. Discussion • Mismatch in Goals • ---problems for Q-Learning & A-Learning

25. Suppose our sample is infinite. Then in general neither or is close to

26. Open Problems • How might we “guide” Q-Learning or A-Learning so as to more closely achieve our goal? • Dealing with high dimensional X-- feature extraction---feature selection.

27. This seminar can be found at: http://www.stat.lsa.umich.edu/~samurphy/seminars/ ibc_asc_0704.ppt My email address: samurphy@umich.edu