1 / 39

Using Data to Improve Treatment: Research and Practice

2.

issac
Download Presentation

Using Data to Improve Treatment: Research and Practice

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. 1 Ben Ogles Ohio University Using Data to Improve Treatment: Research and Practice

    2. 2 “If providers don’t measure outcomes, someone else will.” - M. Naditch (1994)

    3. 3 Overview Outcomes and Accountability Outcome Research with Youth Patient Oriented Research Feedback Using the Ohio Scales for Youth

    4. 4 OUTCOMES! A Central Focus over the last 15 years Behavioral health, education, healthcare Books, articles, task forces Public and private Administrators, providers, consumers, and payers

    5. 5 AN EXAMPLE FROM HEALTH CARE “Programs focused on outcomes research, measurement, and management have emerged over the past decade as a result of rapidly rising healthcare costs, questions about the effectiveness of medical intervention, and the need for efficient delivery of care (Mowinski & Staggers, 1997).” -Maloney & Chaiken, 1999

    6. 6 What does the Outcome Research with Children Tell Us?

    7. 7 Types of Research Efficacy – Does therapy work under tightly controlled experimental conditions? (RCT) Effectiveness – Does the treatment work in practice? (mental health services research & program evaluation) Patient Oriented – Is treatment working for this client? (Howard et al. 1996)

    8. 8 Broad Meta-analyses

    9. 9 More Narrow Meta-analyses

    10. 10 Specific Meta-analyses

    11. 11 General Findings from Meta-analysis – What can we expect? Treatment for children works Treatment effects are stable over time Different types of treatment are relatively equivalent (Miller, Wampold, & Varhely, 2008) Trained therapists are effective with a bigger range of clients Evidence based treatments exceed treatment as usual (Weisz, Doss, Hawley, 2006)

    12. 12 A Big Problem in Child Outcome Research Most “lab-based” studies are not representative of clinical practice Recruiting and Homogeneous samples Concentrated training of therapists Adherence to treatment Structured, manual guided treatment Length of treatment “Studies were particularly weak in clinical representativeness of their samples, therapists, and settings, suggesting a need for increased emphasis on external validity in youth treatment research.” (Weisz, Doss, & Hawley, 2005)

    13. 13 Effectiveness Initial evidence suggested child treatment in clinical settings was not effective (mean es = .01 as opposed to .77 in research settings; Weisz, Donenberg, Han, & Weiss, 1995) Recent evidence suggests treatment in clinical settings can produce similar results to laboratory settings (Hunsley & Lee, 2007)

    14. 14 From Program Evaluation to Quality Improvement Program evaluation/psychotherapy research is important for demonstrating the benefits of treatment in controlled situations Program evaluation data can be integrated into clinical practice in order to move beyond the typical aggregate retrospective benefits to patient oriented research.

    15. 15 Feedback through Patient Oriented Research Feedback - "information about work behavior and task performance that is relatively factual and incontrovertible" (Kopelman, 1986). Simple and Effective organizational (Balcazar, Hopkins, & Suarez, 1985), educational (Fleming & Sulzer-Azaroff, 1989), medical (Geary, Hovell, & Black, 1985), and mental health (Shook, Johnson, & Uhlman, 1978). broadly effective (Kluger & DeNisi, 1996) (From Melendez, 2002)

    16. 16 Example in Education Classroom intervention Teachers Focused on student academic performance. Teachers were provided performance feedback when treatment integrity scores fell below 70% and student academic performance fell below 80%. Teacher performance, as measured by student academic performance, improved immediately and was maintained. (Mortensen & Witt, 1998; as cited in Melendez, 2002)

    17. 17 Examples in Mental Health Lambert et al. studies Clients randomly assigned to groups Therapist receive feedback Signal alarm cases have greatest effect .39 “simply knowing was enough” Meta-analysis of 30 studies (Sapyta, Riemer, & Bickman, 2005) Overall .21 effect size (better than 58%) Especially useful with “flagged” cases es =.31

    18. 18 Example in Ohio Hamilton County Consumer Quality Review Teams Use of the Ohio Scales in treatment was significantly related to Youth perception of progress Parent perception of progress Parent satisfaction with treatment (Stewart, 2006)

    19. 19 ODMH Youth Outcomes Ohio Scales (Parent) Ohio Scales (Youth – ages 12-18) Ohio Scales (Agency Worker)

    20. 20 Use of Data for Initial Assessment and Treatment Planning Development of Treatment Plan Critical items Identify target problems Identify functional strengths (resilience) Charting initial scores – level of severity Comparison of parent and child All can be used to engage the family during the in person contact

    21. 21 Training for Clinical Use of Outcome Data Streaming Video available for Training on line at: http://www.scchildren.com/ohio-scales.php DVD also available – includes PDFs, case examples, handouts, powerpoints, etc.

    22. 22 Critical Items Parent or youth report of - #7 drug or alcohol use #8 breaking the law #12 hurting self #13 talking or thinking about death

    23. 23 Target Problems Identify 3 or 4 problems rated the highest by the parent and youth. This may help to focus an interview or reveal nuances of consumer perceptions following an interview Useful for initial treatment planning

    24. 24 Functional Strengths Identify functioning items rated as 3 or 4 Identify problem items rated 0 May help to locate strengths that can facilitate change. Useful for initial treatment planning Can incorporate resilience research

    25. 25 Initial Severity Chart scores using figures in the User’s Manual (reproduce as needed). Clinical cutoff Problem Severity 20 Functioning 51 (60 for youth) Internalizing/Externalizing Pattern

    26. 26 Level of Care via Calibration

    27. 27 Compare Sources Compare parent and youth Internalizing – youth tend to rate higher Externalizing – parents tend to rate higher Compare parent and youth with agency worker Situational issues (court referral)

    28. 28 Factors associated with Agreement Gender (better total agreement with daughters than sons; more item agreement with sons - d2) Rater Relationship (parents>others) Ethnicity (Hispanic>Caucasian, multi-ethnic>African American) Area of assessment (internalizing > externalizing & conduct ) Diagnosis (internalizing > externalizing; reverse at item level; d2) (Carlston, 2003)

    29. 29 Discrepancy and Change Bigger discrepancy -> increased dropout Bigger discrepancy (with parent rating higher) -> poorer outcome (Carlston, 2003)

    30. 30 Tracking Change Items Total scores Compare scales (problems/functioning) Clinical significance Recovery Improvement

    31. 31 Individual Items Can focus on target problems Simulates the “Target Complaints” measures used in many outcome studies Example using the TDCRP study

    32. 32 Total Scores Primary use of the scales to track individual change using the total scores Plot total problem severity and functioning over time to monitor progress

    33. 33 Clinical Significance Improvement (amount of change) Recovery (end point of change) Both - Clinical Significance (Jacobson & Truax, 1991) See recently released ODMH reports #16 & #18: http://www.mh.state.oh.us/oper/outcomes/reports.quarterly.html

    34. 34 Graphic Representation

    35. 35 Using of Data to Improve Quality Individual clinicians develop tracking protocols for use with each client Clinical Supervision Aggregate data for management, program improvement, and marketing Outcome implementation provides opportunities to move beyond measuring to monitoring or even managing the quality of mental health services.

    36. 36 Example – Provider profiling

    37. 37 Summary Youth treatments work Outcome assessment methods can be integrated in to clinical routines for: Treatment planning Outcome monitoring Quality improvement Patient oriented research demonstrates that feedback can improve practice

    38. 38 Additional Resources My Web Site http://oak.cats.ohiou.edu/~ogles/ My Book Ogles, B., Lambert, M., & Fields, S. (2002). Essentials of Outcome Assessment. New York: John Wiley. Ohio Dept. of Mental Health Web Site http://www.mh.state.oh.us/oper/outcomes/outcomes.index.html

    39. 39 Additional Resources Ogles, B. M., Dowell, K., Hatfield, D, Melendez, G., & Carlston, D. (2004). The Ohio Scales. In M. E. Maruish (Ed.), The use of psychological testing for treatment planning and outcome assessment (3rd ed., Vol. 2) (pp. 275-304). Hillsdale, New Jersey: Lawrence Earlbaum. Ogles, B. M., Melendez, G., Davis, D. C., & Lunnen, K. M. (2001). The Ohio Scales: Practical Outcome Assessment. Journal of Child and Family Studies, 10, 199-212.

    40. 40 Thanks To Office of Program Evaluation & Research, Ohio Department of Mental Health Southern Consortium for Children Participating agencies Participating families Multiple graduate students

More Related