1 / 28

Assessing Change in Clinical Practice

Assessing Change in Clinical Practice. Richard Bost, Ph.D., ABPP Frances Wen, Ph.D. Michael Basso, Ph.D. Oklahoma Psychological Association 11/6/04. Key issue…. How does one demonstrate statistically significant change in an individual patient?

leila-marsh
Download Presentation

Assessing Change in Clinical Practice

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessing Change in Clinical Practice Richard Bost, Ph.D., ABPP Frances Wen, Ph.D. Michael Basso, Ph.D Oklahoma Psychological Association 11/6/04

  2. Key issue… How does one demonstrate statistically significant change in an individual patient? • To document the clinical utility of an intervention • To document spontaneous recovery or deterioration

  3. Compare Groups

  4. Compare Individual from Group

  5. Compare New Individualto Group

  6. Presenters Richard H. Bost, Ph.D., ABPP, BCIAC • Assoc Prof Behavioral Science at OSU-COM • Board certified in Rehabilitation Psychology (ABPP) and in Biofeedback (BCIAC) • Clinical experience in neurological impairment and physical health problems • Relative newcomer to research in computerized interventions in ABI

  7. Presenters Frances Wen, Ph.D. • Asst Prof Family Medicine at OSU-COM • Focus on consulting with physicians on research regarding clinical health questions, grant writing • Clinical experience in physical and mental health problems • Researches physician competencies, assessment in medical education, availability/utilization of behavioral and health care services in rural areas

  8. Presenters Michael Basso, Ph.D. • Assoc Prof of Psychology, Director of Clinical Training at TU • Teaches neuropsychology, cognitive psychology, and tests and measures • Researches cognitive function in psychiatric impairment, immune disorders, and MS

  9. Organization of Presentation • Independent but coordinated presentations: • Basic concepts (Bost) • Statistical foundations (Basso) • Application to emotional/behavioral medicine measures (Wen) • Application to cognitive measures (Bost) • Some content overlap; repetition of key points • Some review of familiar concepts; new application • Today’s slides and handouts are available at http://www.healthsciences.okstate.edu/bost.html

  10. Organization (con’t) Software availability: • RCI Generator 2.0 (Devily, 2004)http://www.swin.edu.au/victims/resources/software/reliablechange/reliable_change_generator.html • SRB [Bivreg] (Crawford & Howell, 1998)http://www.abdn.ac.uk/~psy086/dept/Journal_urls_redirection.htm • Multiple Regression [Multreg] (Crawford & Howell, 1998)http://www.abdn.ac.uk/~psy086/dept/Journal_urls_redirection.htm • SRB Data Prep (Bost, 2004)http://www.healthsciences.okstate.edu/bost.html

  11. Key task Distinguish error in measurement from clinically meaningful change • Specify random variation • Identify & minimize effects of bias

  12. Common Sources of Variance in Serial Assessment • Bias – → systematic change • Extraneous events (e.g. 9/11 ↑ anxiety) • Practice effects • Memory for content • Procedural learning • Familiarity with examiner & testing context • Performance anxiety

  13. Common Sources of Variance in Serial Assessment Bias (con’t) • Demographics • Pre-test abilities • Age • Education • Gender • Ethnicity

  14. Common Sources of Variance in Serial Assessment • Error – sources of random change; statistical error • Measurement error (SEM) • Regression to the Mean (Seest)

  15. Guidelines for Selecting Tests • Determine examination goals • Determine type of data generated by each test considered: nominal, ordinal, interval, ratio. • Only interval and ratio can be used in RCI and SRB analyses • Appraise validity • Appraise reliability • Consider reading ability, time, costs

  16. Change Assessment Strategies • Alternate forms • Reliable Change Index (RCI) • Adjusted RCI • Standardized Regression-Based change (SRB) • Multiple Regression

  17. Standard Error of Difference (SEdiff)

  18. Standard Error of Difference (SEdiff)

  19. Standard Error of Difference (SEdiff)

  20. Standard Error of Difference (SEdiff)

  21. Standard Error of Prediction (SEpred)

  22. Standard Error of Prediction (SEpred)

  23. Standard Error of Prediction (SEpred)

  24. Standard Error of Prediction (SEpred)

  25. Application to Cognitive Measures • Must have raw data from norm sample to use regression analysis • This info usually not in test manuals • Many clinicians will have this info in their clinical files

  26. Application to Cognitive Measures (con’t) • Guidelines for using raw data from clinical files: • Specify norm sample characteristics • Pull T1 and T2 test scores for 30+ patients • Apply to “SRB Data Prep” spreadsheet (demo spreadsheet)

  27. On-line Cognitive Measure Headminder Cognitive Stability Index (CSI) • Designed for serial measurement to document change over time • 10 subtests • Administration time—25-30 minutes • Four factors: • Processing speed • Response speed • Memory/learning • Attention

  28. On-line Cognitive Measure (con’t) Headminder Cognitive Stability Index (CSI) • Displays results in standard scores • Displays RCIp with repeated administrations • Available at http://www.headminder.com (demo CSI)

More Related