290 likes | 468 Views
Assessing Change in Clinical Practice. Richard Bost, Ph.D., ABPP Frances Wen, Ph.D. Michael Basso, Ph.D. Oklahoma Psychological Association 11/6/04. Key issue…. How does one demonstrate statistically significant change in an individual patient?
E N D
Assessing Change in Clinical Practice Richard Bost, Ph.D., ABPP Frances Wen, Ph.D. Michael Basso, Ph.D Oklahoma Psychological Association 11/6/04
Key issue… How does one demonstrate statistically significant change in an individual patient? • To document the clinical utility of an intervention • To document spontaneous recovery or deterioration
Presenters Richard H. Bost, Ph.D., ABPP, BCIAC • Assoc Prof Behavioral Science at OSU-COM • Board certified in Rehabilitation Psychology (ABPP) and in Biofeedback (BCIAC) • Clinical experience in neurological impairment and physical health problems • Relative newcomer to research in computerized interventions in ABI
Presenters Frances Wen, Ph.D. • Asst Prof Family Medicine at OSU-COM • Focus on consulting with physicians on research regarding clinical health questions, grant writing • Clinical experience in physical and mental health problems • Researches physician competencies, assessment in medical education, availability/utilization of behavioral and health care services in rural areas
Presenters Michael Basso, Ph.D. • Assoc Prof of Psychology, Director of Clinical Training at TU • Teaches neuropsychology, cognitive psychology, and tests and measures • Researches cognitive function in psychiatric impairment, immune disorders, and MS
Organization of Presentation • Independent but coordinated presentations: • Basic concepts (Bost) • Statistical foundations (Basso) • Application to emotional/behavioral medicine measures (Wen) • Application to cognitive measures (Bost) • Some content overlap; repetition of key points • Some review of familiar concepts; new application • Today’s slides and handouts are available at http://www.healthsciences.okstate.edu/bost.html
Organization (con’t) Software availability: • RCI Generator 2.0 (Devily, 2004)http://www.swin.edu.au/victims/resources/software/reliablechange/reliable_change_generator.html • SRB [Bivreg] (Crawford & Howell, 1998)http://www.abdn.ac.uk/~psy086/dept/Journal_urls_redirection.htm • Multiple Regression [Multreg] (Crawford & Howell, 1998)http://www.abdn.ac.uk/~psy086/dept/Journal_urls_redirection.htm • SRB Data Prep (Bost, 2004)http://www.healthsciences.okstate.edu/bost.html
Key task Distinguish error in measurement from clinically meaningful change • Specify random variation • Identify & minimize effects of bias
Common Sources of Variance in Serial Assessment • Bias – → systematic change • Extraneous events (e.g. 9/11 ↑ anxiety) • Practice effects • Memory for content • Procedural learning • Familiarity with examiner & testing context • Performance anxiety
Common Sources of Variance in Serial Assessment Bias (con’t) • Demographics • Pre-test abilities • Age • Education • Gender • Ethnicity
Common Sources of Variance in Serial Assessment • Error – sources of random change; statistical error • Measurement error (SEM) • Regression to the Mean (Seest)
Guidelines for Selecting Tests • Determine examination goals • Determine type of data generated by each test considered: nominal, ordinal, interval, ratio. • Only interval and ratio can be used in RCI and SRB analyses • Appraise validity • Appraise reliability • Consider reading ability, time, costs
Change Assessment Strategies • Alternate forms • Reliable Change Index (RCI) • Adjusted RCI • Standardized Regression-Based change (SRB) • Multiple Regression
Application to Cognitive Measures • Must have raw data from norm sample to use regression analysis • This info usually not in test manuals • Many clinicians will have this info in their clinical files
Application to Cognitive Measures (con’t) • Guidelines for using raw data from clinical files: • Specify norm sample characteristics • Pull T1 and T2 test scores for 30+ patients • Apply to “SRB Data Prep” spreadsheet (demo spreadsheet)
On-line Cognitive Measure Headminder Cognitive Stability Index (CSI) • Designed for serial measurement to document change over time • 10 subtests • Administration time—25-30 minutes • Four factors: • Processing speed • Response speed • Memory/learning • Attention
On-line Cognitive Measure (con’t) Headminder Cognitive Stability Index (CSI) • Displays results in standard scores • Displays RCIp with repeated administrations • Available at http://www.headminder.com (demo CSI)