1.64k likes | 1.82k Views
F Focus on the O Outcomes of C Communication U Under S Six. Adjusting our FOCUS! Measuring Meaningful Clinical Outcomes. Team FOCUS. Prof. Nancy Thomas-Stonell, PI Dr. Bruce Oddson, Co-PI Dr. Peter Rosenbaum, Co-PI
E N D
F Focus on the O Outcomes of C Communication U Under S Six Adjusting our FOCUS!Measuring Meaningful Clinical Outcomes Paediatric Symposium '11 Building a Brighter Future
Team FOCUS • Prof. Nancy Thomas-Stonell, PI • Dr. Bruce Oddson, Co-PI • Dr. Peter Rosenbaum, Co-PI • Dr. Karla Washington, Post-Doctoral Fellow • Ms. Bernadette Robertson, Research Coordinator • Ms. Joan Walker, Research Assistant Funding SickKids Foundation Canadian Institutes of Health Research (CIHR) Bloorview Childrens Hospital Foundation Paediatric Symposium '11 Building a Brighter Future
Eleven Research Partners in Five Provinces Across Canada • Eastern Healthcare, St. Johns, NL • Nova Scotia Hearing and Speech Centres, NS • Beyond Words, Preschool Speech and Language Program, York Region, ON • Waterloo Preschool Speech and Language Program, ON • Holland Bloorview Kids Rehabilitation Hospital, ON • Hamilton Preschool Speech and Language Service, ON • Technology Access Clinic, ON • Wellington-Dufferin Guelph Region Preschool Sp & Lang Service, ON • ErinoakKids Centre for Treatment and Development & Halton-Peel Preschool Speech and Language Program, ON • Calgary Health Region, AB • BC Centre for Ability, BC Paediatric Symposium '11 Building a Brighter Future
A Typical Day in the Life of a SLP! Example prepared with thanks to Laurie Graham Paediatric Symposium '11 Building a Brighter Future
Preschool Speech-Language Pathologist 4 year old boy, A.B., presents for service Formal assessment results (currently available standardized tests) indicate a moderate speech and language disorder Paediatric Symposium '11 Building a Brighter Future
Assessment reveals… • Has a lot of trouble with sounds • i.e. ‘ish’ instead of ‘fish’; ‘ • og’ instead of ‘frog’ • Has trouble with pronouns • i.e. often says ‘he’ instead of ‘she’ Paediatric Symposium '11 Building a Brighter Future
Parents reports the boy is: extremely frustrated, has tantrums teased at school kids and teacher have trouble understanding him shy, evidence of low self-esteem Paediatric Symposium '11 Building a Brighter Future
Clinical Goals for 8 week treatment block A.B. will produce /f/ in word initial position in structured settings 80% of the time. A.B. will produce ‘she’ appropriately in phrases in structured settings 80% of the time. Speech-Language Pathologist documents parent comments in client file. Paediatric Symposium '11 Building a Brighter Future
Eight weeks later… During the last session, the Speech-Language Pathologist decides her goals have not been met as the child is only performing tasks at a 50% success rate. The parents state that tantrums have decreased, A.B. is no longer being teased at school, and seems more confident in communicating. Paediatric Symposium '11 Building a Brighter Future
The importance of parent comments The Speech-Language Pathologist has chosen to include many parent comments in her client notes because they give her an indication of the child’s interactions with others. Paediatric Symposium '11 Building a Brighter Future
The importance of parent comments • Interaction is fundamental to the development of communication - the more you interact, the more you practice communication skills. • The ability to communicate with peers and teachers is fundamental to academic and social success (i.e. group work). Paediatric Symposium '11 Building a Brighter Future
Parent comments continued.. Including parent comments in client notes, although recommended by regulatory bodies of the profession, is not required. There are no valid and reliable measures (i.e. tests/questionnaires) that capture the real-world changes observed by parents and clinicians! Paediatric Symposium '11 Building a Brighter Future
Outcome measurement for A.B. The Speech-Language Pathologist wishes she could more thoroughly document the behavioural, interactive and social changes seen by A.B.’s parent. She suspects that A.B.’s gains are associated with therapy (not just normal development) but there is no way to prove her hypothesis. Paediatric Symposium '11 Building a Brighter Future
Decision time: There are other children on the waitlist. Given the limited funding available, A.B. is not offered a second block of therapy. Paediatric Symposium '11 Building a Brighter Future
Increasing concern Clinicians, researchers and disability advocates are concerned that changes which may be important results of therapy are overlooked. Paediatric Symposium '11 Building a Brighter Future
What are clinically meaningful outcomes? Paediatric Symposium '11 Building a Brighter Future
World Health Organization (WHO) Health Frameworks • In 1980 WHO (1980) came out with their first health framework the International Classification of Impairments, Disabilities and Handicaps (ICIDH) • Impairment (what’s broken) • Disability (what can’t you do) • Handicap (limitations in the real world) Impairment Disability Handicap Paediatric Symposium '11 Building a Brighter Future
International Classification of Functioning, Disability and Health (ICF-2003)Children & Youth Version (ICF-CY - 2007) Health Condition Body Functions & Structures Activities Participation Environmental Factors Personal Factors Paediatric Symposium '11 Building a Brighter Future
ICF & ICF-CY Domains • Body Functions: Physiological • (e.g., voice, oral motor, speech production) • Body Structures: anatomical • (e.g., hearing loss; CL/P) Paediatric Symposium '11 Building a Brighter Future
ICF & ICF-CY Domains • Activities: • Tasks and actions by an individual. • ‘Capacity’ - performance of a task in a standard environment. • Participation: • Involvement in a life situation. • ‘Performance’ performance of tasks in a in the current environment. Paediatric Symposium '11 Building a Brighter Future
Contextual Factors • Environmental Factors: • external influences on functioning and disability related to physical, social and attitudinal world. (stairs, culture, support system) • Personal Factors: • internal influences on functioning and disability (personality influences on coping style) Paediatric Symposium '11 Building a Brighter Future
ICF Health Framework • Has positive and negative components. • Uses a bi-directional model. • Doesn’t take developmental stages into account. • e.g. temper tantrums/frustrations for 2 year olds Paediatric Symposium '11 Building a Brighter Future
ICF-CY Health Framework • New codes to capture the functional characteristics of a developing child. Expanded codes include: • Learning new skills • Play • Adaptability • Persistence • Exploration Paediatric Symposium '11 Building a Brighter Future
Why use ICF-CY model? • Outcomes need to be evaluated across ICF-CY domains. • Several studies (Sarno, 1969, Aten, 1986) have noted poor correlation between body structure/function outcomes and activity and participation outcomes • depends not only on skill levels, but also personality, coping skills, social support systems... Paediatric Symposium '11 Building a Brighter Future
How do we measure these outcomes? Paediatric Symposium '11 Building a Brighter Future
We need treatment outcome measures! We need outcomes measures to evaluate the impact of treatment on children’s lives. Paediatric Symposium '11 Building a Brighter Future
Outcomes vs. Outcome Measures • Any consequence of healthcare is an outcome! Outcome = environment + treatment + client + severity Paediatric Symposium '11 Building a Brighter Future
There are many types of outcomes. • Avoiding adverse affects of care (nobody dies) • Improved physiologic status ( phonation time) • Reduction in symptoms ( dysfluencies) • Improved functional status (use telephone) • Minimizing costs • Minimizing length of care Paediatric Symposium '11 Building a Brighter Future
Outcome Measure • A treatment outcome measure is a validated test designed to measure change in function. • It measures, in quantitative terms, the impact of routinely delivered care on client’s lives. Paediatric Symposium '11 Building a Brighter Future
Treatment Outcome Measures need to be proven to work! • Garbage in – garbage out! • Information generated by outcome studies is only useful if the measure is clinically useful and scientifically sound (van der Putten et al., 1999). Paediatric Symposium '11 Building a Brighter Future
Why can’t we use our standardized tests? Paediatric Symposium '11 Building a Brighter Future
Standardized tests... • Determine the presence or absence of a communication disorder. They do not change. • They provide too little information (insufficient number and variety of items) to monitor progress. Huang, Hopkins & Nippold (1997). Satisfaction with Standardized Language Testing: A survey of Speech-Language Pathologists. Language Speech & Hearing Services in Schools 28, 12-29. Paediatric Symposium '11 Building a Brighter Future
Treatment Outcome measures • Outcomes measures at a minimum need to be proven to reliably distinguish between children who improve from therapy and those who do not improve. Paediatric Symposium '11 Building a Brighter Future
Why use treatment outcome measures? • To improve treatment services in an evidence-based manner. • To measure clinically important change. • To determine optimal length for treatment. • To select the best treatment approach for each child. Paediatric Symposium '11 Building a Brighter Future
CASLPA Position Statement on Outcome Measures CASLPA encourages and supports the use and development of outcome measures by speech-language pathologists and audiologists Outcome measures should be used to improve practice in an evidence-based manner in the best interests of clients. CASLPA Position Statement on Outcome Measures – May, 2010 (www.caslpa.ca) Paediatric Symposium '11 Building a Brighter Future ASHA Convention November 18-20 2010 35
The FOCUS journey began in 1998 • Holland Bloorview Kids Rehabilitation Hospital wanted an outcome measure for speech-language therapy that could be used across programs. • Diverse population • CP/CLP/ABI Paediatric Symposium '11 Building a Brighter Future
Began our search for a treatment outcome measure. • TOMS and AusTOMS are very broad measures of change. Scale has many descriptors. Hard to know what changed. • GAS (individualized and time consuming). • ASHA NOMS had no proven reliability or validity. We completed a two-year study to evaluate the NOMS. Results indicated poor sensitivity to change. Paediatric Symposium '11 Building a Brighter Future
What do we do now? • …with no existing valid, reliable and responsive communication outcome measure for preschool children available? Paediatric Symposium '11 Building a Brighter Future
Development of the ‘FOCUS’ • F ocus on the • O utcomes of • C ommunication • U nder • S ix • Thomas-Stonell, N., Oddson, B., Robertson, B. & Rosenbaum, P. Development of the FOCUS (Focus on the Outcomes of Communication Under Six), a communication outcome measure for preschool children. Developmental Medicine and Child Neurology: 2010, 52:47-53. ] Paediatric Symposium '11 Building a Brighter Future
Our Goal To develop a valid,reliable,responsive treatment outcome measure that captures ‘real world’ changes following speech and language treatment. Paediatric Symposium '11 Building a Brighter Future
Developing the FOCUS • In the previous outcome study, we collected data from parents of 210 preschool children receiving speech-language treatment and their clinicians. (Thomas-Stonell, Oddson, Robertson & Rosenbaum, Predicted and Observed Outcomes in Preschool Children Following Speech and Language Treatment: Parent and Clinician Perspectives. JCD 42 (2009) 29-42.) Paediatric Symposium '11 Building a Brighter Future
Developing the FOCUS • They were asked to describe the changes they observed in their child during/following therapy. • My child is now able to… • What other changes did you see? • Why is that important? Paediatric Symposium '11 Building a Brighter Future
Method: 6 Linked-Steps • Content analysis of our descriptive data to create FOCUS items. 2. Test the measure with clinicians and families. 3. Revise the measure using the parent and clinician feedback. 4. Test the revised measure again. 5. Revise measure a second time. 6. Test measure a third time to obtain preliminary reliability and validity data. Paediatric Symposium '11 Building a Brighter Future
Content Analysis • Content analysis is the “systematic, objective analysis of message characteristics” to make valid inferences from text. (Neuendorf, 2002) Paediatric Symposium '11 Building a Brighter Future
Content Analysis • Identify recurring categories of change and calculate percentages of occurrence for each category. • The recurring categories reflected the ICF-CY framework. Paediatric Symposium '11 Building a Brighter Future
Coding Comments Working Slides Paediatric Symposium '11 Building a Brighter Future
Developing the FOCUS • FOCUS is driven by DATA, • no preconceived ideas • FOCUS items were developed from categories cited by >10% of parents & clinicians. • Resulted in 200 items, reduced to 103. • Items used parents’ own wording. Paediatric Symposium '11 Building a Brighter Future
Sample Body Functions Item Development Paediatric Symposium '11 Building a Brighter Future
Item Development: Body Functions Parent Comment “Pronounces words much more clearly (specifically F sounds,L sounds when prompted - he still has work to do with L's)” Category/ICF-CY coding: Body Functions • Articulation Functions; b320 FOCUS Item “My child’s speech is clear.” Paediatric Symposium '11 Building a Brighter Future
Sample Activities/CapacityItem Development Paediatric Symposium '11 Building a Brighter Future