Using systematic reviews to inform practice

Using systematic reviews to inform practice Statewide School-based OT/PT Conference October 28, 2005 Steven M. Cope, Sc.D., OT

Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values Sackett et al. (2000)

An evidence-based occupational therapy practice uses research evidence together with clinical knowledge and reasoning to make decisions about interventions that are effective for a specific client(s) Law & Baum (1998)

Although there is an undeniable art to pediatric physical therapy, the heart of our practice should be the scientific basis of our interventions. The challenge is to integrate art and science in making clinical decisions that allow us to provide our patients and families with optimal care. M.J. Barry

Evidence-based practice is… • based on a single patient • based on clinical judgement and patient values (client-centered) • guided by research rather than dictated by it

Being an evidence-based practitioner means… • You value the importance of scientific literature as a foundation for clinical decision making; • You frequently ask yourself, “what evidence exists in the literature to support or refute the intervention I am about to provide?” • You believe your interventions will be more effective when research evidence is integrated

Assumptions of EBP • Scientific research provides an important basis for verifying the effectiveness of our interventions • The effectiveness of interventions are improved under an evidence-based approach • Life-long learning is important to effective clinical practice

Diagnostic and technical skills increase with experience; however, clinical effectiveness deteriorates with time unless current knowledge is used to modify practice patterns

Barriers to EBP • Time for individual study and group discussion (lack of) • Access to scientific literature (lack of) • Interpretation of published findings (difficult to understand) • Attitudes towards EBP may be negative

Evidence-based practice: Getting started • Pose a researchable question • Search literature for best evidence • Do critical appraisal of study’s validity • Integrate evidence into clinical decision • Evaluate clinical effectiveness

Pose a Researchable Question • Identify clinical problem • Example: Children with cerebral palsy have spasticity and underlying muscle weakness leading to functional movement difficulties • Identify intervention(s) -Strength training • Relative outcomes -strength, spasticity, and functional movement • Patient characteristics -children with spastic CP

The Research Question • Is strength training [intervention] effective at increasing strength and functional movement [relevant outcomes] in children with spastic CP [patient characteristics]?

Search Literature for “Best Evidence”:What does this mean? • Research on subjects whose characteristics are similar to your patient’s; • Research on interventions that match the one you want to provide; • Research on outcomes that are of interest and apply to both you and the patient; • Research that is credible and believable;

Critical Appraisal • Design (I-V) • Sample Size (A-C) • Internal Validity (1-3) • External Validity (a-c)

Design

Sample Size • A: n ≥ 20 per group • B: n < 20 per group

Internal Validity • 1: High internal validity • No alternate explanation for outcome • 2: Moderate internal validity • Attempt to control for lack of randomization biases • 3: Low internal validity • Two or more serious alternative explanations for outcome

Threats to Internal Validity • Hawthorne effect • Maturation • Testing effect • Experimenter bias • Co-intervention effects • Errors with data • No Randomization • Attrition

External Validity • a: High external validity • Participants represent population, and treatments represent current practice • b: Moderate external validity • Between high and low • c: Low external validity • Heterogeneous sample without being able to understand whether effects were similar for all diagnoses or treatments do not represent current practice

Critical Appraisal • Strongest evidence • IA1a • Weakest evidence • VB3b

Integrate Evidence into Clinical Decision • The evidence reviewed should help you decide: 1) do I start an intervention I’m not currently using; 2) do I stop doing an intervention I am currently doing? 3) continue? • Was the evidence you found best evidence? • Does the evidence support or refute the intervention you want to provide? For which outcomes? Which patients responded well? • How much evidence exists? • How much evidence is needed to make this decision?

Evaluate Clinical Effectiveness • Reflection/assessment after performing the intervention • Did your patient experience a positive change?

Systematic Review: What is it? • A summary of several research articles on one topic presented in table and narrative formats • Qualitative rather than quantitative analysis • Subjective interpretation • Time consuming process • Strongest form of evidence

Systematic Review #1:Research Question • What is the effectiveness of sensori-motor and motor learning handwriting interventions on improving writing legibility and speed in children with handwriting difficulties?

Search strategies • Key terms (handwriting, writing, written communication, treatment, OT, intervention, …) • Data bases (PubMed…) • Entrance criteria (level of evidence, diagnosis, age, years since publication…)

Critical Appraisal • Two or more people read articles independently and reported relevant information on a form for discussion • Group discussion to develop consensus on key decisions (journal club) • Use AACPDM methodology to present results (table and narrative format)

Results • Sixteen studies located • Eight eliminated from review because they didn’t meet entrance criteria • Eight studies selected for review

Systematic Review of Handwriting Interventions

Table 2

Systematic Review Discussion • 4 studies used a combination of sensorimotor and motor learning-based intervention; however, in these studies, emphasis was on sensorimotor-based intervention; • 1 study used only sensorimotor intervention; • 2 studies used only motor learning (practice); • 2 studies compared sensorimotor with motor learning (practice);

Discussion continued • Children in all the studies ranged from 5-11 years of age and were WNL for cognitive function; • All the children receiving intervention were identified as having handwriting difficulties; • Interventions ranged from 3 hours to 30 hours and from 6 days to 7 months. Most sessions were 30 minutes long;

Discussion continued • 3 articles were randomized controlled trials and represented strong evidence; the remaining articles were either non-randomized or lacked control groups; • 4 of 9 of the studies showed significant improvement for handwriting and an additional 3 demonstrated trends toward improvement; • Studies that looked at quality of handwriting legibility showed that children achieved modest benefits; • Studies that looked at handwriting speed showed no change in this variable; • Long term effects were not studied; • Outcomes measured primarily focused on activity level (handwriting), but all levels were represented;

Clinical Bottom Line • We do know that handwriting is an area that can improve with intervention; • The evidence to date shows some benefit of intervention, however, the studies have not determined what the benefits from these interventions are long-term. Future studies should examine the relative effectiveness of sensorimotor and motor learning approaches and the long-term effects of each intervention

Clinical Bottom Line • This systematic review of handwriting interventions reveals a relatively small number of studies, only a few with rigorous designs; • After reviewing the scientific evidence we are still not sure of the best method for improving handwriting. More studies need to be done that isolate sensorimotor and motor learning interventions to determine if one approach is more effective than the other.

Credits • This review was completed in May 2004 by Rula LaLicata, Stephanie Beilke, Mary Lassanske, Lisa Villardita, Nicole Rosalez, and Steve Cope; it was updated in May 2005 by Cathy Payne and Steve Cope

Systematic Review #2Research Question • What do we know about the effectiveness of strengthening exercises for children with spastic cerebral palsy?

Focused Research Questions • What is the effect of strengthening exercises on force production in children with spastic CP? • Are there adverse effects to strengthening spastic muscles? • What is the effect of strengthening exercises on functional outcome in children with spastic CP?

Method of Review • Literature search was performed using on-line databases: Pubmed, PEDro, CINAHL; • Each study selected for review was read and critically appraised by two people; • Reviewers collaborated on classification of the article in terms of threats to internal validity, level of evidence rating, statistical significance, and clinical importance;

Method of Review cont. • Information and findings from all articles were summarized in table and narrative format according to AACPDM methodology

Results • The literature search yielded 15 articles specifically addressing strengthening exercise and spastic CP; • 6 of the 15 articles were eliminated because they did not meet entrance criteria; • 9 articles were reviewed

Table 3 Table 3 Summary of Studies

Abbreviations for Table • EEI – Energy Expenditure Index • LSUT – Lateral Step-Up Test • MAS-STS – Motor Assessment Scale – Sit to Stand • GMFM – Gross Motor Function Measure : • SPPA – Self-Perception Profile for Adolescent • 1 Fowler, et.al. (2001) • 2 Damiano, et.al. (1995) • 3 Eagleton, et.al. (2004) • 4 Blundell, et.al. (2003) • 5 MacPhail & Kramer, (1995) • 6 Damiano & Abel, (1998) Colored = no clinical importance • 7 O’Connell & Barnhart, (1995) Italics = small clinical importance • 8 Dodd, et.al. (2003) Regular font = clinically important • 9 Darrah, et.al. (1999)

Abbreviations cont. Level of Evidence I = Randomized controlled trial III = One group pre-post test Ratings of Internal Validity S = Strong, M = Moderate, W = Weak

What is the effect of strengthening exercises on force production in children with spastic cerebral palsy? • Evidence from the studies reviewed showed that for the majority of studies, children, and muscles studied, children with CP did improve their strength, primarily in their lower extremities

Are there adverse effects to strengthening spastic muscles? • Only one study specifically examined the effects of muscle strengthening on spasticity, and it provided evidence that spasticity was not increased

Using systematic reviews to inform practice