1 / 38

Mohammed TA, Omar Ph.D., PT , CLT Rehabilitation health Science CAMS-KSU

Measuring change: responsiveness or sensitivity. Mohammed TA, Omar Ph.D., PT , CLT Rehabilitation health Science CAMS-KSU. Objective. Discusses ways of establishing if an OM can measure change over time and how this property can be reported using descriptive and statistical methods.

pleong
Download Presentation

Mohammed TA, Omar Ph.D., PT , CLT Rehabilitation health Science CAMS-KSU

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring change: responsiveness or sensitivity Mohammed TA, Omar Ph.D., PT , CLT Rehabilitation health Science CAMS-KSU

  2. Objective • Discusses ways of establishing if an OM can measure change over time and how this property can be reported using descriptive and statistical methods. • Critically appraise evidence about the development and measurement proprieties of OMs • Discuss the consideration related to the application of evidence base to OMs

  3. Introduction Psychometric properties • Reliable • Valid • Responsiveness

  4. Responsiveness to change • Responsiveness ability of instrument to detect/measures meaningful or important change in a clinical statewithin patients over time, or contrast between groups which may be indicative of therapeutic effects. N.B 1- Normally considered as an element of validity N.B 2- Change my reflect natural recovery + response to intervention N.B 3- Responsiveness is not considered a generalizable property

  5. Study responsiveness Determine clinical outcome assessment

  6. Study responsiveness

  7. Reporting responsiveness There are different ways reported when the responsiveness of a measurement is being analyzed. They include: • Effect size (standardized mean difference • Minimal detectable change • Minimal clinically important difference • Receiver operator characteristic curve.

  8. Effect size (ES) “average changes” The magnitude of the differences between two mean, may be standardized by dividing this differences by the pooled standard deviation to compare effects measured by differences scales Magnitude of effect size scores • Small ≤ 0.4 • Moderate = 0.5 • Large > 0.8 ES is often displayed as a correlation coefficient, r

  9. Effect size (ES) “average changes” • Easy to calculate and understand • Used with ratio or interval level of measures • Determine extent of a relationship/difference between variables • Useful for comparing responsiveness of different health measures • Enables sample size calculations. • Facilitates comparison between scientific studies

  10. Statistical significance versus Effect size compute an effect size for each study, we will find the following: Formula: d = 2(t)/df Smith d = 0.50 Jones d = 0.50 The amount of change associated with the outcome is the same in both studies.

  11. Differences

  12. MDC (Minimal Detectable Change) • The MDC was calculated based on standard error of measurement (SEM) according to the following formulae: • MDC=1.96×2×SEM • SEM=SDall testing scores×(1−r) • The √2 is used to account for the underlying extra uncertainty during measurement in two time points. • The value 1.96 is the z score associated with the 95% confidence level, • The ris the coefficient of the test–retest reliability, which was estimated by ICC. • MDC% was calculated as (MDC/mean) × 100%, • An MDC% of <30% was considered as acceptable and <10% as excellent

  13. Now Start use and interpret ES, MDC I think I have given input to `u’

  14. Minimal important difference and responsiveness of 2-minute walk test performance in people with COPD undergoing pulmonary rehabilitation Int J Chron Obstruct Pulmon Dis. 2017; 12: 2849–2857.

  15. Study responsiveness Determine clinical outcome assessment Objective examined responsiveness, minimal important difference (MID), test–retest reliability, and construct validity of the 2MWT in people with stable COPD attending outpatient pulmonary rehabilitation (PR).

  16. Study responsiveness • Inclusion • People with stable COPD confirmed by spirometry • Exclusion • severe comorbidities that prevented exercise training (severe cardiac, neurological, or musculoskeletal conditions) • met any of the absolute contraindications to conduct of a 6MWT.

  17. Reporting responsiveness Effect sizes for these measures were calculated using the formula d= (mean post − mean pre)/SD pooled. For MID, were calculated using the formula MDC=1.96× √ 2×SEM Based on standard error of measurement (SEM) was calculated according to the formula SEM = σ1√(1 − r) where σ1 is the baseline standard deviation and r is the test–retest reliability

  18. Findings

  19. Test–retest reliability of the 10-metre fast walk test and 6-minute walk test in ambulatory school-aged children with cerebral palsy Developmental Medicine & Child Neurology 2008, 50: 370–376

  20. Study responsiveness Determine clinical outcome assessment Objective to estimate the test–retest reliability of the 10mFWT and 6MWT in ambulatory school-aged children with CP and within Gross Motor Function Classification System (GMFCS)18subgroups (i.e. GMFCS Levels I, II, and III). The secondary objective was to estimate the minimum detectable change (MDC) in the total sample and subgroups

  21. Study responsiveness • Inclusion • (1) diagnosis of spastic CP; • (2) between 4 and 18 years of age; • (3) GMFCS Levels I, II, or III as determined by their developmental pediatrician; • (4) ability to walk independently without stopping for 6 minutes, with or without a walking aid; • (5) ability to follow verbal instructions in English; and • (6) ability to cooperate for at least 30 minutes as judged by their treating physiotherapist • The exclusion • orthopaedic surgery within the past 6 months; and • botulinum toxin type A (BoNT-A) injections within the preceding 3 months.

  22. Reporting responsiveness For MID, were calculated using the formula MDC95=1.96× √ 2×SEM Based on standard error of measurement (SEM) was calculated according to the formula SEM = SDb √(1−ICC) where σ1 is the baseline standard deviation and r is the test–retest reliability

  23. Findings

  24. Evidence-Based Approach to Choosing Outcome Measures

  25. Evidence-Based Approach to Choosing OMs • An evidence-based approach to selecting outcome measures involves making judgements about the quality of the validity and reliability studies, interpreting the findings and deciding whether they are applicable to one’s own specific practice.

  26. Asking a Question

  27. Searching for evidence CINAHL PT now

  28. Critical appraisal of OMs Research

  29. Appraising Evidence of OMS Several checklists for measurement properties exist • Scientific advisory committee (SAC) of the medical outcome trust in 2002 • COSMIN checklist (www.cosmin.). • Christina Jerosch-Herold”s Checklist for critical appraisal of OMs • Oxford center for evidence base No consensus on terminilogy and defintions None is generally accepted and widely used

  30. COSMIN Checklist • COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed in an international Delphi study with the aim to provide tools for evidence-based instrument selection • The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Quality of Life Research • May 2010, Volume 19, Issue 4, pp 539–549 • available at http://www.cosmin.nl/).

  31. COSMIN Checklist The COSMIN checklist comprises 12 boxes: • Ten boxes evaluate whether a study meets the standard for good methodological quality. • A=Internal consistency, • B=Reliability, • C=Measurement error, • D=Content validity, • E=Structural validity, • F=Hypothesis-testing, • G=Cross-cultural validity, • H=Criterion validity, • I=Responsiveness • J=Interpretability. • Two additional boxes were included to meet studies that use the Item Response Theory method (IRT box) and general requirements for results generalization (Generalizability box).

  32. COSMIN Applications COSMIN aims to improve the selection of outcome measurement instruments both in research and in clinical practice by developing methodology and practical tools for selecting the most suitable outcome measurement instrument. • Systematic review of measurement properties • Measure instrument selection • Designing and reporting study on measurement properties • Identification the need for further research on measurement properties

  33. COSMIN Checklist Manual

  34. COSMIN Checklist Four steps should be taken to complete the COSMIN checklist

  35. Application of evidence to patient/client

  36. Now Start appraising OMs I think I have given input to `u’

More Related