1 / 21

Lecture 6: Reliability and validity of scales (cont)

Lecture 6: Reliability and validity of scales (cont). 1. In relation to scales, define the following terms: - Content validity - Criterion validity (concurrent and predictive) - Construct validity (discriminant and convergent) - Responsiveness 2. Use of proxy respondents. Validity.

hanley
Download Presentation

Lecture 6: Reliability and validity of scales (cont)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 6: Reliability and validity of scales (cont) 1. In relation to scales, define the following terms: - Content validity - Criterion validity (concurrent and predictive) - Construct validity (discriminant and convergent) - Responsiveness 2. Use of proxy respondents

  2. Validity • Depends on purpose: • screening: discrimination • outcome of treatment: responsiveness, sensitivity to change • prognosis: predictive validity

  3. Content and face validity • Judgment of “experts” and/or members of target population • Does measure adequately sample domain being measured? • Does it appear to measure what it is intended to measure? (eyeball test)

  4. Criterion validity • Criterion (“gold” standard) • Concurrent criterion validity • e.g., screening test vs diagnostic test • Predictive criterion validity • e.g., cancer staging test vs 5-year survival

  5. Construct validity • Is the theoretical construct underlying the measure valid? • Development and testing of hypotheses • Requires multiple data sources and investigations: • Convergent validity: measure is correlated with other measures of similar constructs • discriminant validity: measure is not correlated with measures of different constructs

  6. Responsiveness of measures • Ability to detect clinically important change over time or differences between treatments • Requirement of evaluative measures • Two approaches: • external responsiveness (validation against change in external criterion) • internal responsiveness: compute effect size of ratio of change score to measure of variability (different formulae)

  7. Validity of Delirium Index (DI) • Convergent validity • DI will be correlated with measures of current health/function: • Current ADL disability (Barthel Index) • Current severity of illness (clinical judgment and physiological severity score from APACHE)

  8. Validity of Delirium Index (DI) • Discriminant validity: • DI will be weakly correlated with measures of previous health/function: • premorbid level of ADL disability • severity of dementia • comorbidity (prior)

  9. Validity of DI (cont) • DI score will predict probability of one-year survival • DI score will be sensitive to: • changes in medication exposures • changes in environmental factors

  10. Spearman correlation coefficients between Delirium Index and 3 baseline measures of current status

  11. Spearman correlation coefficients between Delirium Index and 3 baseline measures of prior status

  12. Delirium severity and survival • Proportional hazards regression of delirium severity in delirium cohort • Mean of 1st 2 DI scores • Results • significant interaction: DI predicted survival in patients with delirium alone, not in those with dementia

  13. Effects of medications and environmental factors • Repeated in-hospital measures of DI, medications (medical record), and environmental factors (using checklist) • Repeated measurements analysis performed to investigate associations of changes in risk factors with change in DI score.

  14. Environmental risk factors for delirium Source: McCusker et al, JAGS 2001, 49:1327-34

  15. Medications and DI • Medication changes (increase in # medications, use of anticholinergic medications) were associated with increase in DI score.

  16. Responsiveness of DI • Internal responsiveness: ability to capture real change over time • effect size (change in mean DI/SD at baseline) • standardized response mean (change in mean DI/SD of change) • External responsiveness: are changes in the DI correlated with other measures of change?

  17. Internal responsiveness of the Delirium Index (DI) at 8 weeks

  18. Validity of DI: Conclusions • In patients with delirium +/- dementia: • Correlated with measures of current function • Responsive to change over time • Responsive to changes in environmental factors and medications • In patients with delirium alone: • Predicts survival

  19. Example: Inflammatory Bowel Disease Questionnaire (IBDQ) • Disease-specific HRQoL measure • 30 items covering 4 domains • bowel symptoms • systemic symptoms • emotional function • social function • How to validate?

  20. Validation of Inflammatory Bowel Disease Questionnaire • Questionnaires administered to 42 patients with inflammatory bowel disease on 2 occasions, 1 month apart: • IBDQ • Global ratings of function • Global ratings of change by physician and a relative • Disease Activity Index • Emotional function scale from generic HRQoL instrument

  21. Validation of IBDQ: predictions and results • Correlation of 0.5 or more between patients’s global rating of change and IBDQ subscale on bowel symptoms (result: 0.42) • Correlation of 0.3 or more between change in Disease Activity Index and IBDQ subscale on bowel symptoms (result: 0.33) • Correlation of 0.3 or more between change in Disease Activity Index and IBDQ subscale on systemic symptoms (result: 0.04) • Correlation of 0.5 or more between change in generic measure of emotional function and IBDQ subscale on emotional function (result: 0.76)

More Related