1 / 23

MEASUREMENT: VALIDITY

MEASUREMENT: VALIDITY. Lu Ann Aday, Ph.D. The University of Texas School of Public Health. VALIDITY: Definition.

toby
Download Presentation

MEASUREMENT: VALIDITY

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MEASUREMENT: VALIDITY Lu Ann Aday, Ph.D. The University of Texas School of Public Health

  2. VALIDITY: Definition • Extent of systematic departure in answers to questions in relationship to the meaning of the concept itself (content), equivalent measures of the same concept (criterion), or hypothesized associations with measures of similar or different concepts(construct).

  3. VALIDITY: Types • Content validity • Criterion validity • Construct validity

  4. VALIDITY: Computation • Requires analyzing the association between survey measures and documented definitions of the underlying concept (content); an equivalent “gold standard” of the survey measures (criterion); or theoretical associations with other abstract concepts (construct).

  5. VALIDITY: Content • Definition: extent to which the survey measures adequately represent the concept of interest

  6. VALIDITY: Content • Factors affecting: • Failure to thoroughly conduct background research on the definition & measurement of the study concept • Failure to clearly define the study concept

  7. VALIDITY: Content • Computation: • 1. Identify domain or universe of interest through a thorough review of the literature. • 2. Select or construct questions based on the review. • 3. Ask subject matter experts to review the items you have selected or constructed.

  8. VALIDITY: Content • Validity criteria: • 1. Is there at least one item to represent the concept? • 2. Are there enough items? • 3. Are the items well accepted, based on the literature & expert opinion? • 4. Are they reliable?

  9. VALIDITY: Criterion • Definition: extent to which the survey measure predicts (predictive) or agrees with(concurrent) a criterion indicator of the concept of interest

  10. VALIDITY: Criterion • Factors affecting: • Underreporting due to problems with recalling non-salient or frequently occurring events • Overreporting due to problems with telescoping salient events

  11. VALIDITY: Criterion- Predictive: Correlation Analysis • Computation: Compute correlation coefficient between answers to survey question and futurecriterion source value for same concept : RespondentSurvey QuestionMD Observation 1 ADLs=3 ADLs=3 2 ADLs=2 ADLs=2 3 ADLs=1 ADLs=1 4 ADLs=0 ADLs=3 5 ADLs=0 ADLs=0

  12. VALIDITY: Criterion- Concurrent: Correlation Analysis • Computation:Compute correlation coefficient between answers to survey question and currentcriterion source value for same concept: • RespondentSurvey QuestionClinic Record 1 BP=140/90 BP=140/90 2 BP=150/80 BP=150/80 3 BP=145/95 BP=145/95 4 BP=145/95 BP=120/80 5 BP=140/90 BP=140/90

  13. VALIDITY: Criterion-Concurrent: Sensitivity-Specificity Analysis

  14. VALIDITY: Criterion-Concurrent: Sensitivity-Specificity Analysis

  15. VALIDITY: Construct • Definition: extent to which relationships between measures agree with relationships predicted by theories or hypotheses • Convergent – Ho: Related • Discriminant – Ho:Not Related

  16. VALIDITY: Construct • Factors affecting: • Inadequate conceptualization or measurement of study constructs • Inadequate theory to guide specification of hypothesized relationships

  17. VALIDITY: Construct: Correlation Analysis • Computation: Compute correlation coefficient between survey question or scales measuring construct and related (or unrelated) constructs : RespondentPhysicalMentalOverall 1 Good Fair Good 2 Good Fair Good 3 Good Fair Good 4 Good Fair Good 5 Good Fair Good

  18. VALIDITY: Construct- Known Groups Validity Analysis (McHorney, et al., 1993, Table 3, p. 255)

  19. VALIDITY: CONSTRUCT-Factor Analysis(McHorney, et al., 1993, Table 1, p. 251)

  20. VALIDITY: Construct—Multi-Trait, Multi-Method

  21. VALIDITY: Effect of Reliability • Theoretical • Variable can be no more valid than it is reliable. • Empirical • Rt = theoretical correlation between study & criterion variables (unobservable) • Rx = reliability of study variable (computed) • Ry = reliability of criterion variable (computed) • Rt * sqrt [ Rx * Ry] = Max. achievable correlation • 1.00 * sqrt [.60 * .80] = .69

  22. WHEN TO UNDERTAKE VALIDITY ANALYSIS

  23. REFERENCES • DeVellis, Robert F. (2003). Scale Development: Theory and Applications. Second Edition. Thousand Oaks, CA: Sage. • Gandek, B., & Ware, J.E., Jr., for the IQOLA Project (1998). Methods for validating and norming translations of health status questionnaires: The IQOLA Project Approach. J. Clinical Epidemiology, 51 (11), 953-959. • McHorney, C.A., Ware, J.E., Jr., & Raczek, A.E. (1993). The MOS 36-item short-form health survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical Care, 31(3), 247-263. [Note: Tables 1 and 3 adapted and reprinted with the permission of Lippincott, Williams & Wilkins, Publishers.]

More Related