1 / 20

Psychometric Evaluation and Calibration Plan

Psychometric Evaluation and Calibration Plan. Ron D. Hays, Ph.D. October 14, 2006 ISOQOL (Opala) 2:00-3:30 pm. PROMIS Domains. Physical functioning (Hays/Bjorner) Pain (Revicki/Cook) Fatigue (Lai) Emotional distress (Choi/Reise) Social/role participation (Bode/Hahn). Datasets.

dea
Download Presentation

Psychometric Evaluation and Calibration Plan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Psychometric Evaluation and Calibration Plan Ron D. Hays, Ph.D. October 14, 2006 ISOQOL (Opala) 2:00-3:30 pm

  2. PROMIS Domains • Physical functioning (Hays/Bjorner) • Pain (Revicki/Cook) • Fatigue (Lai) • Emotional distress (Choi/Reise) • Social/role participation (Bode/Hahn)

  3. Datasets • Cancer Item Banks (Northwestern) • Digitalis Investigation Group Study--randomized double-blind placebo-controlled trial evaluating effect of digoxin on mortality in 581 patients with heart failure and sinus rhythm. • IMMPACT--internet-based survey of individuals with chronic pain from the American Chronic Pain Association website • Medical Outcomes Study--observational study of persons with hypertension, diabetes, heart disease, and/or depression in Boston, Chicago, and Los Angeles • WHOQOL-100 data (n = 442 from U.S. field center)

  4. Types of Analyses • Classical Test Theory Statistics • IRT Model Assumptions • Model Fit • Differential Item Functioning • Item Calibration

  5. Classical Test Theory Statistics • Out of range • Item frequencies and distributions • Inter-item correlations • Item-scale correlations • Internal consistency reliability

  6. IRT Model Assumptions • (Uni)dimensionality • Local independence • Monotonicity

  7. Sufficient Unidimensionality • Confirmatory factor models • One factor • Bifactor (general and group factors)

  8. Local Independence • After controlling for dominant factor(s), item pairs should not be associated. • Look at residual correlations (> 0.20)

  9. Monotonicity • Probability of selecting a response category indicative of better health should increase as underlying health increases. • Item response function graphs with • y-axis: proportion positive for item step • x-axis: raw scale score minus item score

  10. Category Response Curves for Samejima’s Graded Response Model

  11. Model Fit • Compare observed and expected response frequencies by item and response category • Items that do not fit and less discriminating items identified and reviewed by content experts

  12. Differential Item Functioning • Uniform DIF • Threshold parameter • Non-uniform DIF • Discrimination parameter • Gender, race/ethnicity, age, disease

  13. Dichotomous Items Showing DIF(2-Parameter Model) Hispanic White White Slope DIF Location DIF Hispanic

  14. Item Calibration • Item parameters (threshold, discrimination) • Mean differences for studied disease groups

  15. Example of Lessons Learned in Secondary Analyses • Emotional distress • Cannot be adequately modeled as a unidimensional construct. • Limited representation of positive end of construct • Several items having some response options that provide little information.

  16. Documentation • Public website:http://www.nihpromis.org/ • Peer-reviewed manuscripts, e.g.: • Hays, R. D. et al. (in press). Item response theory analyses of physical functioning items in the Medical Outcomes Study. Medical Care. • Reeve, B. B. (in press). Psychometric evaluation and calibration of health-related quality of life items banks: Plans for the Patient-Reported Outcome Measurement Information System (PROMIS). Medical Care.

  17. Datasets Subjected to Psychometric Analysis • Cancer Fatigue:Cancer Item Banking Project at NWU • Cancer Pain:Cancer Item Banking Project at NWU • Cancer Social:Cancer Item Banking Project at NWU • CSSCD:Cooperative Study of Sickle Cell Disease (pediatric) • CHC:Chronic Hepatitis C Study • CHS:Cardiovascular Health Study • DIG:Digitalis Investigation Group Quality of Life Sub-study • IMMPACT:Multiple Pain Projects • MOS:Medical Outcomes Study • NGHS:National Growth and Health Study (peds.) • Q-Score:Cancer Quality of Life Project at NWU • WHOQOL:World Health Organization Quality of Life Project

  18. Datasets Subjected to Psychometric Analysis PROMIS Domains Emotional Distress Fatigue Pain Physical Function Social Role Participation CHS (NWU/Cook) CHC (Medtap/Revicki & Chen) Cancer Pain (NWU/Lai) CHS (NWU/Cook) CHS (NWU/Cook) DIG (UCLA/Hays) DIG (UCLA/Hays) CHS (NWU/Cook) IMMPACT (Medtap/Revicki & Chen) Cancer Social (NWU/Bode) MOS (UCLA/Hays & Spritzer) Q-Score (NWU/Bode) Cancer Fatigue (NWU/Lai) WHOQOL (UCLA/Hays) WHOQOL (UCLA/Hays) Q-Score (NWU/Lai) WHOQOL (UCLA/Hays) WHOQOL (UCLA/Hays) CSSCD (NWU) WHOQOL (UCLA/Hays) NGHS (NWU) NGHS (NWU)

More Related