ITEM RESPONSE THEORY MODELS & COMPOSITE MEASURES Sharon-Lise T. Normand

1. ITEM RESPONSE THEORY MODELS & COMPOSITE MEASURESSharon-Lise T. Normand FOCUS: How to deal with data that are dichotomous or ordinal? “s” index subject “j” index item (or measure) s = “true” unobserved score.

2. WHAT IS AN ITEM RESPONSE THEORY (IRT) MODEL? A statistical model that relates the probability of response to an item to item-specific parametersand to the subject’s underlying latent trait.

3. Classical Test Theory Estimate reliability of items (coefficient ). Model: Ysj = s + sj Ysj = response s = underlying trait sj = error Normal with expectation 0 and constant variance. Item Response Theory Estimate discriminating ability of items using item-specific parameters. Responses within a subject are independent conditional on latent trait. Normality & constant variance not assumed s ~ N(0,1)

4. DICHOTOMOUS OR ORDINAL RESPONSES Item response formulation: Observed response is ysj; generalized linear model formulation: h(P(ysj = 1 given s)) = j(s - j) • h = link function (logit or probit) • j and j are “item” parameters.

5. RASCH MODEL (1-PARAMETER LOGISTIC) Simplest IRT Model • Ysj = 1 if subject s responds correctly to item j and 0 otherwise. • s =latent ability for subject s. • j = difficulty of jth item. Probability subject s responds correctly jth item: P(Ysj=1|s) = exp(s - j ) 1+exp(s - j )

6. RASCH MODEL: 3 SUBJECTS WITH DIFFERENT TRAITS = DIFFICULTY

7. 2-PARAMETER LOGISTIC • Ysj = 1 if subject s responds correctly to item j and 0 otherwise. • s =latent ability for subject s. • j = difficulty of jth item. • j = discrimination of jth item (j > 0) Probability subject s responds correctly jth item: P(Ysj=1|s) = exp(j(s - j) 1+exp(j(s - j))

8. 2-PARAMETER LOGISTIC: 3 ITEMS ( = 1)  = 1  = 0.5  = 3

9. 2-PARAMETER LOGISTIC: 3 ITEMS & DIFFERENT ’s  = 3,  = 0  = 0.5,  = 1  = 1,  = -1

10. LVF = left ventricular function; LVSD = left ventricular systolic dysfunction Teixeira-Pinto and Normand – Statistics in Medicine (2008)

11. EXAMPLE 1: Hospital Performance

12. LS = Latent Score

13. Comparing Composites:(Teixeira-Pinto and Normand, Statistics in Medicine (2008)) 2005 Data

14. EXAMPLE 2: BASIS-32 Background. BASIS-32, an instrument to assess subjective distress was originally developed using classical testing theory based on a sample of psychiatric inpatients from one hospital. Data. Self-reports of symptom and problem difficulty obtained from 2,656 psychiatric inpatients discharged from 13 US hospitals between May 2001 and April 2002.(BASIS-32 = Behavior and Symptom Identification Scale) Normand, Belanger, Eisen – Health Services Outcomes Research Methodology (2006)

16. GRADED RESPONSE MODEL(IRT MODEL) When response options are ordinal categorical, e.g., Ysj = 0, 1, 2, 3, or 4 where 0 = No difficulty; 1 = A little difficulty; 2 = Moderate difficulty; 3 = Quite a bit of difficulty; 4 = Extreme difficulty Need to model probability of responding in each category.

17. GRADED RESPONSE MODEL Probability subject s responds in threshold category kor higher: P(Ysj k|s)= Pjk*(s) = exp[j(s- jk)] 1 + exp[j(s - jk)] s = latent trait (e.g., subjective distress) j = discrimination of jth item (j > 0) j4  j3  j2  j1 = threshold parameters

18. CUMULATIVE PROBABILITIES  = 0.90 P1*()  = 6.00 P4*()

19. CUMULATIVE PROBABILITIES

21. Concluding Remarks:(Kaplan & Normand 2006)