Comparison of dimension-aligning techniques in a multidimensional IRT context

1. Comparison of dimension-aligning techniques in a multidimensional IRT context Hiroyuki Yamada, Karen Draney, Tzur Karelitz, Stephen Moore, and Mark Wilson University of California at Berkeley

2. Purpose Compare two dimension-equating techniques: Person or Theta approach place the other dimensions� scales onto one �reference� dimension�s scale Item or Delta approach place multiple dimensions� scales onto a common scale constructed from unidimensional calibration

3. Data Set 703 preschoolers in the Desired Results Developmental Profile (DRDP) Calibration Study Preschool DRDP Instrument A total of 39 items (11 items for D1, 22 items for D2, and 6 items for D3) 5 developmental levels per item (scored as 0 � 4)

4. Original Data Analysis Partial credit model (Masters, 1982) 3 dimensional (3D) model 1 dimensional (1D) model Derived estimates of item and person parameters were needed for dimension-equating ConQuest (Wu, Adams, & Wilson, 1998)

5. Theta Approach Create 2 artificial data sets: one response vector for each raw score on the reference dimension (D2), missing all the rest one response vector for every possible raw score combination across all of the non-reference dimensions (D1 & D3), missing all the data on the reference dimension

6. Theta Approach Run two 3D anchored analyses with these artificial data sets, anchoring item, variance-covariance, and mean parameters on the estimates from the original 3D run Obtain expected a posterior (EAP) estimates

7. Theta Approach Using these EAPs, obtain two sets of slopes and intercepts by regressing the reference dimension (D2) on each of the non-reference dimensions (D1 & D3) Conduct the same regressions using the EAPs from the original 3D run

8. Linear relationships of D2 with D1 and D3 in EAP

9. Theta Approach Apply linear transformation to the original 3D item estimates of the D1 and D3 using these 3 sets of slopes and intercepts Run three anchored 3D analyses using the 3 sets of the transformed item estimates as anchoring values

10. Delta Approach Compute the means and SDs for all 3 subsets of item location estimates from the original 1D run Apply linear transformation to the original 3D item location and step estimates using the following formulas:

11. Delta Approach did = did3D * (SDd1D/SDd3D) + MEANd1D timd = timd3D * (SDd1D/SDd3D), where i = item 1-39, m = step 1-3, and d = dimension 1-3 Run an anchored 3D analysis with the transformed item estimates as anchoring values

12. Comparison in mean and variance