SHOJIMA Kojiro The National Center for University Entrance Examinations shojima@rd.dnc.ac.jp

1. Asymmetric Triangulation Scaling:An asymmetric MDS for extracting inter-item dependency structure from test data SHOJIMA Kojiro The National Center for University Entrance Examinations shojima@rd.dnc.ac.jp

2. Purpose of Research • Development of method for visualizing inter-item dependency structure • Especially important for analyzing math test data • Proposal: ATRISCAL • Asymmetric TRIangulation SCALing • An asymmetric multidimensional scaling • Conditional correct response rate matrix is object of analysis

3. Joint correct response rate matrix • n×nsymmetry matrix • The j-th diagonal elementP(j,j)=P(j) • Correct response rate of item j • The ij-thoff-diagonal elementP(i,j) • Joint correct response rate of items iand j • symmetryP(i,j)=P(j,i)

4. Conditional correct response rate matrix • n×nasymmetry matrix • The j-th diagonal elementP(j|j)=P(j)/P(j)=1.0 • The ij-th off-diagonal element P(j|i)=P(i,j)/P(i) • The correct response rate of item j when item i is answered correctly • P(i|j)≠P(j|i): Usually asymmetric

5. Multidimensional scaling (MDS) QM Q2 X15 X7 X11 X4 X12 X5 X13 X1 X2 X10 X3 X9 X14 X6 X8 Q1 O

6. Relationship betweenitems i and j Xi Xij Xｊ O

7. Relationship betweenitems i and j Xi Xij Xｊ O

8. Expanded Asymmetric correct response rate matrix • The asymmetric matrix lacks information about the correct response rate of each item • So we add the imaginary n+1-th item whose correct response rate is 1.0 • P(j|n+1)=P(j,n+1)/P(n+1)=P(j) • P(n+1|j)=P(j,n+1)/P(j)=1.0

9. Stress function

10. δ(delta) Xi Xi Xｊ Well-formed triangle Not well-formed triangle The foot from O does NOT fall on line segment XiXj Xｊ Xij • The perpendicular foot from O falls on line segment XiXj Xij O O δij=δji=1 δij=δji=0

11. λ（lambda） 0.5 1 0.5 1

12. Spatial indeterminacy and fixed coordinates • Number of dimensions=3 • Coordinates of item n+1 • (xn+1=0, yn+1=0, zn+1=1) • Coordinate of item k, which has the lowest correct response rate • (xk=0, yk>0, zk) • Coordinate of item l, which has a moderate P(・|k) • (xl>0, yl, zl)

13. Demonstration of exametrika www.rd.dnc.ac.jp/~shojima/exmk/index.htm

14. Result of Analysis: Radial Map • Red dots • Estimated coordinates • Orange dots • Points of intersections of extensions of red line segments and the surface of the hemisphere

15. Relationship betweenimaginary item n+1and item j • P(j)→1.0 • P(k)→0.0 Xn+1 Xj 1 P(j) Xk P(k) O

16. Relashinship between items i and j • P(i)<P(j) • P(i|j)→1.0 • P(i|j)→0.0 Xn+1 Xj Xi O

17. Topographic Map • The coordinates of orange points are projected onto the XY plane • Voronoi tessellation • Lift each Voronoi region by the length of the orange line segment • Separate height with different colors

18. Mastery Maps • For each examinee

19. Demonstration of exametrika www.rd.dnc.ac.jp/~shojima/exmk/index.htm

20. Thank you for listening. SHOJIMA Kojiro The National Center for University Entrance Examinations shojima@rd.dnc.ac.jp