Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure

1. Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure SHOJIMA Kojiro The NationalCenter for University Entrance Examinations shojima@rd.dnc.ac.jp

2. Purpose of Research • Development of method for visualizing inter-item dependency structure • Especially important for analyzing math test data • Proposal: ATRISCAL • Asymmetric Triangulation Scaling • An asymmetric multidimensional scaling • Data: Conditional correct response rate matrix

3. Joint correct response rate matrix • n×n symmetry matrix • The j-th diagonal elementP(j,j)=P(j) • Correct response rate of item j • The ij-thoff-diagonal elementP(i,j) • Joint correct response rate of items i and j • symmetryP(i,j)=P(j,i)

4. Conditional correct response rate matrix • n×n asymmetry matrix • The j-th diagonal elementP(j|j)=P(j)/P(j)=1.0 • The ij-th off-diagonal element P(j|i)=P(i,j)/P(i) • The correct response rate of item j when item i is answered correctly • P(i|j)≠P(j|i): Usually asymmetric

5. Multidimensional scaling (MDS) QM Q2 X15 X7 X11 X4 X12 X5 X13 X1 X2 X10 X3 X9 X14 X6 X8 Q1 O

6. Relationship betweenitems i and j Xi Xij Xｊ O

7. Relationship betweenitems i and j Xi Xij Xｊ O

8. Expanded Asymmetric correct response rate matrix • The asymmetric matrix lacks information about the correct response rate of each item • So we add the imaginary n+1-th item whose correct response rate is 1.0 • P(j|n+1)=P(j,n+1)/P(n+1)=P(j) • P(n+1|j)=P(j,n+1)/P(j)=1.0

9. Stress function

10. δ(delta) Xi Xi Xｊ Well-formed triangle Not well-formed triangle The foot from O does NOT fall on line segment XiXj Xｊ Xij • The perpendicular foot from O falls on line segment XiXj Xij O O δij=δji=1 δij=δji=0

11. λ（lambda） 0.5 1 0.5 1

12. Spatial indeterminacy and fixed coordinates • Number of dimensions=3 • Coordinates of item n+1 • (xn+1=0, yn+1=0, zn+1=1) • Coordinate of item k, which has the lowest correct response rate • (xk=0, yk>0, zk) • Coordinate of item l, which has a moderate P(・|k) • (xl>0, yl, zl)

13. Optimization of Stress Function • Two-stage optimization • Stage 1: Simple genetic algorithm (SGA) • Stage 2: Steepest descent method (SDM) The author is grateful to Dr. Akinori Okada (Tama University) for speaking about this strategy at the spring seminar of the Behaviormetric Society of Japan (Gotemba, Japan) in March 2010.

14. Demonstration of exametrika www.rd.dnc.ac.jp/~shojima/exmk/index.htm

15. Result of Analysis: Radial Map • Red dots • Estimated coordinates • Orange dots • Points of intersections of extensions of red line segments and the surface of the hemisphere

16. Relationship betweenimaginary item n+1and item j • P(j)→1.0 • P(k)→0.0 Xn+1 Xj 1 P(j) Xk P(k) O

17. Relashinship between items i and j • P(i)<P(j) • P(i|j)→1.0 • P(i|j)→0.0 Xn+1 Xj Xi O

18. Topographic Map • The coordinates of orange points are projected onto the XY plane • Voronoi tessellation • Lift each Voronoi region by the length of the orange line segment • Separate height with different colors

19. Mastery Maps • For each examinee

20. Demonstration of exametrika www.rd.dnc.ac.jp/~shojima/exmk/index.htm

21. Thank you for listening. SHOJIMA Kojiro The National Center for University Entrance Examinations Tokyo Institute of Technology shojima@rd.dnc.ac.jp