EPSY 546: LECTURE 1 SUMMARY

1. EPSY 546: LECTURE 1SUMMARY George Karabatsos

2. REVIEW

3. REVIEW • Test (& types of tests)

4. REVIEW • Test (& types of tests) • Item response scoring paradigms

5. REVIEW • Test (& types of tests) • Item response scoring paradigms • Data paradigm of test theory (typical)

6. DATA PARADIGM

7. REVIEW: Latent Trait • Latent Trait   Re (unidimensional)

8. REVIEW: Latent Trait • Latent Trait   Re (unidimensional) • Real Examples of Latent Traits

9. REVIEW: IRF • Item Response Function (IRF)

10. REVIEW: IRF • Item Response Function (IRF) • Represents different theories about latent traits.

11. REVIEW: IRF • Item Response Function (IRF) • Dichotomous response: Pj() = Pr[Xj = 1] = Pr[Correct Response to item j | ]

12. REVIEW: IRF • Item Response Function (IRF) • Polychotomous response: Pjk() = Pr[Xj> k | ] = Pr[Exceed category k of item j | ]

13. REVIEW: IRF • Item Response Function (IRF) • Dichotomous or Polychotomous response: Ej() = [Expected Rating for item j | ] 0 < Ej() < K

14. IRF: Dichotomous items

15. IRF: Polychotomous items

16. REVIEW: SCALES • The unweighted total score X+n stochastically orders the latent trait  • (Hyunh, 1994; Grayson, 1988)

17. REVIEW: SCALES • 4 Scales of Measurement • Conjoint Measurement

18. REVIEW • Conjoint Measurement • Row Independence Axiom

19. REVIEW • Conjoint Measurement • Row Independence Axiom • Property: Ordinal Scaling and unidimensionality of  (test score)

20. INDEPENDENCE AXIOM (row)

21. REVIEW • Conjoint Measurement • Row Independence Axiom • Property: Ordinal Scaling and unidimensionality of  (test score) • IRF: Non-decreasing over 

22. REVIEW • Conjoint Measurement • Row Independence Axiom • Property: Ordinal Scaling and unidimensionality of  (test score) • IRF: Non-decreasing over  • Models: MH, 2PL, 3PL, 4PL, True Score, Factor Analysis

23. 2PL:

24. 3PL:

25. 4PL:

26. Monotone Homogeneity (MH)

27. REVIEW • Conjoint Measurement • Column Independence Axiom (adding)

28. REVIEW • Conjoint Measurement • Column Independence Axiom (adding) • Property: Ordinal Scaling and unidimensionality of both  (test score)and item difficulty (item score)

29. INDEPENDENCE AXIOM (column)

30. REVIEW • Conjoint Measurement • Column Independence Axiom (adding) • Property: Ordinal Scaling and unidimensionality of both  (test score)and item difficulty (item score) • IRF: Non-decreasing and non-intersecting over 

31. REVIEW • Conjoint Measurement • Column Independence Axiom (adding) • Property: Ordinal Scaling and unidimensionality of both  (test score)and item difficulty (item score) • IRF: Non-decreasing and non-intersecting over  • Models: DM, ISOP

32. DM/ISOP (Scheiblechner 1995)

33. REVIEW • Conjoint Measurement • Thomsen Condition (adding)

34. REVIEW • Conjoint Measurement • Thomsen Condition (adding) • Property: Interval Scaling and unidimensionality of both  (test score)and item difficulty (item score)

35. Thomsen condition(e.g.,double cancellation)

36. REVIEW • Conjoint Measurement • Thomsen Condition (adding) • Property: Interval Scaling and unidimensionality of both  (test score)and item difficulty (item score) • IRF: Non-decreasing and parallel (non-intersecting) over 

37. REVIEW • Conjoint Measurement • Thomsen Condition (adding) • Property: Interval Scaling and unidimensionality of both  (test score)and item difficulty (item score) • IRF: Non-decreasing and parallel (non-intersecting) over  • Models: Rasch Model, ADISOP

38. RASCH-1PL:

39. REVIEW • 5 Challenges of Latent Trait Measurement

40. REVIEW • 5 Challenges of Latent Trait Measurement • Test Theory attempts to address these challenges

41. REVIEW • Test Construction (10 Steps)

42. REVIEW • Test Construction (10 Steps) • Basic Statistics of Test Theory

43. REVIEW • Total Test Score (X+) variance = Sum[Item Variances] + Sum[Item Covariances]

44. EPSY 546: LECTURE 2TRUE SCORE TEST THEORY AND RELIABILITY George Karabatsos

45. TRUE SCORE MODEL • Theory: Test score is a random variable. X+nObserved Test Score of person n, Tn True Test Score (unknown) enRandom Error (unknown)

46. TRUE SCORE MODEL • The Observed person test scoreX+n is a random variable (according to some distribution) with mean Tn = E(X+n) and variance 2(X+n)= 2(en).

47. TRUE SCORE MODEL • The Observed person test scoreX+n is a random variable (according to some distribution) with mean Tn = E(X+n) and variance 2(X+n)= 2(en). • Random Error en = X+n– Tn is distributed with mean E(en) = E(X+n–Tn) = 0, and variance 2(en) = 2(X+n) .

48. TRUE SCORE MODEL • True Score: Tn true score of person n E (Xn) expected score of person n s Possible score s{0,1,…,s,…,S} pns Pr[Person n has test score s]

49. TRUE SCORE MODEL • 3 Assumptions: • Over the population of examinees, error has a mean of 0. E[e] = 0 • Over the population of examinees, true scores and error scores have 0 correlation. [T, e] = 0

50. TRUE SCORE MODEL • 3 Assumptions: • For a set of persons, the correlations of the error scores between two testings is zero. [e1, e2] = 0 • “Two testings”: when a set of persons take two separate tests, or complete two testing occasions with the same form. • The two sets of person scores are assumed to be randomly chosen from two independent distributions of possible observed scores.