1 / 72

EPSY 546: LECTURE 1 SUMMARY

EPSY 546: LECTURE 1 SUMMARY. George Karabatsos. REVIEW. REVIEW. Test (& types of tests). REVIEW. Test (& types of tests) Item response scoring paradigms. REVIEW. Test (& types of tests) Item response scoring paradigms Data paradigm of test theory (typical). DATA PARADIGM.

lana
Download Presentation

EPSY 546: LECTURE 1 SUMMARY

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. EPSY 546: LECTURE 1SUMMARY George Karabatsos

  2. REVIEW

  3. REVIEW • Test (& types of tests)

  4. REVIEW • Test (& types of tests) • Item response scoring paradigms

  5. REVIEW • Test (& types of tests) • Item response scoring paradigms • Data paradigm of test theory (typical)

  6. DATA PARADIGM

  7. REVIEW: Latent Trait • Latent Trait   Re (unidimensional)

  8. REVIEW: Latent Trait • Latent Trait   Re (unidimensional) • Real Examples of Latent Traits

  9. REVIEW: IRF • Item Response Function (IRF)

  10. REVIEW: IRF • Item Response Function (IRF) • Represents different theories about latent traits.

  11. REVIEW: IRF • Item Response Function (IRF) • Dichotomous response: Pj() = Pr[Xj = 1] = Pr[Correct Response to item j | ]

  12. REVIEW: IRF • Item Response Function (IRF) • Polychotomous response: Pjk() = Pr[Xj> k | ] = Pr[Exceed category k of item j | ]

  13. REVIEW: IRF • Item Response Function (IRF) • Dichotomous or Polychotomous response: Ej() = [Expected Rating for item j | ] 0 < Ej() < K

  14. IRF: Dichotomous items

  15. IRF: Polychotomous items

  16. REVIEW: SCALES • The unweighted total score X+n stochastically orders the latent trait  • (Hyunh, 1994; Grayson, 1988)

  17. REVIEW: SCALES • 4 Scales of Measurement • Conjoint Measurement

  18. REVIEW • Conjoint Measurement • Row Independence Axiom

  19. REVIEW • Conjoint Measurement • Row Independence Axiom • Property: Ordinal Scaling and unidimensionality of  (test score)

  20. INDEPENDENCE AXIOM (row)

  21. REVIEW • Conjoint Measurement • Row Independence Axiom • Property: Ordinal Scaling and unidimensionality of  (test score) • IRF: Non-decreasing over 

  22. REVIEW • Conjoint Measurement • Row Independence Axiom • Property: Ordinal Scaling and unidimensionality of  (test score) • IRF: Non-decreasing over  • Models: MH, 2PL, 3PL, 4PL, True Score, Factor Analysis

  23. 2PL:

  24. 3PL:

  25. 4PL:

  26. Monotone Homogeneity (MH)

  27. REVIEW • Conjoint Measurement • Column Independence Axiom (adding)

  28. REVIEW • Conjoint Measurement • Column Independence Axiom (adding) • Property: Ordinal Scaling and unidimensionality of both  (test score)and item difficulty (item score)

  29. INDEPENDENCE AXIOM (column)

  30. REVIEW • Conjoint Measurement • Column Independence Axiom (adding) • Property: Ordinal Scaling and unidimensionality of both  (test score)and item difficulty (item score) • IRF: Non-decreasing and non-intersecting over 

  31. REVIEW • Conjoint Measurement • Column Independence Axiom (adding) • Property: Ordinal Scaling and unidimensionality of both  (test score)and item difficulty (item score) • IRF: Non-decreasing and non-intersecting over  • Models: DM, ISOP

  32. DM/ISOP (Scheiblechner 1995)

  33. REVIEW • Conjoint Measurement • Thomsen Condition (adding)

  34. REVIEW • Conjoint Measurement • Thomsen Condition (adding) • Property: Interval Scaling and unidimensionality of both  (test score)and item difficulty (item score)

  35. Thomsen condition(e.g.,double cancellation)

  36. REVIEW • Conjoint Measurement • Thomsen Condition (adding) • Property: Interval Scaling and unidimensionality of both  (test score)and item difficulty (item score) • IRF: Non-decreasing and parallel (non-intersecting) over 

  37. REVIEW • Conjoint Measurement • Thomsen Condition (adding) • Property: Interval Scaling and unidimensionality of both  (test score)and item difficulty (item score) • IRF: Non-decreasing and parallel (non-intersecting) over  • Models: Rasch Model, ADISOP

  38. RASCH-1PL:

  39. REVIEW • 5 Challenges of Latent Trait Measurement

  40. REVIEW • 5 Challenges of Latent Trait Measurement • Test Theory attempts to address these challenges

  41. REVIEW • Test Construction (10 Steps)

  42. REVIEW • Test Construction (10 Steps) • Basic Statistics of Test Theory

  43. REVIEW • Total Test Score (X+) variance = Sum[Item Variances] + Sum[Item Covariances]

  44. EPSY 546: LECTURE 2TRUE SCORE TEST THEORY AND RELIABILITY George Karabatsos

  45. TRUE SCORE MODEL • Theory: Test score is a random variable. X+nObserved Test Score of person n, Tn True Test Score (unknown) enRandom Error (unknown)

  46. TRUE SCORE MODEL • The Observed person test scoreX+n is a random variable (according to some distribution) with mean Tn = E(X+n) and variance 2(X+n)= 2(en).

  47. TRUE SCORE MODEL • The Observed person test scoreX+n is a random variable (according to some distribution) with mean Tn = E(X+n) and variance 2(X+n)= 2(en). • Random Error en = X+n– Tn is distributed with mean E(en) = E(X+n–Tn) = 0, and variance 2(en) = 2(X+n) .

  48. TRUE SCORE MODEL • True Score: Tn true score of person n E (Xn) expected score of person n s Possible score s{0,1,…,s,…,S} pns Pr[Person n has test score s]

  49. TRUE SCORE MODEL • 3 Assumptions: • Over the population of examinees, error has a mean of 0. E[e] = 0 • Over the population of examinees, true scores and error scores have 0 correlation. [T, e] = 0

  50. TRUE SCORE MODEL • 3 Assumptions: • For a set of persons, the correlations of the error scores between two testings is zero. [e1, e2] = 0 • “Two testings”: when a set of persons take two separate tests, or complete two testing occasions with the same form. • The two sets of person scores are assumed to be randomly chosen from two independent distributions of possible observed scores.

More Related