1 / 25

Everything you wanted to know about psychometrics*

Everything you wanted to know about psychometrics*. But didn’t even know you could ask ** ** When I’m thru, you’ll be afraid to. A Perspective. To show the developers what’s possible To show the users how to judge if the test is valid. Psychometric Theory.

sanam
Download Presentation

Everything you wanted to know about psychometrics*

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Everything you wanted to know about psychometrics* But didn’t even know you could ask ** ** When I’m thru, you’ll be afraid to

  2. A Perspective • To show the developers what’s possible • To show the users how to judge if the test is valid

  3. Psychometric Theory • Models of the cognitive processes involved when an individual responses to an item • Repealing rules that no longer apply • Old way vs. New way

  4. Outline • Robert’s Seven New Rules of Measurement • Andrea applies the rules • Kirk shows how to graph the data • Basically, I just want to show off

  5. Rule 1 • Items do not have to be items • Nominal model • Partial credit model • Graded response model

  6. Rule 2 • Items do not have to have similar formats • Mix & match T-F, nominal, multiple choice (varying # of options), etc.

  7. Rule 3 • Tests may be of different lengths • The new reliability: test information • Adaptive testing  Tailored to individuals

  8. Rule 4 • Tests may be used to model cognitive and learning processes • Construct representation vs. construct irrelevant variance • Multi-component latent trait model of Embretson

  9. Rule 5 • Reliability is less important than precision of measurement • Reliability is an average over a well-defined population

  10. Rule 6 • Scale distortion in the measurement of change due to guessing can be reduced with the appropriate psychometric model

  11. Rule 7 • Construction of mental representations (e.g., neural networks) from psychometric data • Concept similarity judgments

  12. Two Versions of a Question • P1. You are a doctor testing a blood-born disease. You know that in the overall population, 2 out of 100 people have the disease. All positives are accurately detected. You also know that the test returns a positive result for 5 out of 100 people tested who do not have the disease. Portions of the related contingency table are given below. What is the probability that a patient will test positive? • 0.02 • 0.05*0.98 • 0.02 + 0.05*0.98 (Correct) • 0.95*0.98 • 0.02+0.05 (omitted from P1)

  13. The 2PL IRT Curves

  14. Nominal Response Curves

  15. 16 16 16 16 14 14 14 14 12 12 12 12 10 10 10 10 Frequency Frequency Frequency 8 8 8 Frequency 8 6 6 6 6 4 4 4 4 2 2 2 2 0 0 0 0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 Scores Scores Scores Scores One more… G6. The following are histograms of quiz scores for four different classes. Which distribution shows the most variability?

  16. Nominal Response Curve G6

  17. Criterion Validity: Comparing Chemistry CI scores to AP scores

  18. Quartile scores

  19. Nominal Discrimination

  20. Empirical Item Response Curves (“IRT for dummies”)

  21. Item Characteristic Curve Item Information function (solid) and Standard error (dash)

  22. Test information function and Standard error

  23. Comparison of IRT ability estimates and number correct

  24. Hanging Rootogram (a la Tukey) test of normality

More Related