1 / 18

Reliability

Today's Objectives. Define measurement error and describe how it interferes with psychological studiesKnow that reliability is the ratio of true variability to observed variabilityName major sources of error varianceDescribe methods for assessing reliability. Reliability. In psychometrics, reliab

ella
Download Presentation

Reliability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Reliability

    2. Today’s Objectives Define measurement error and describe how it interferes with psychological studies Know that reliability is the ratio of true variability to observed variability Name major sources of error variance Describe methods for assessing reliability

    3. Reliability In psychometrics, reliability refers to consistency in measurement

    4. Error Variance Score X is a reflection of the true score on the psychological construct being measured plus error. X = T + E

    5. Error Variance If we wanted to measure spelling ability, the best technique would be to systematically go through the dictionary and have a person spell every word, which would be the TRUE score for spelling ability. But, that would be a terrible way to spend the summer. Better method would be to take a random sample of words from the dictionary. Reliability analysis is estimating how much error we would make by using the score from the shorter test as an estimate of spelling ability.

    6. Error Variance s 2 = s T 2 + s E 2 + s S 2 The greater the proportion of variance attributed to the s E , the less reliable the test.

    7. Error Variance

    8. Error Variance This item is probably an unreliable measure of a person’s feelings toward iguanas. Why? Ample individual differences abound regarding feelings toward mating.

    9. Measuring Reliability Suppose you measured the height of every member of your family with a crude stick. The goal is to reliably distinguish between individuals. What would be more reliable: Measuring everyone once, or measuring each person 100 times and then taking the average across the 100 measurements as the final measure of each person’s height?

    10. Measuring Reliability Principle Numero Uno: The more items on a test, the better the reliability. This is because errors will be randomly distributed across items, and sum to zero. For example, when given “Flying iguanas are ugly” as a test item, people who are for/against mating will be randomly for/against flying.

    11. Measuring Reliability

    12. Measuring Reliability

    13. Sources of Error Variance Test construction Test administration Test scoring and interpretation

    14. Sources of Error Variance Misinterpretation Social Desirability Misunderstanding the Instructions Attention Seeking Forgetting

    15. Reliability Estimates Test-Retest Method The test is administered twice at two different points in time. Assumes that there will be no change in the quality or construct being measured. Test-retest reliability is best used for things that are stable over time, such as intelligence.

    16. Reliability Estimates Parallel-Forms/Alternate Forms Method Parallel tests have identical true scores and identical error variances. Alternate forms are two forms of a tests whose items are intended to measure the same thing and do not differ from each other in any systematic way. E.g., Watson-Glaser Critical thinking test items: Test A: Terry don't worry about it. You'll graduate someday. You're a college student. Right? And all college students graduate sooner or later. Test B: Charlie, don't worry about it. You'll get a promotion someday. You're working for a good company. Right?. And everyone who works for a good company gets a promotion sooner or later.

    17. Reliability Estimates Split Half Correlate scores from equivalent halves of a single test administered once. Goal is usually to create parallel forms. Divide the items on the test into equivalent halves Correlate the scores between the two halves Adjust the result to take into account that if you were to reduce the items, reliability would decrease (Spearman-Brown formula)

    18. Reliability Estimates Interrater Reliability Method Evaluate the consistency among judges who are rating the same behavior

    19. Summary

More Related