Measurement Error

1. Measurement Error

2. Rating Sheet 1 Manage time effectively 2 Manage resources effectively. 3 Scan a multitude of information and decide what is important. 1 2 3 4 5 4 Decide how to manage multiple tasks. 5 Organize the work when directions are not specific. 1 2 3 4 5 1 2 3 4 5 3 Scan a multitude of information and decide what is important. 1 Manage time effectively 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 True Score Theory Observed score True ability Random error + = X e + T

3. The Error Component X e + T Two components:

4. The Error Component X e + T Two components: er

5. The Error Component X e + T Two components: er • Random error

6. The Error Component X e + T Two components: er • Random error es

7. The Error Component X e + T Two components: er • Random error es • Systematic error

8. The Revised True Score Model X er es + + T

9. What Is Random Error? • Any factors that randomly affect measurement of the variable across the sample. • For instance, each person’s mood can inflate or deflate performance on any occasion. • Random error adds variability to the data but does not affect average performance for the group.

10. Random Error Frequency The distribution of X with no random error X

11. Random Error The distribution of X with random error Frequency The distribution of X with no random error X

12. Random Error The distribution of X with random error Frequency The distribution of X with no random error Notice that random error doesn’t affect the average, only the variability around the average. X

13. What Is Systematic Error?

14. Any factors that systematically affect measurement of the variable across the sample. • Systematic error = bias. • For instance, asking questions that start “do you agree with right-wing fascists that...” will tend to yield a systematic lower agreement rate. • Systematic error does affect average performance for the group.

15. Systematic Error Frequency The distribution of X with no systematic error X

16. Systematic Error The distribution of X with systematic error Frequency The distribution of X with no systematic error X

17. Systematic Error The distribution of X with systematic error Frequency The distribution of X with no systematic error Notice that systematic error does affect the average; we call this a bias. X

18. Reducing Measurement Error • Pilot test your instruments -- get feedback from respondents. • Train your interviewers or observers • Make observation/measurement as unobtrusive as possible. • Double-check your data. • Triangulate across several measures that might have different biases.