1 / 39

Types of Reliability

Types of Reliability. Reliability of Consistency of What?. Observers or raters Tests over time Different versions of the same test A test at one point in time. Inter-Rater or Inter-Observer Reliability. Object or phenomenon. Inter-Rater or Inter-Observer Reliability. Object or

jkyle
Download Presentation

Types of Reliability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Types of Reliability

  2. Reliability of Consistency of What? • Observers or raters • Tests over time • Different versions of the same test • A test at one point in time

  3. Inter-Rater or Inter-Observer Reliability Object or phenomenon

  4. Inter-Rater or Inter-Observer Reliability Object or phenomenon Observer 1

  5. Inter-Rater or Inter-Observer Reliability Object or phenomenon Observer 1 Observer 2

  6. Inter-Rater or Inter-Observer Reliability Object or phenomenon ? = Observer 1 Observer 2

  7. Inter-Rater or Inter-Observer Reliability • Are different observers consistent? • Can establish this outside of your study in a pilot study. • Can look at percent of agreement (especially with category ratings). • Can use correlation (with continuous ratings).

  8. Test-Retest Reliability Time 1 Time 2

  9. Test-Retest Reliability Test Test = Time 1 Time 2

  10. Test-Retest Reliability Stability over time Test Test = Time 1 Time 2

  11. Test-Retest Reliability • Measure instrument at two times for multiple persons. • Compute correlation between the two measures. • Assumes there is no change in the underlying trait between time 1 and time 2.

  12. Parallel-Forms Reliability Time 1 Time 2

  13. Parallel-Forms Reliability Form A = Form B Time 1 Time 2

  14. Parallel-Forms Reliability Stability across forms Form A = Form B Time 1 Time 2

  15. Parallel-Forms Reliability • Administer both forms to the same people. • Get correlation between the two forms. • Usually done in educational contexts where you need alternative forms because of the frequency of retesting and where you can sample from lots of equivalent questions.

  16. Internal Consistency Reliability Average inter-item correlation

  17. Internal Consistency Reliability Average Inter-Item Correlation Test

  18. Internal Consistency Reliability Item 1 Average Inter-Item correlation Item 2 Item 3 Test Item 4 Item 5 Item 6

  19. Internal Consistency Reliability Item 1 Average inter-item correlation Item 2 I1 I2 I3 I4 I5 I6 1.00 .89 1.00 .91 .92 1.00 .88 .93 .95 1.00 .84 .86 .92 .85 1.00 .88 .91 .95 .87 .85 1.00 I1 I2 I3 I4 I5 I6 Item 3 Test Item 4 Item 5 Item 6

  20. Internal Consistency Reliability Item 1 Average inter-item correlation Item 2 I1 I2 I3 I4 I5 I6 1.00 .89 1.00 .91 .92 1.00 .88 .93 .95 1.00 .84 .86 .92 .85 1.00 .88 .91 .95 .87 .85 1.00 I1 I2 I3 I4 I5 I6 Item 3 Test Item 4 Item 5 Item 6 .90

  21. Internal Consistency Reliability Average item-totalcorrelation

  22. Internal Consistency Reliability Average item-totalcorrelation Test

  23. Internal Consistency Reliability Average item-totalcorrelation Item 1 Item 2 Item 3 Test Item 4 Item 5 Item 6

  24. Internal Consistency Reliability Average item-totalcorrelation Item 1 I1 I2 I3 I4 I5 I6 Item 2 I1 I2 I3 I4 I5 I6 Total 1.00 .89 1.00 .91 .92 1.00 .88 .93 .95 1.00 .84 .86 .92 .85 1.00 .88 .91 .95 .87 .85 1.00 .84 .88 .86 .87 .83 .82 1.00 Item 3 Test Item 4 Item 5 Item 6

  25. Internal Consistency Reliability Average item-totalcorrelation Item 1 I1 I2 I3 I4 I5 I6 Item 2 I1 I2 I3 I4 I5 I6 Total 1.00 .89 1.00 .91 .92 1.00 .88 .93 .95 1.00 .84 .86 .92 .85 1.00 .88 .91 .95 .87 .85 1.00 .84 .88 .86 .87 .83 .82 1.00 Item 3 Test Item 4 Item 5 Item 6 .85

  26. Internal Consistency Reliability Split-half correlations

  27. Internal Consistency Reliability Split-half correlations Test

  28. Internal Consistency Reliability Split-half correlations Item 1 Item 2 Item 3 Test Item 4 Item 5 Item 6

  29. Internal Consistency Reliability Split-half correlations Item 1 Item 1 Item 3 Item 4 Item 2 Item 3 Test Item 4 Item 5 Item 6

  30. Internal Consistency Reliability Split-half correlations Item 1 Item 1 Item 3 Item 4 Item 2 Item 3 Test Item 4 Item 5 Item 2 Item 5 Item 6 Item 6

  31. Internal Consistency Reliability Split-half correlations Item 1 Item 1 Item 3 Item 4 Item 2 Item 3 Test .87 Item 4 Item 5 Item 2 Item 5 Item 6 Item 6

  32. Internal Consistency Reliability Cronbach’s alpha ()

  33. Internal Consistency Reliability Cronbach’s alpha () Test

  34. Internal Consistency Reliability Cronbach’s alpha () Item 1 Item 2 Item 3 Test Item 4 Item 5 Item 6

  35. item 1 item 3 item 4 item 1 item 3 item 4 item 1 item 3 item 4 .87 .87 .87 item 2 item 5 item 6 item 2 item 5 item 6 item 2 item 5 item 6 Internal Consistency Reliability Cronbach’s alpha () Item 1 Item 2 Item 3 Test Item 4 Item 5 Item 6

  36. item 1 item 3 item 4 item 1 item 3 item 4 item 1 item 3 item 4 .87 .87 .87 item 2 item 5 item 6 item 2 item 5 item 6 item 2 item 5 item 6 Internal Consistency Reliability Cronbach’s alpha () Item 1 Item 2 Item 3 Test SH1 .87 SH2 .85 SH3 .91 SH4 .83 SH5 .86 ... SHn .85 Item 4 Item 5 Item 6

  37. item 1 item 3 item 4 item 1 item 3 item 4 item 1 item 3 item 4 .87 .87 .87 item 2 item 5 item 6 item 2 item 5 item 6 item 2 item 5 item 6 Internal Consistency Reliability Cronbach’s alpha () Item 1 Item 2 Item 3 Test SH1 .87 SH2 .85 SH3 .91 SH4 .83 SH5 .86 ... SHn .85 Item 4 Item 5 Item 6  = .85

  38. item 1 item 3 item 4 item 1 item 3 item 4 item 1 item 3 item 4 .87 .87 .87 item 2 item 5 item 6 item 2 item 5 item 6 item 2 item 5 item 6 Internal Consistency Reliability Cronbach’s alpha () Item 1 Item 2 Item 3 Test SH1 .87 SH2 .85 SH3 .91 SH4 .83 SH5 .86 ... SHn .85 Like the average of all possible split-half correlations Item 4 Item 5 Item 6  = .85

  39. Internal Consistency Reliability - Summary • Average inter-item correlation • Average item-total correlation • Split-half reliability • Cronbach’s alpha ()

More Related