Reliability Presentation Test-Retest

1. ReliabilityPresentationTest-Retest James Blackwood – AED 615 Fall Semester 2006

2. Test-Retest Reliability • Test-Retest method of determining reliability is accomplished by administering a test to a group • After a period of time has passed, the same test is re-administered to the same group. • It is also known as stability reliability • It is used in both qualitative and quantitative research (qualitative requires a different technique of analysis)

3. Reliability Coefficient • After two tests have been administered, a reliability coefficient is calculated to determine the relationship between the two scores obtained. • If the same results are obtained from the two tests then the coefficient = 1.0 • The coefficient is influenced by the amount of time that has passed between the administration of the two tests.

4. Reliability Coefficient Measurement • The reliability coefficient is expected to be lower the longer the time interval between the tests due to the possibility of changes in the population taking the test. • The shorter the time gap, the higher the correlation; the longer the time gap, the lower the correlation.

5. Test-Retest Issues • There cannot be any measurable change in the construct being measured between the two tests. • This method will not work when measuring a variable that is not stable in an individual. • Unless the instrument is reliable, relationships with other variables in the study will not be identified.

6. Testing.. • You can obtain considerably different estimates of reliability depending on the interval between tests. • For educational research, examination of scores over a two to three month period is sufficient for test-retest reliability verification. • The time interval between the two tests should always be reported when using test-retest as a measure of reliability.

7. Test-Retest Equation

8. Test-Retest Issues • Requires twice the data collection • Population for the test would need to be willing to repeat the test (impractical) • Higher costs due to multiple tests being administered • Only works well when practical (better for smaller population rather than a large one) • Researcher may not be able to retest the population

9. Examples of Test-Retest • Educational assessment • Drug testing • Testing Measurement Equipment • Medical Evaluations

10. Research Literature Example American Journal of Agricultural Economics Volume 71 - Number 1 (Feb 1989), pp 76-84 Test-Retest Reliability of the Contingent Valuation Method: A Comparison of General Population and Visitor Responses John B. Loomis Abstract: The reliability of the contingent valuation method is evaluated by resurveying the same general households and visitors nine months after their original survey. Test-retest correlations on willingness to pay are statistically significant and ranged from .422 for the general population sample to .782 for the visitor sample. Using a paired T-test, there was no statistical difference between an individual's first and second reported willingness to pay. Chow tests comparing the original and resurvey willingness-to-pay functions showed no statistical difference at the .01 level. Reported willingness to pay is reasonably stable over the time period surveyed.

11. References Fraenkel, J. R., & Wallen, N. E. (2006). How to design and evaluate research in education (6th ed.). New York: Mc-Graw-Hill. Guttman, L. (1946). The test-retest reliability of qualitative data. Psychometrika, 11(2), 81-95. Abstract retrieved October 15, 2006 from Springer Link database. Loomis, J. B. (1989) Test-retest reliability of the contingent valuation method: A comparison of general population and visitor responses. American Journal of Agricultural Economics, 71(1), 78-84. Trochim, W. M., (2006). Types of reliability. Research Methods Knowledge Base. Retrieved from http://www.socialresearchmethods.net/kb/reltypes.htm