1 / 17

S519: Evaluation of Information Systems

S519: Evaluation of Information Systems. Social Statistics Inferential Statistics Chapter 16: reliability and validity. Last week. This week. What are reliability and validity. Reliability and validity.

ccleary
Download Presentation

S519: Evaluation of Information Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. S519: Evaluation of Information Systems Social Statistics Inferential Statistics Chapter 16: reliability and validity

  2. Last week

  3. This week • What are reliability and validity

  4. Reliability and validity • Leathers, S. (2003). Parental visiting, conflicting allegiances, and emotional and behavioral problems among foster children. Family Relations, 52, 53-63

  5. Your measurement • How do I know that the test, scale, instrument, etc., It works every time I use it (reliability)? • How do I know that the test, scale, instrument, etc., I use measures what it is supposed to (validity)?

  6. Scales of measurement • What is measurement: • The assignment of values to outcomes following a set of rules • The scales of measurement have four types: • Nominal, ordinal, interval and ratio

  7. Nominal level of measurement • An outcome can fit into one and only one class or category • E.g. gender, political affiliation • The least precise level of measurement • Categories should be mutually exclusive

  8. The ordinal level of measurement • Things are ordered • E.g., a rank of candidates for a job

  9. The interval level of measurement • Underlying continuum such that we can talk about how much more a higher performance is than a lesser one. • 10 words correct is twice as many as five words correct • 10 words correct is two more than eight correct and three more than five correct

  10. Ratio level of measurement • The presence of an absolute zero on the scale • In biological sciences, zero molecular movement, zero light • In social and behavioral sciences, it is a bit harder

  11. In sum • Any outcome can be assigned to one of the four scales of measurement • Scales of measurement have an order, from the least precise being nominal, to the most precise being ratio • The “higher up” the scale of measurement, the more precise the data being collected, and the more detailed and informative the data are

  12. In sum

  13. Reliability • Whether a test, or whatever you use as a measurement tool, measures something consistently

  14. Observed score • Observed score = true score + error score

  15. Some ways for reliability • Test-retest reliability • Want to examine whether a test is reliable over time • Compute the Pearson correlation coefficient on scores from a test at Time 1and Time 2

  16. Validity • The property of an assessment tool that indicates that the tool does what it says it does • Content validity • Validate through domain expert • Criterion validity • Validate your criteria with existing tests or criteria • Literature support

  17. More resources • Winning textbook: • http://www.psychstat.missouristate.edu/introbook/sbk00.htm • Good statistics course • http://surfstat.anu.edu.au/surfstat-home/surfstat-main.html • Statistical glossary • http://www.davidmlane.com/hyperstat/index.html

More Related