1 / 12

Lecture 7: reliability & validity

Lecture 7: reliability & validity. Aims & objectives This lecture will explore a variety of techniques for ensuring that research is conducted with reliable and valid measures You should understand internal and external reliability and a variety if techniques for ensuring validity. Definitions.

shandra
Download Presentation

Lecture 7: reliability & validity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 7: reliability & validity • Aims & objectives • This lecture will explore a variety of techniques for ensuring that research is conducted with reliable and valid measures • You should understand internal and external reliability and a variety if techniques for ensuring validity

  2. Definitions • ‘Reliability is the agreement between two efforts to measures the same trait through maximally similar methods. Validity is represented in the agreement between two attempts to measure the same trait through maximally different methods.’ (Campbell & Fiske, 1959) • Reliability is a necessary but not sufficient condition for validity

  3. Types of research/practice • Clinical, educational, research, forensic, occupational • Questionnaire development and use • Private, face to face • Interviews • Face to face, telephone

  4. Ethics/morals • Need to ensure that the measures being used and reliable and valid • Equal opportunities and cultural biases • Stability over time

  5. Classical theory of error measurement • ACTUAL score = TRUE score + Error score • Standard error of measurement • Universe of items • All items correlate to some extent with the true score • Reliability s related to the average correlation between items and test length

  6. Reliability • Internal • Coefficient alpha • Split halves • Parallel forms • External • Test-retest (correlations and ANOVA) • Inter-rater • Kappa • Agreement does not imply accuracy • Intra-rater

  7. Sources of unreliability • Guessing • Ambiguous items • Test length • Instructions • Temperature, illness • Item order effects • Response rate • Social desirability

  8. Item generation • Past research • From subject populations • Unambiguous • Address a specific issue • Simple language • No jargon • Clear instructions

  9. Context effects • Affected by the nature of the questions, the order of questions and the type of response scale use. • Example (availability & accessibility) • Control: General > dating (r = .12) • Prime: Dating > general (r = .66)

  10. Validity • Faith • Face • Content • Construct • Predictive

  11. A model Task Specific Global Retrieve info Mood Select comparison Use mood Social aspect Public Private

  12. Generalizability theory • Basically, a test once shown to be reliable is not always going to be reliable • Culture changes • Time of day effects • Time of year effects • Use ANOVA procedure to show under what circumstances a test is reliable

More Related