Validity

1. Validity

2. Today�s Objectives What is validity and what are the 4 broad categories of methods for establishing validity evidence? Why is the base rate of the predictor and of the criterion important considerations in establishing criterion validity? Define construct and name the methods for establishing construct validity What are some difficulties in interpreting differences between pre- and post-test scores?

3. Validity Validity refers to an estimate of how adequately a test measures what it purports to measure

4. Validation Validity Evidence: Face Validity Content Validity Criterion Validity Construct Validity

5. Face Validity To the person being tested, what does the test appear to be measuring?

6. Content Validity Extent to which the measure represents all facets for a given construct. Are the test items representative with regard to the application context?

7. Criterion-Related Validity Extent to which test score can be used to infer an individual�s most probable standing on some criterion (i.e., measure of interest) Concurrent versus Predictive

8. Criterion-Related Validity Validity coefficient Restricted range Incremental validity Degree to which additional predictor explains something about the criterion measure that is not explained by the predictors already in use

9. Criterion-Related Validity

10. Construct Validity Construct: Informed scientific idea hypothesized to describe behavior Unobservable & Presupposed Underlies differences in behavior (such as differences in test performance, or such as differences on the criterion variable) Useful in that proposed construct should pay its way by doing some work (cf. Occam�s razor, principle of parsimony)

11. Construct Validity Construct Validity: Degree to which inferences can legitimately be made from the operationalizations in your study to the theoretical constructs on which those operationalizations were based. Reification Error: If the only way you can 'prove' the existence of something is through language, then does that something exist in reality? Could it be only word play?

12. Construct Validity Construct Validity: If empirical results are contrary to theory, then: A. Test does not measure the construct B. Theory is flawed C. Method was flawed

13. Construct Validity Construct Validity Evidence: Homogeneity Age related changes Pretest/Posttest changes Contrasted groups evidence Convergent Evidence Discriminant Evidence Multitrait-Multimethod Matrix

14. Construct Validity Homogeneity Item analysis: Relationship between test takers� scores on individual items and their score on the entire test

15. Construct Validity Age Related Some constructs lend themselves to the prediction that the construct will change predictably over time Reading ability versus marital satisfaction

16. Construct Validity Pretest/Posttest Compare scores on a pretest with a posttest

17. Construct Validity Rival explanations for significant Pre/Post test differences: Regression toward the mean Instrumentation History Maturation Mortality Test experience

18. Construct Validity Method of Contrasted Groups Demonstrates that scores on a test predictably vary as a function of group membership

19. Construct Validity Convergent Validity: Correlation of test with other tests that measure the construct (or related constructs) Spurious convergence: Positively correlated measures may be related only because they share the same method of measurement (i.e., they share common method variance)

20. Construct Validity Discriminant Evidence Correlation between test scores and other measures that should NOT theoretically be related

21. Construct Validity Multitrait-Multimethod Matrix (Campbell & Fiske, 1959) Two or more traits, two or more methods of measurement examined for convergent and divergent validity Data for each trait indicator obtained by multiple methods and/or from multiple sources