Validity

1. Validity Notes Chapter 4

2. Overview • Validity and reliability are independent of each other • Validity = Accuracy • Reliability = Precision • Marksman example • Validity and reliability are continuous • Errors in measurement are always either the result of • Systematic biasing of your scale (validity) • Random error introduced by your scale (reliability)

3. Validity • Four major types of experimental validity (Cook & Campbell, 1979) • Statistical conclusion validity • Internal validity • External validity • Construct validity • Most concerned with when creating a scale • Extent to which the measurements taken in a study properly represent the underlying theoretical construct

4. Construct Validity • Measures the match between a variable representing a “true” measure of the construct and the scale responses • Applies only when attempting to relate a scale to a theoretical construct • May have validity for one purpose but not for another

5. Criterion Validity (continued) • When there is an objectively correct way to measure an underlying construct your scale was designed to represent • Demonstrate your scale is related to the correct measure • When there is no objective measurement there is no single procedure to measure validity • Build an argument for how to interpret the scale by demonstrating that measurements are consistent with the theoretical variable motivating the responses

6. Face Validity • Items composing the scale are logically related to the underlying construct • Scale “looks” appropriate

7. Convergent Validity • Most important to demonstrate • Shows that the responses to your scale are related to other measurements that are supposed to be affected by the same variable • Assess it numerous ways • Each time you demonstrate consistency with the underlying construct makes a more convincing argument that your scale provides an accurate representation of that construct

8. Divergent Validity • Demonstrating that your scale is not related to measurements that represent different variables • Shows that your scale is measuring a new concept • Assess it at the same time as convergent validity • Unclear whether the relation does not exist or your study lacked enough power to detect it • If you show significant relations in your study, it makes the argument that the nonsignificant findings in your divergent validity are not due to faults in your study

9. Unique Utility • Demonstrate that your scale does something beyond similar measures that already exist • Shows that your scale can explain unique portions of the variance • Can cause people to use or not use your scale

10. Final Points on Validity • Validity and reliability are independent concepts but are related in important ways • Difficult to determine validity of a highly unreliable scale • Involve showing statistically significant relations between the scale and other measures • Validity measures how successfully your scale matches onto the theory you propose • Failure to validate the scale does not mean there is something wrong with your scale • May indicate there is something wrong with the theory underlying the validation