1 / 31

Class 4 Experimental Studies: Validity Issues Reliability of Instruments

Class 4 Experimental Studies: Validity Issues Reliability of Instruments. Chapters 7 Spring 2017. Dependent Variables: Continuous. Psychometric Properties Reliability Validity. Is reliability a property of :. The psychological instrument ( e.g. depression)

vandana
Download Presentation

Class 4 Experimental Studies: Validity Issues Reliability of Instruments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Class 4Experimental Studies: Validity IssuesReliability of Instruments Chapters 7 Spring 2017

  2. Dependent Variables: Continuous • Psychometric Properties • Reliability • Validity

  3. Is reliability a property of : • The psychological instrument (e.g. depression) • Scores in the psychological instrument

  4. Reliability The extent to which scores show true variance in attributes within or between participants as opposed to error variance A score = true variance + error variance More items of good quality = higher reliability One-item scales have very low internal reliability; estimated around r =.25

  5. Sources of Measurement Error Specific error something unique about the instrument that differs from what the researcher intended (e.g. social desirability; reading level; idioms) Transient error some temporary factor that affects the measurement (e.g. order of instruments; historical events; noise while observations occur; tiredness; inattention)

  6. Types of Measures • Observational Measures • Self-Report Paper-and-Pencil Measures • Content tests right/wrong answers • Likert-scales (3 to 7 response options)

  7. Types of Reliability • Inter-Scorer Agreement - Observation Measures • Test/Re-Test • Alternate Forms (achievement) • Internal Consistency • how items correlate with each other

  8. Internal Consistency Reliability • Split-Half • Kuder-Richardson • Dichotomous items- right /wrong; Yes/No • Chronbach Alpha α • Average correlation of all possible split-half reliability calculations

  9. Internet Addiction Measure • 12-item measure rx= .70 to rx= .95 among college students • Expected reliability estimates among the adolescent sample • Three-item measure: rx=.40 to rx=.55

  10. Internet Addiction: Q 2a • 12-item measure: internal consistency reliability coefficients rx= .70 to rx= .95 among college students • 70% to 95% of variability in respondents’ scores is due to and the rest is due to .

  11. Reliability Estimates Extent to which variability is due to true variation versus error Cronbach alpha =. 70; 70% of variation in scores is due to true differences in internet addiction & 30% is due to error Cronbach alpha =. 95; 95% of variation in scores is due to true differences in internet addiction & 5% is due to error

  12. 12-item measure rx= .70 to rx= .95 among college-age samples in the U.S. Expected reliability estimates among any adolescent sample: • At least .70 • Between .70 and .95 • Unkown

  13. Reliability Refers to scores with specific and . It’s a property of not of the instruments.

  14. Reliability Refers to scores with specificpopulations andconditions. It’s a property of scoresnot of the instrument per se.

  15. More accurate? • The internal consistency of the Internet Addiction Scale (IAS) has ranged from .70 to .95. • The internal consistency of scores in the Internet Addiction Scale (IAS) with college students in the U.S. has ranged from .70 to .95.

  16. Wellbeing Measure: Q 2c • Three-item measure: rx=.40 to rx=.45. To improve the scores’ reliabilityjustadd 5 or 6 items: True False Not sure

  17. Internet Addiction Measure: Q 2c • Three-item measure: rx=.40 to rx=.45 – New items increase reliability only if they are ofgood quality

  18. Reliability and Correlation: • In correlational research, how does the reliability of two scores(e.g. depression and self -esteem)affect the probability that the observed correlation coefficient between scores in the two variables approximates the “true” correlation coefficientin the population?

  19. Internal Reliability and Correlation Depression 1 Cronbachα = .45 Depression 2 Cronbach α= .90 Same sample: which r dep-se below will be larger? DEP1 (α= .45) correl. Self Esteem DEP2 (α= .90) correl. Self Esteem

  20. Validity of Measures Construct Validity Factor structure -- latent constructs Convergent and Discriminate Validity Correlation with similar/dis-similar measures Predictive Validity Correlation among different constructs based on expected relations Cross-sectional or Longitudinal

  21. Reliability vs. Validity Observed correlation coefficient will be smaller and less accurate with the less reliable measure Correlations between constructs are attenuated by the (internal) reliability of the measures The reliability of a measure puts a ceiling on its validity

  22. Validity Of Experimental Designs Do inferences from an outcome study results reflect how things actually are in the population? Does the manipulation (treatment) causes the observed outcome? vs. other reasons explains the findings Threats to ExperimentalValidity

More Related