Variation, Validity, & Variables

1. Variation, Validity, & Variables Lesson 3

2. Research Methods & Statistics • Integral relationship • Must consider both during planning • Research Methods • How data are collected • What kind of data • Statistics • Analysis & interpretation depends on data & how it is collected ~

3. Scientific Validity • Scientific conclusions • About relationships b/n variables • Validity • Soundness, legitimacy, truth • Internal validity • About cause & effect • External (ecological) validity • About broad applicability ~

4. How are data collected? • 2 scientific approaches • Same or similar statistical analysis • NOT same confidence in conclusions • Observational methods • Observe co-occurrence of variables • Naturalistic observation, case studies, archival research, surveys, etc. • Experimental method • Manipulate a variable  observe effect on another variable ~

5. The Experimental Method • At least 2 variables: • Independent (IV) & Dependent (DV) • At least 2 groups (levels of IV) • control group - no treatment • experimental - receives treatment • random assignment to groups • Control extraneous variables • Which might also affect DV • Weakens internal validity ~

6. Experimental Variables • Independent (IV) • Predictor (or cause) • Manipulated • Dependent (DV) • Outcome (or effect) • Measured • Extraneous variables • Or confounding • Might also affect outcome (DV) ~

7. Variation within an Experiment • Systematic • Variation due to manipulation of IV • Difference between groups • Unsystematic • Individual differences • Variation due to random or uncontrolled variables • Potentially confounding variables ~

8. Variation within an Experiment

9. Internal Validity • Legitimacy of conclusions • about cause & effect • High internal validity • Confident that only changes in IV cause change in DV • Low internal validity • Confounding variables influence outcome ~

10. Randomization • Important for validity • Helps avoid bias • Random sampling (or selection) • Selection of participants for study • Representative sample from population •  external validity • Random assignment to condition (groups) • Minimize biasing of groups •  internal validity ~

11. Observational vs. Experimental • Internal vs External validity • Inverse relationship based on control • Observational? •  internal vs  external • Cannot determine causality • Experimental •  internal vs  external • Establishes cause & effect relationships • For useful conclusions need both ~

12. Observational vs. Experimental:Statistical Methods • Misperception • Observational  only correlational • Experiment  hypothesis tests • Method not sole determinant of analysis • Strength of cause & effect conclusions • Observational  weaker • Experiment  stronger ~

13. Planning Research • Observational or experimental research • Research design • Between-groups or within-subjects • Operational definition of variables • Data categorical or quantitative • Statistical analysis • Depends on all of the above ~

14. What are data? • Information from measurement • datum = single observation • Variables • Dimensions that can take on different values • IQ, height, shoe size, hair color • Is not the same for all individuals being measured ~

15. Measuring Variables • Operational definitions • Variables often abstract • Intelligence, anxiety, fitness, etc. • Need to objectively measure • Hypothesis: Exercise increases fitness • Independent: Exercise • Operational definition? • Dependent: fitness • Operational definition? ~

16. Levels of Measurement • Limits type of statistical analysis possible • Qualitative • Categorical • Frequency data • Discrete: only whole numbers • Quantitative • Continuous or discrete • represents magnitude • infinite # intermediate values ~

17. Levels of Measurement: Categorical • Nominal scale • categorical • order NOT meaningful • can assign arbitrary values • Ordinal scale • Categorical + meaningful order • No info about magnitude of differences • If assign numerical value, must reflect order ~

18. Levels of Measurement: Quantitative • Interval scale (numbers) • Continuous or discrete • Equal intervals  equal differences • Ratio scale • same characteristics as interval • Ratios of values must be meaningful for magnitude • scale must have true zero point • Most statistics: interval/ratio treated the same ~

19. Levels of Measurement: SPSS • Variable view tab • Formatting of variable • Measure • Nominal scale • Ordinal scale • Scale • Interval & ratio • Reminder: IV must be nominal for most statistical tests ~

20. Measurement Error • Discrepancy • between actual value of observation and the reported value • Sources of measurement error • Sensitivity of measuring instrument • Conscientiousness of observer • Surveys: inaccurate or untruthful • Low reliability of instrument •  unsystematic variation ~

21. Reliability & Validity • Accurate measurement requires both • Reliability • Consistency of measurement • Criterion validity • Extent instrument actually measures what it claims to measure • Score on IQ test measures intelligence? •  pulse rate a measure of fear? • Important for internal & external validity ~