Understanding Hypothesis Testing: Key Concepts, Significance, and Errors

1. Chapter 4 Hypothesis Testing, Power, and Control: A Review of the Basics

2. From Question to Hypothesis • Finding the TRUTH starts with asking a question that comes from • Curiosity • Necessity • Past Research • As scientists we PREDICT the answer from • Theory • Past Research • Common Sense • That prediction is EDUCATED not random • An educated prediction is a HYPOTHESIS • To ANSWER the question we TEST the HYPOTHESIS

3. Three levels of hypotheses • Conceptual hypotheses • State expected relationships among concepts. • Research hypotheses • Concepts are operationalized so that they are measurable. • Statistical hypotheses • State the expected relationship between or among summary values of populations, called parameters. • Null hypothesis (H0) • Alternative hypothesis (H1)

4. Example • Question – What is the role of neurotransmitters in memory? • Conceptual – Increasing certain neurotransmitter will increase memory • Research – Smoking 1 crack rock before testing will increase performance on a standard test of memory compared to placebo control • Statistical – HO: MT = MC HA: MT >MC

5. Testing the null hypothesis • Null hypothesis • The hypothesis being statistically tested when you use inferential statistics. • The researcher hopes to show that the null is not likely to be true (i.e.. hopes to nullify it). • Alternative hypothesis • The hypothesis the researcher postulated at the outset of the study. • If the researcher can show that the null is not supported by the data, then he or she is able to accept the alternative hypothesis.

6. Testing the null hypothesis • Steps in testing a research hypothesis: • State the null and the alternative. • Collect the data and conduct the appropriate statistical analysis. • Reject the null and accept the alternative or fail to reject the null. • State your inferential conclusion.

7. Statistical significance • Statistical difference • The probability that the groups are the same is very low. • Significance levels (α) • Alpha (α) is the level of significance chosen by the researcher to evaluate the null hypothesis. • 5% or 1%

8. Inferential Errors: Type I and Type II • Type I Error • Rejecting a true null. • Probability is equal to alpha (α). • Type II Error • Failing to reject a false null. • Probability is beta (β). • Power – our ability to reject false nulls.

9. Inferential Errors: Type I and Type II Our decision

10. Why Power is Important • A powerful test of the null is more likely to lead us to reject false nulls than a less powerful test. • Powerful tests are more sensitive than less powerful tests to differences between the actual outcome (what you found) and the expected outcome (null hypothesis). • Power, or the probability of rejecting a false null, is 1 – β.

11. Power and How to Increase it • How one measures variables • Interval or ratio scales are better • In testing the effects of alcohol intoxication on aggression… • Intoxication – BAC better than # of drinks • Aggression – Level of shock (1-10) as opposed to shock or no shock

12. Power and How to Increase it • Use more powerful statistical analyses • Parametric vs. Nonparametric • ANOVA vs. Chi-Square

13. Power and How to Increase it • Use designs that provide good control over extraneous variables. • Remove unintended variation • Experimental vs. Correlational Designs • Laboratory vs. Field

14. Power and How to Increase it • Restrict your sample to a specific group of individuals. • Use selection procedures to reduce nuisance variables

15. Power and How to Increase it • Increase your sample size  reduces error variance

16. Power and How to Increase it • Maximize treatment manipulation • Precision • Separation

17. Effect size • Effect size – a measure of the strength of the relationship between/among variables. • Effect size helps us determine if differences are not only statistically significant, but also whether they are important. • Powerful tests should be considered to be tests that detect large effects.

18. Effect size • Ways to calculate effect size: • Cohen’s d – use with t-tests. • Coefficient of determination (r2) – use with correlations. • eta-squared (η2) – use with ANOVAs. • Cramer’s v – use with Chi-square analyses.

19. Power and the role of replication in research • Power increases when we replicate findings in a new study with different participants in a different setting.

20. External and internal validity • External validity • When the findings of a study can be generalized to other populations and settings. • Internal validity • Refers to the validity of the measures within the study. • The internal validity of an experiment is directly related to the researcher’s control of extraneous variables.

21. Confounding and extraneous variables • Extraneous variable • A variable that may affect the outcome of a study but was not manipulated by the researcher. • Confounding variable • A variable that is systematically related to the independent and dependent variable. • Spurious effect • An outcome that was influenced not by the independent variable itself but rather by a variable that was confounded with the independent variable.

22. Confounding and extraneous variables • Controlled variable • A variable that the researcher takes into account when designing the research study or experiment. • Nuisance variables • Variables that contribute variance to our dependent measures and cloud the results.

23. Controlling extraneous variables • Elimination • Get rid of the extraneous variables completely (e.g.. by conducting research in a lab). • Constancy • Keep the various parts of the experiment constant (e.g.. instructions, measuring instruments, questions). • Secondary variable as an IV • Make variables other than the primary IV secondary variables to study (e.g.. gender).

24. Controlling extraneous variables • Randomization: Random assignment of participants to groups • Randomly assigning participants to each of the treatment conditions so that we can assume the groups are initially equivalent. • Repeated measures • Use the same participants in all conditions. • Statistical control • Treat the extraneous variable as a covariate and use statistical procedures to remove it from the analysis.