Hypothesis Testing and Results Interpretation

1. Hypothesis Testing and Results Interpretation By Minjuan Wang ED 690 Educational Technology

2. Types of Hypothesis • Null hypothesis • There is no change, difference or relationship between A and B • A starting point or a benchmark • Many articles have implied null hypothesis, but may not clearly stated. • Few studies are designed to verify the nonexistence of a relationship • Coffee maker is broken • no relationship between its broken and the humming birds on the tree outside my balcony.

3. Alternative/Research Hypothesis • Hypothesis that is implicitly accepted if the null hypothesis is rejected • Directional (one-tailed test) • Non-directional (two-tailed test) • Hypothesis: not to be proven but to be supported • Does your study fail if your hypothesis is not supported by the data?

4. Hypothesis testing • Core of Inferential Statistics • contributes to the science of education primarily by expanding, refining, or revising its knowledge base.

5. Hypothesis Testing Procedure • Come up with a hypothesis • Set a: level of risk you are willing to take; or the cut-off point for a test result to be significant • Select the test • Compute the obtained value: t, f, r, etc. • Find the critical value in the respective table • To reject a null hypothesis->Obtained value must be > critical value • Otherwise, fail to reject null hypothesis (H0) • But, never “accept” null hypothesis • Many tests are needed to confirm that A is not different from or associated with B. • When rejecting null-P, the alternative 2-tailed P is implicitly accepted.

6. P from Fancy Schmancy Software • Inferential statistics • T, ANOVA, Correlation, Regression • P is the probability of chance (indicator of significance) • Free us from the test tables • Results vary • P<.05 • P<.001 • P=.013 (the exact probability of the outcome/effect due to chance—SPSS) • Outcome: difference, change, or association • P>.05 or p=ns (nonsignificant) • The probability of rejecting a null-P exceeds 0.05 (the cut-off point) (Salkind) • So reject it

7. Examples & Exercise • Scenario: • two groups of patients: anti-depression drug group; and placebo group • Run t for two (null-HP) • t(58)=2.45, p<.05 • t: the test that was used • 58: degree of freedom • 2.45: the obtained value • P: the probability of chance is within the cut-off point (level of significance/or risks allowed) • Significant mood difference exists between the two groups • So the treatment (drug) did work

8. Examples & Exercise • Scenario: • two groups of patients: anti-depression drug group; and placebo group • Run t for two • t(58)=0.14, p>.05 • t: the test that was used • 58: degree of freedom • 0.14: the obtained value • The probability of rejecting the null-p exceeds the cut-off point, so reject it! • Also means, the probability of chance exceeds the cut-off point (level of significance/or risks allowed) • No significant mood difference exists between the two groups • So the treatment (drug) did not work

9. Examples & Exercise • Scenario: • two groups of patients: anti-depression drug group; and placebo group • Run t for two • t(58)=0.14, p=.891 • t: the test that was used • 58: degree of freedom • 0.14: the obtained value < critical value 2.001 • Not extreme enough for us to conclude the difference is due to treatment • P: the exact probability that the outcome (difference) is due to chance • Also means, the probability of chance exceeds the cut-off point (level of significance/or risks allowed) • No significant mood difference exists between the two groups • So the treatment (drug) did not work

10. Shaprio-W Normality Test • Seems to have a different interpretation • But only used in Analyse-it • The higher the p, the more normal the distribution is.