Review of Null, alternate, Type I and Type II

1. Review of Null, alternate, Type I and Type II • Null: Excessive CO emission does not cause global warming (climate change) • Alternate: Excessive CO emission causes global warming (climate change) • Type I error: False alarm, the null is right • Type II error: Miss, the alternate is right • Should you believe in the null or alternate? Which error (Type I and Type II) is more serious?

2. Consequence • Consequence of Type I: There is no climate change or CO emission does not lead to climate change. All investments in alternate energy are misdirected. But we have alternate energy sources. The air is cleaner in big cities. And we no longer depend on Middle East’s oil. • Consequence of Type II: Global warming is real and CO emission is the cause. Sea level rises and costal cities, including LA and New Orleans, are under water.

3. Null: There is no God and no afterlife. • Alternate: God and afterlife are real. • Type I error: False alarm, the null is right • Type II error: Miss, the alternate is right

4. Pascal Wager • If you don't believe in God and you're right (There is no God), you earn nothing • If you don't believe in God but you're wrong (God is real), you lose everything. • If you believe in God and you're right, you win the eternal life. • If you believe in God but you're wrong, there is nothing to lose.

5. Hypothesis testing result • Two options only: null or alternate • The answer is dichotomous: reject the null or not to reject the null • But, is the real world as simple as black and white only? • The answers may be: “The treatment works for one population, but not for another.” “The construct is a continuum. The difference is not clear-cut.”

6. Alpha level: Critical probability level—0.1, 0.05, 0.01 • Directional hypothesis: The treatment by Professor Yu will improve your test performance in 299 (one-tailed test). • Non-directional: The treatment by Professor Yu will make a difference in your test performance in 299. It could be better or worse (Two-tailed test).

7. What is effect size? • Because an effect is statistically significant, it doesn’t necessarily follow that the effect is an important or meaningful one. Important effects often depend on the effect size as well as reliability.

8. What is effect size? • Before discussing the effect size, I would like to introduce a broader concept: comparison in terms of a standard. Many statistical formulas seem to be difficult. Indeed, many of them are nothing more than a standardized comparison. Take comparing wealth as a metaphor. How could we compare the net assets of American IBM corporation and Japanese Sony corporation? The simplest way is to compare them in US dollars, the standard currency for international trade. By the same token, a t-test is a mean comparison in terms of the standard deviation. Many statistics follow this thread of logic. 

9. What is effect size? • Effect size can be conceptualized as a standardized difference. In the simplest form, effect size, which is denoted by the symbol "d", is the mean difference between groups in standard score form i.e. the ratio of the difference between the means to the standard deviation. 

10. What is effect size? • If only the null hypothesis is available and is rejected, at most the conclusion is that "the difference is not zero." The following figure shows that the difference, indicated by red arrows, can be anything. When the President asks the five-star general to estimate the war casualty, can he give "not zero" as a satisfactory answer?

11. What is effect size? • Researchers should be concerned with not only whether a null hypothesis is false or not, but also how false it is. In other words, if the difference is not zero, how large the difference one should expect?