Today: Quizz 4 Tomorrow: Lab 3 – SN 4117 Wed: A3 due Friday: Lab 3 due

1. Today: Quizz 4 • Tomorrow: Lab 3 – SN 4117 • Wed: A3 due • Friday: Lab 3 due • Mon Oct 1: Exam I  this room, 12 pm • Mon Oct 1: No grad seminar

2. Key concepts so far • Quantity • Measurement scale • Dimensions & Units • Equations • Data Equations • Sums of squared residuals quantify improvement in fit, compare models • Quantify uncertainty through frequency distributions • Empirical • Theoretical • 4 forms, 4 uses

3. Today Selected examples from: Read lecture notes

4. Logic of Hypothesis Testing Reject JUST LUCK Hypothesis Izaak Walton A B C D Skill! Just Luck!!

5. Reject JUST LUCK • Compared observed outcome to all possible outcomes  more tractable to restrict to all possible outcomes given that JUST LUCK hyp is true Arrangements of 8 fish such that IW catches 7?

6. Reject JUST LUCK Arrangements of 8 fish such that IW catches 7? Assign probabilities to each outcome, assuming that the H0 ‘JUST LUCK’ is true For each fish, there is a 1 in 5 chance that IW will catch it IW=8 IW=7 IW=7 IW=7 IW=7 A=1 B=1 C=1 D=1 (1/5)8 (1/5)7 (1/5)7 (1/5)7 (1/5)7 0.00000256 0.0000128 0.0000128 0.0000128 0.0000128 p=0.00005376, i.e. 5 times in 10,000

7. Hypothesis Testing • Set of rules for making decisions in the face of uncertainty • Logic is inductive: from specific to general • Structure is binary

8. 3 styles of statistical inference • Likelihood, frequentist and Bayesian inference • All based on the principle of maximum likelihood Definition: a model that makes the data more probable (best predicts the observed data) is said to be more likely to have generated the data

9. 3 styles of statistical inference Likelihood inference Which model is more likely to have generated the data? Frequentist inference Use expected distribution of outcomes to calculate a probability

10. 3 styles of statistical inference Bayesian inference Find the probability that a hypothesis is true, given the observed data Contrast to: finding the probability of observing the data I observed (or more extreme data), assuming that the null hypothesis is true Integrates prior knowledge we have on the system with new observations to make an informed decision

11. 3 styles of statistical inference Bayesian inference e.g.: coin flip. Hypothesis: the coin is biased Observe flips: HTHHHTTHHHH

12. Frequentist approachNull Hypothesis H0 • H0 just chance • Research hypothesis (what we really care about) is stated as HA • So, why work with H0 and not HA? • Easier to work out probabilities • Permits yes/no decision • Working with H0 is not intuitive. Logic is backwards because we want to reject H0, not explain how the world functions through H0

13. Choice of HA • Start with research hyp, then challenge it with H0 • HA/H0 defined with respect to population, not sample • HA/H0 must be defined prior to analysis • Choice of HA/H0 determines how we calculate p-value • HA/H0 pair must be exhaustive • HA/H0 must be mutually exclusive

14. Choice of HA How do we choose it? Often HA=effect, H0= no effect BUT, more informative choices are available: G: growth rate of plants. c:Control, t: treated with fertilizer ‘tails’ ‘scale’ • . • . • .

15. Type I & Type II error • Type I (α): reject H0 when it is true ‘false positive’ e.g. in a trial, accused is innocent but goes to jail H0: • Type II (β): not rejecting H0 when it is false ‘false negative’ e.g. in a trial, accused is guilty but is set free H0:

16. Type I & Type II error • Type I (α): reject H0 when it is true ‘false positive’ • Type II (β): not rejecting H0 when it is false ‘false negative’

17. Type I & Type II error True H0 Reject H0 when it is true

18. Type I & Type II error Draw not rejecting H0 when it is false, i.e. β Tradeoff between α and β Draw rejecting H0 when H0 is false, i.e. power True HA

19. Selected examples from: Will present 2 examples (if time allows) More examples in lecture notes

20. Table 7.1 Generic recipe for decision making with statistics • State population, conditions for taking sample • State the model or measure of pattern…………………………… • State null hypothesis about population…………………………… • State alternative hypothesis………………………………………… • State tolerance for Type I error……………………………………… • State frequency distribution that gives probability of outcomes whenthe Null Hypothesis is true. Choices: • Permutations: distributions of all possible outcomes • Empirical distribution obtained by random sampling of all possibleoutcomes when H0 is true • Cumulative distribution function (cdf) that applies when H0 is trueState assumptions when using a cdf such as Normal, F, t or chisquare • Calculate the statistic. This is the observed outcome • Calculate p-value for observed outcome relative to distribution of outcomes when H0 is true • If p less than α then reject H0 in favour of HAIf greater than α then not reject H0 • Report statistic, p-value, sample sizeDeclare decision

21. Example: jackal bones Length of bones from 10 female and 10 male jackals (Manly 1991) L = length of mandible (L=mm) of Golden jackals

22. Example: jackal bones • Population: All possible measurements on these bonesAll jackals in the world? Need to know if sample representative • Measure of pattern: ST = D0 = • H0: • HA: • α= • Theoretical dist of D0? UnknownSolution: construct empirical freq dist of D0 when H0 is true by randomization….

23. Example: jackal bones • D0 = mean(Lmale)-mean(Lfem) 3.H0: D0<=0 4.HA:D0>0 5. α=5% • Empirical FD. Randomization • Assign bones randomly to 2 groups (forget M/F) • Compute mean(gr1) and mean(gr2) • D0,res= mean(gr1) - mean(gr2) • Repeat many times (the more the better, continued later) • Assemble random differences into a FD • Statistic. Do= 113.4 – 108.6 = 4.8 mm

24. Example: jackal bones • D0 = mean(Lmale)-mean(Lfem) 3.H0: D0<=0 4.HA:D0>0 5. α=5% • Compute p-value:100,000 values of D0,res360 values exceed 4.8p = 360/100000 p = 0.0036 • p =0.0036< α=0.05 reject H0in favour of HA (D0>0) • D0 = 4.8 mmn = p = male jackal mandible bones significantly longer than those of females

25. Example: jackal bones This was laborious Can be made easier by using theoretical frequency distributions Trade off: must make assumptions

26. Example: jackal bones6d) repeat many times 100,000 repetitions

27. Example: jackal bones6d) repeat many times 10,000 repetitions

28. Example: jackal bones6d) repeat many times 1,000 repetitions

29. Example: Oat Yield data Yield of oats in 2 groups • Control • Chemical seed treatment 1 common mean 1 mean per group Is the improvement better than random?

30. Example: Oat Yield data • Sample: 8 measurementsPopulation: all possible measurements taken with a stated procedure • Measure of pattern: ST = SSmodel • H0: E(SSmodel) = 0 • HA:E(SSmodel) > 0 • α=5% • Theoretical dist of SSmodel? UnknownSolution: construct empirical freq dist of SSmodel when H0 is true by randomization….

31. Example: Oat Yield data • Empirical FD • Assign yields to 2 groups (forget treatment/control) • Fit common mean model • Fit 2 means model • Calculate SSmodel • Repeat many times (1000) • Assemble random differences into a FD • Statistic. SSmodel=192.08

32. Example: Oat Yield data • Compute p-value:1,000 values of SSmodel161 values exceed 192.08p = 161/1000 p = 0.161 • p = 0.161 > 0.05  do not reject H0The improvement is not better than random • SSmodel = 192.08n = 8 p = 0.161we can not reject the JUST LUCK hypothesis

33. QUIZZ 4 Good luck!