Testing a hypothesis

1. Testing a hypothesis (Session 09)

2. Learning Objectives By the end of this session, you will be able to • set up and carry out a test of hypothesis concerning a population mean • interpret the results of such a t-test • explain what is meant by a p-value and how it may be interpreted • state conclusions resulting from a hypothesis test according to the size of the p-value

3. An illustrative example • Farmers growing maize in upland areas of Chiradzulu EPA in southern Malawi were getting average yields of 2900 kg/ha with s.d. = 780 kg/ha. • A “new” Integrated Pest Management (IPM) approach was attempted with 16 farmers. • Objective: To determine if the new approach results in an increase in maize yields. • Yields from these 16 farmers (after using IPM) gave = 3454 kg/ha with standard error 168. • Can we determine whether IPM has really increased maize yields?

4. Is the yield increase real? • In above example, clearly the sample mean of 3454 kg/ha is greater than 2990 kg/ha • But the question of interest is “does this result indicate a significant increase in the yield or is it just a result of the usual random variation of yield” • Hypothesis testing seeks to answer such questions by looking at the observed change relative to the “noise”, i.e. the standard error in the sample estimate

5. Review of Session 8 ideas • A hypothesis is a statement or a claim concerning a population or populations. • The main hypothesis of interest is called the null hypothesis and usually denoted H0. • The alternative hypothesis, denoted H1, is one that is considered to be true if the null hypothesis is false. • It is thus always the case that either H0 is true or H1 is true, both cannot be true at the same time.

6. H0 and H1 for example Null hypothesis H0:  = 2900 where  is the true mean yield of farmers in the area using the new approach The promoters of the new approach are confident that yields with the new approach cannot possibly decrease. Hence the above null hypothesis needs to be tested against the alternative hypothesis H1:  > 2900 Note: H0 is very specific. H1 is more general

7. Classical approach to testing We use the distribution of the sample mean ( ) to test H0, i.e. ~ N(, 2/n) Assume the null hypothesis is true, i.e. IPM has no effect on yields. Then From tables of the standard normal distn, we see that the above probability is about 0.0022

8. Interpretation This probability is very small. Unlikely to get this probability value, based on the observed mean of 3454 kg/ha, if the null hypothesis is really true. In other words, the chance of getting a z-value as large as that observed (i.e. 2.84) is very small at 0.22%. We conclude that H0 cannot be true. IPM has really had an effect.

9. Test statistic based approach (a) Set up the null and alternative hypotheses (b) Compute the test statistic z = ( - )/(/n) = (3454 – 2900)/(780/4) = 2.84 (c) Find the probability of getting a result, which is as extreme, or more extreme than the one observed, given H0 is true. The smaller this probability value, the greater is the evidence against the null hypothesis. This probability is called the p-valueor significance level of the test

10. Test for mean when  is unknown It is usually the case that  is unknown. Instead, we could use the sample standard deviation s. The test statistic is now t = ( - )/(s/n) = (3454 – 2900)/(168) = 3.30 which follows a t-distribution with n-1=15 degrees of freedom. Exercise: Look up tables of the t-distribution and assess roughly what the p-value may be…

11. Interpretation and conclusions It is clear from t-tables that the p-value is smaller than 0.01. Using statistical software, we can get the exact p-value as 0.0024. (Can also use Excel’s function tdist(3.30,15,1) to get this) This p-value is so small, there is sufficient evidence to reject H0. Conclusion: Use of the new IPM technology has led to an increase in maize yields (p-value=0.0024)

12. Summary of actions to take • If get a small p-value, i.e. close to zero, • Conclude null-hypothesis is unlikely to be true and select the alternative • Reject the null hypothesis and declare that the result is “statistically significant” • Note that “Statistical significance” does not necessarily imply that the effect observed is practically important (more on this later) • If the p-value is not small, • There is insufficient evidence to reject H0. • The result is not significant

13. More on interpretation of p-value • Traditionally, 0.05 has been regarded as a cut-off point for a result to be declared as statistically significant • Note that the p-value is essentially the chance (probability) that the null hypothesis is rejected when it is actually true • Thus the p-value =  = Probability of the Type I error of test

14. Reporting the results Take care when reporting results. It is not a good idea to regard a 5% level of significance, i.e. a p-value of 0.05, as an absolute cut off value. Report results according to size of p-value. For example, evidence of an effect is : • almost conclusive if p-value < 0.001 and could be said to be strong if p-value < 0.010 • If 0.01< p-value < 0.05, results indicate some evidence of an effect. • If p-value > 0.05, but close to 0.05, it may indicate something is going on, but further confirmatory study is needed.

15. References Altman, D.G., Machin, D., Bryant, T.N., and Gardner, M.J. (2000) Statistics with confidence. (2nd Edition). BMJ Books, Bristol, UK. pp 240. (See next session for more references) Some practical work follows…