Tests

1. Tests Jean-Yves Le Boudec

2. Contents • The Neyman Pearson framework • Likelihood Ratio Tests • ANOVA • Asymptotic Results • Other Tests 1

3. Tests • Tests are used to give a binary answer to hypotheses of a statistical nature • Ex: is A better than B? • Ex: does this data come from a normal distribution ? • Ex: does factor n influence the result ? 2

4. Example: Non Paired Data • Is red better than blue ? • For data set (a) answer is clear (by inspection of confidence interval) no test required 3

5. Is this data normal ? 4

6. 5.1 The Neyman-Pearson Framework 5

7. Example: Non Paired Data • Is red better than blue ? 6

8. Critical Region, Size and Power 7

9. Example : Paired Data 8

10. Power 9

11. Grey Zone 10

12. 11

13. p-value of a test 12

14. 13

15. Tests are just tests 14

16. Test versus Confidence Intervals • If you can have a confidence interval, use it instead of a test 15

17. 2. Likelihood Ratio Test • A special case of Neyman-Pearson • A Systematic Method to define tests, of general applicability 16

18. 17

19. 18

20. A Classical Test: Student Test • The model : • The hypotheses : 19

21. 20

22. 21

23. Here it is the same as a Conf. Interval 22

24. The “Simple Goodness of Fit” Test • Model • Hypotheses 23

25. 1. compute likelihood ratio statistic 24

26. 2. compute p-value 25

27. Mendel’s Peas • P= 0.92 ± 0.05 => Accept H0 26

28. 3 ANOVA • Often used as “Magic Tool” • Important to understand the underlying assumptions • Model • Data comes from iid normalsample with unknown means and same variance • Hypotheses 27

29. 28

30. 29

31. The ANOVA Theorem • We build a likelihood ratio statistic test • The assumption that data is normal and variance is the same allows an explicit computation • it becomes a least square problem = a geometrical problem • we need to compute orthogonal projections on M and M0 30

32. The ANOVA Theorem 31

33. Geometrical Interpretation • Accept H0 if SS2 is small • The theorem tells us what “small” means in a statistical sense 32

34. 33

35. ANOVA Output: Network Monitoring 34

36. The Fisher-F distribution 35

37. 36

38. Compare Test to Confidence Intervals • For non paired data, we cannot simply compute the difference • However CI is sufficient for parameter set 1 • Tests disambiguate parameter sets 2 and 3 37

39. Test the assumptions of the test… • Need to test the assumptions • Normal • In each group: qqplot… • Same variance 38

40. 39

41. 4 Asymptotic Results 2 x Likelihood ratio statistic 40

42. 41

43. The chi-square distribution 42

44. Asymptotic Result • Applicable when central limit theorem holds • If applicable, radically simple • Compute likelihood ratio statistic • Inspect and find the order p (nb of dimensions that H1 adds to H0) • This is equivalent to 2 optimization subproblemslrs = = max likelihood under H1 - max likelihood under H0 • The p-value is 43

45. Composite Goodness of Fit Test • We want to test the hypothesis that an iid sample has a distribution that comes from a given parametric family 44

46. Apply the Generic Method • Compute likelihood ratio statistic • Compute p-value • Either use MC or the large n asymptotic 45

47. 46

48. Is it normal ? 47

49. 48

50. 49