Maximum Likelihood Estimates and the EM Algorithms II

1. Maximum Likelihood Estimates and the EM Algorithms II Henry Horng-Shing Lu Institute of Statistics National Chiao Tung University hslu@stat.nctu.edu.tw http://tigpbp.iis.sinica.edu.tw/courses.htm

2. Part 1Computation Tools

3. Include Functions in R • source(“file path”) • Example • In MME.R: • In R:

4. Part 2Motivation Examples

5. Example 1 in Genetics (1) Two linked loci with alleles A and a, and B and b A, B: dominant a, b: recessive A double heterozygote AaBb will produce gametes of four types: AB, Ab, aB, ab A a b B A b a a A A a b B B B b F ( Female) 1- r’ r’ (female recombination fraction) M (Male) 1-r r (male recombination fraction) 5

6. Example 1 in Genetics (2) r and r’ are the recombination rates for male and female Suppose the parental origin of these heterozygote is from the mating of . The problem is to estimate r and r’ from the offspring of selfed heterozygotes. Fisher, R. A. and Balmukand, B. (1928). The estimation of linkage from the offspring of selfed heterozygotes. Journal of Genetics, 20, 79–92. http://en.wikipedia.org/wiki/Geneticshttp://www2.isye.gatech.edu/~brani/isyebayes/bank/handout12.pdf 6

7. Example 1 in Genetics (3) 7

8. Example 1 in Genetics (4) Four distinct phenotypes: A*B*, A*b*, a*B* and a*b*. A*: the dominant phenotype from (Aa, AA, aA). a*: the recessive phenotype from aa. B*: the dominant phenotype from (Bb, BB, bB). b* : the recessive phenotype from bb. A*B*: 9 gametic combinations. A*b*: 3 gametic combinations. a*B*: 3 gametic combinations. a*b*: 1 gametic combination. Total: 16 combinations. 8

9. Example 1 in Genetics (5) 9

10. Example 1 in Genetics (6) Hence, the random sample of n from the offspring of selfed heterozygotes will follow a multinomial distribution: 10

11. Example 1 in Genetics (7) Suppose that we observe the data of y = (y1, y2, y3, y4) = (125, 18, 20, 24), which is a random sample from Then the probability mass function is 11

12. Maximum Likelihood Estimate (MLE) Likelihood: Maximize likelihood: Solve the score equations, which are setting the first derivates of likelihood to be zeros. Under regular conditions, the MLE is consistent, asymptotic efficient and normal! More: http://en.wikipedia.org/wiki/Maximum_likelihood 12

13. MLE for Example 1 (1) Likelihood MLE: A B C 13

14. MLE for Example 1 (2) Checking: (1) (2) (3) 14

15. Part 3Numerical Solutions for the Score Equations of MLEs

16. A Banach Space • A Banach space B is a vector space over the field K such that • Every Cauchy sequence of B converges in B (i.e., B is complete). (http://en.wikipedia.org/wiki/Banach_space)

17. Lipschitz Continuous • A closed subset and mapping • F is Lipschitz continuous on A with if . • F is a contraction mapping on A if F is Lipschitz continuous and (http://en.wikipedia.org/wiki/Lipschitz_continuous)

18. Fixed Point Theorem • If F is a contraction mapping on A if F is Lipschitz continuous and • F has an unique fixed point such that • initial , k=1,2,… (http://en.wikipedia.org/wiki/Fixed-point_theorem) (http://www.math-linux.com/spip.php?article60)

19. Applications for MLE (1)

20. Applications for MLE (2) • Optimal ?

21. Parallel Chord Method (1) • Parallel chord method is also called simple iteration.

22. Parallel Chord Method (2) s

23. Plot the Parallel Chord Method by R

24. Define Functions for Example 1 in R We will define some functions and variables for finding the MLE in Example 1 by R

25. Parallel Chord Method by R (1)

26. Parallel Chord Method by R (2)

27. Parallel Chord Method by C/C++

28. Newton-Raphson Method (1) • http://math.fullerton.edu/mathews/n2003/Newton'sMethodMod.html • http://en.wikipedia.org/wiki/Newton%27s_method

29. Newton-Raphson Method (2) s

30. Plot the Newton-Raphson Method by R

31. Newton-Raphson Method by R (1)

32. Newton-Raphson Method by R (2)

33. Newton-Raphson Method by C/C++

34. Halley’s Method • The Newton-Raphson iteration function is • It is possible to speed up convergence byusing more expansion terms than the Newton-Raphson method does when the object function is very smooth, like the method by Edmond Halley (1656-1742): (http://math.fullerton.edu/mathews/n2003/Halley'sMethodMod.html)

35. Halley’s Method by R (1)

36. Halley’s Method by R (2)

37. Halley’s Method by C/C++

38. Bisection Method (1) • Assume that and that there exists a number such that . If and have opposite signs, and represents the sequence of midpoints generated by the bisection process, thenand the sequence converges to r. • That is, . (http://en.wikipedia.org/wiki/Bisection_method )

39. Bisection Method (2) 1

40. Plot the Bisection Method by R

41. Bisection Method by R (1) > fix(Bisection)

42. Bisection Method by R (2)

43. Bisection Method by R (3)

44. Bisection Method by C/C++ (1)

45. Bisection Method by C/C++ (2)

46. Secant Method (http://en.wikipedia.org/wiki/Secant_method ) (http://math.fullerton.edu/mathews/n2003/SecantMethodMod.html )

47. Secant Method by R (1) >fix(Secant)

48. Secant Method by R (2)

49. Secant Method by C/C++

50. S C B A Secant-Bracket Method • The secant-bracket method is also called the regular falsi method.