Concentrated Likelihood Functions, and Properties of Maximum Likelihood

1. Concentrated Likelihood Functions, and Properties of Maximum Likelihood Lecture XX

2. Concentrated Likelihood Functions • The more general form of the normal likelihood function can be written as:

3. This expression can be solved for the optimal choice of s2 by differentiating with respect to s2:

4. Substituting this result into the original logarithmic likelihood yields

5. Intuitively, the maximum likelihood estimate of m is that value that minimizes the mean square error of the estimator. Thus, the least squares estimate of the mean of a normal distribution is the same as the maximum likelihood estimator under the assumption that the sample is independently and identically distributed.

6. The Normal Equations • If we extend the above discussion to multiple regression, we can derive the normal equations.

7. Taking the derivative with respect to a0 yields

8. Taking the derivative with respect to a1 yields

9. Substituting for a0 yields

11. Properties of Maximum Likelihood Estimators • Theorem 7.4.1: Let L(X1,X2,…Xn|q) be the likelihood function and let q^(X1,X2,…Xn) be an unbiased estimator of q. Then, under general conditions, we have The right-hand side is known as the Cramer-Rao lower bound (CRLB).

12. The consistency of maximum likelihood can be shown by applying Khinchine’s Law of Large Numbers to which converges as long as

13. Asymptotic Normality • Theorem 7.4.3: Let the likelihood function be L(X1,X2,…Xn|q). Then, under general conditions, the maximum likelihood estimator of q is asymptotically distributed as