Maximum likelihood estimation  (Page 2/3)

The likelihood principle is widely accepted among statisticians. In the context of parameter estimation, any reasonable estimator should conform to the likelihoodprinciple. As we will see, the maximum likelihood estimator does.

The maximum likelihood estimator

The maximum likelihood estimator $((x))$ is defined by $$()=(, l(x, ))$$ Intuitively, we are choosing $$ to maximize the probability of occurrence of the observation $x$ .

The MLE rule is an implementation of the likelihood principle. If we have two observations whose likelihoods areproportional (they differ by a constant that does not depend on $$ ), then the value of $$ that maximizes one likelihood will also maximize the other. In other words, both likelihood functions lead to thesame inference about $$ , as required by the likelihood principle.

Understand that maximum likelihood is a procedure , not an optimality criterion. From the definition of the MLE, we have no idea how close itcomes to the true parameter value relative to other estimators. In constrast, the MVUE is defined as the estimatorthat satisfies a certain optimality criterion. However, unlike the MLE, we have no clear produre to follow to compute theMVUE.

Computing the mle

If the likelihood function is differentiable, then $()$ is found by differentiating the likelihood (or log-likelihood), equating with zero, and solving: $$\frac{\partial^1\lg l(x, )}{\partial }=0$$ If multiple solutions exist, then the MLE is the solution that maximizes $\lg l(x, )$ , that is, the global maximizer.

In certain cases, such as pdfs or pmfs with an esponential form, the MLE can beeasily solved for. That is, $$\frac{\partial^1\lg l(x, )}{\partial }=0$$ can be solved using calculus and standard linear algebra.