Logit and probit regressions

The likelihood ratio test

The likelihood ratio test is based on a comparison of the maximum log of likelihood function for the unrestricted model with the maximum log of likelihood function for the model with the restrictions implied by the null hypothesis. Consider the null hypothesis given in (9). Let $L\left(\beta \right)$ be the value of the likelihood function when ${\beta }_1$ be the value of the likelihood function when is restricted to being equal to $\beta$ and $L\left({\stackrel{⌢}{\beta }}_1\right)$ be the value of the likelihood function when there is no restriction on the value of $\beta .$ Then the appropriate test statistic is

$$LR=-2\left[\ln L\left(\beta \right)-\ln L\left({\stackrel{⌢}{\beta }}_1\right)\right].$$

The likelihood ratio statistic has the Chi-square distribution ${\chi }^2\left(r\right),$ where r is the number of restrictions. Thus, using a likelihood ratio test involves two estimations—one with no restrictions on the model and one with the restrictions implied by null hypothesis. Since the likelihood ratio test does not appear to exhibit perverse behavior with small sample sizes, it is an attractive test. Thus, we will run through an example of how to execute the test using Stata . The example we are using is from the Stata manual, volume 2, pp. 353-355.

Underweight births.

In this model we estimate a model that explains the likelihood that a child will be born with a weight under 2,500 grams (low). The eight explanatory variables used in the model are listed in Table 1. The model to be estimated is:

Also, we want to test the null hypothesis that the coefficients on Age, Lwt, Ptl, and Ht are all zero. The first step is to estimate the unrestricted regression using the command:

. logistic low age lwt raceb raceo smoke ptl ht ui

The results of this estimation are shown in column 2 of Table 2. Next we save the results of this regression with the command:

. estimates store full

where “full” is the name that we will refer to when we want to recall the estimation results from this regression. Now we estimate the logistic regression with the omitting the variables whose parameters are to be restricted to being equal to zero:

. logistic low raceb raceo smoke ui

The results of this estimation are reported in column 3 of Table 2. Finally we run the likelihood ratio test with the command:

. lrtest full .

Notice that we refer to the first regression with the word “full” and to the second regression with the second period. The results of this command are as follows:

Likelihood-ratio test LR chi2(4) = 14.42