2.2 The gamma and chi-square distributions

Gamma and chi-square distributions

In the (approximate) Poisson process with mean $\lambda$ , we have seen that the waiting time until the first change has an exponential distribution . Let now W denote the waiting time until the $\alpha$ th change occurs and let find the distribution of W . The distribution function of W ,when $w\ge 0$ is given by

$$\begin{array}{l}F\left(w\right)=P\left(W\le w\right)=1-P\left(W>w\right)=1-P\left(fewer\_than\_\alpha \_changes\_occur\_in\_\left[\mathrm{0,}w\right]\right)\\ =1-{\displaystyle \sum _{k=0}^{\alpha -1}\frac{{\left(\lambda w\right)}^k{e}^{-\lambda w}}{k!}},\end{array}$$

since the number of changes in the interval $\left[\mathrm{0,}w\right]$ has a Poisson distribution with mean $\lambda w$ . Because W is a continuous-type random variable, $F\text{'}\left(w\right)$ is equal to the p.d.f. of W whenever this derivative exists. We have, provided w >0, that

$$\begin{array}{l}F\text{'}\left(w\right)=\lambda e^{-\lambda w}-e^{-\lambda w}{\displaystyle \sum _{k=1}^{\alpha -1}\left[\frac{k{\left(\lambda w\right)}^{k-1}\lambda }{k!}-\frac{{\left(\lambda w\right)}^k\lambda }{k!}\right]}=\lambda e^{-\lambda w}-e^{-\lambda w}\left[\lambda -\frac{\lambda {\left(\lambda w\right)}^{\alpha -1}}{\left(\alpha -1\right)!}\right]\\ =\frac{\lambda {\left(\lambda w\right)}^{\alpha -1}}{\left(\alpha -1\right)!}{e}^{-\lambda w}.\end{array}$$

Gamma distribution

The gamma function is defined by $$\Gamma \left(t\right)={\displaystyle \underset{0}{\overset{\infty }{\int }}{y}^{t-1}{e}^{-y}dy}\mathrm{,0}<t.$$

This integral is positive for $0<t$ , because the integrand id positive. Values of it are often given in a table of integrals. If $t>1$ , integration of gamma fnction of t by parts yields

$$\Gamma \left(t\right)={\left[-y^{t-1}{e}^{-y}\right]}_0^{\infty }+{\displaystyle \underset{0}{\overset{\infty }{\int }}\left(t-1\right)y^{t-2}{e}^{-y}dy}=\left(t-1\right){\displaystyle \underset{0}{\overset{\infty }{\int }}{y}^{t-2}{e}^{-y}dy=}\left(t-1\right)\Gamma \left(t-1\right).$$

Incidentally, $\Gamma \left(1\right)$ corresponds to 0!, and we have noted that $\Gamma \left(1\right)=1$ , which is consistent with earlier discussions.

Summarizing

The random variable x has a gamma distribution if its p.d.f. is defined by

Hence, w , the waiting time until the $\alpha$ th change in a Poisson process, has a gamma distribution with parameters $\alpha$ and $\theta =1/\lambda$ .

Function $f\left(x\right)$ actually has the properties of a p.d.f., because $f\left(x\right)\ge 0$ and

$${\displaystyle \underset{-\infty }{\overset{\infty }{\int }}f\left(x\right)dx={\displaystyle \underset{0}{\overset{\infty }{\int }}\frac{x^{\alpha -1}{e}^{-x/\theta }}{\Gamma \left(\alpha \right){\theta }^{\alpha }}}}dx,$$ which, by the change of variables $y=x/\theta$ equals

$${\displaystyle \underset{0}{\overset{\infty }{\int }}\frac{{\left(\theta y\right)}^{\alpha -1}{e}^{-y}}{\Gamma \left(\alpha \right){\theta }^{\alpha }}}\theta dy=\frac{1}{\Gamma \left(\alpha \right)}{\displaystyle \underset{0}{\overset{\infty }{\int }}{y}^{\alpha -1}{e}^{-y}dy}=\frac{\Gamma \left(\alpha \right)}{\Gamma \left(\alpha \right)}=1.$$

The mean and variance are: $\mu =\alpha \theta$ and ${\sigma }^2=\alpha {\theta }^2$ .

Chi-square distribution

Let now consider the special case of the gamma distribution that plays an important role in statistics.

Let X have a gamma distribution with $\theta =2$ and $\alpha =r/2$ , where r is a positive integer. If the p.d.f. of X is

$$f\left(x\right)=\frac{1}{\Gamma \left(r/2\right)2^{r/2}}{x}^{r/2-1}{e}^{-x/2}\mathrm{,0}\le x<\infty .$$

We say that X has chi-square distribution with r degrees of freedom, which we abbreviate by saying is ${\chi }^2\left(r\right)$ .

The mean and the variance of this chi-square distributions are

$$\mu =\alpha \theta =\left(\frac{r}{2}\right)2=r$$ and $${\sigma }^2=\alpha {\theta }^2=\left(\frac{r}{2}\right)2^2=2r.$$

That is, the mean equals the number of degrees of freedom and the variance equals twice the number of degrees of freedom.

In the fugure 2 the graphs of chi-square p.d.f. for r =2,3,5, and 8 are given.

Because the chi-square distribution is so important in applications, tables have been prepared giving the values of the distribution function for selected value of r and x ,

Probabilities like that in Example 4 are so important in statistical applications that one uses special symbols for a and b . Let $\alpha$ be a positive probability (that is usually less than 0.5) and let X have a chi-square distribution with r degrees of freedom. Then ${\chi }_{\alpha }^2\left(r\right)$ is a number such that $P\left[X\ge {\chi }_{\alpha }^2\left(r\right)\right]=\alpha$

That is, ${\chi }_{\alpha }^2\left(r\right)$ is the 100(1- $\alpha$ ) percentile (or upper 100a percent point) of the chi-square distribution with r degrees of freedom. Then the 100 $\alpha$ percentile is the number ${\chi }_{1-\alpha }^2\left(r\right)$ such that $P\left[X\le {\chi }_{1-\alpha }^2\left(r\right)\right]=\alpha$ . This is, the probability to the right of ${\chi }_{1-\alpha }^2\left(r\right)$ is 1- $\alpha$ . SEE fugure 3 .