Transform methods  (Page 2/5)

The integral transform character of these entities implies that there is essentially a one-to-one relationship between the transform and the distribution.

Moments

The name and some of the importance of the moment generating function arise from the fact that the derivatives of M _X evaluateed at $s=0$ are the moments about the origin. Specifically

Since expectation is an integral and because of the regularity of the integrand, we may differentiate inside the integral with respect to the parameter.

Upon setting $s=0$ , we have $M_X^{\text{'}}\left(0\right)=E\left[X\right]$ . Repeated differentiation gives the general result. The corresponding result for the characteristic function is ${\phi }^{\left(k\right)}\left(0\right)=i^{k}E\left[X^k\right]$ .

The generating function does not lend itself readily to computing moments, except that

For higher order moments, we may convert the generating function to the moment generating function by replacing s with e ^s , then work with M _X and its derivatives.

Operational properties

We refer to the following as operational properties .

• (T1) If $Z=aX+b$ , then
  $$M_Z\left(s\right)=e^{bs}{M}_X\left(as\right),{\phi }_Z\left(u\right)=e^{iub}{\phi }_X\left(au\right),g_Z\left(s\right)=s^b{g}_X\left(s^a\right)$$
  For the moment generating function, this pattern follows from
  $$E\left[e^{(aX+b)s}\right]=s^{bs}E\left[e^{\left(as\right)X}\right]$$
  Similar arguments hold for the other two.
• (T2) If the pair $\{X,Y\}$ is independent, then
  $$M_{X+Y}\left(s\right)=M_X\left(s\right)M_Y\left(s\right),{\phi }_{X+Y}\left(u\right)={\phi }_X\left(u\right){\phi }_Y\left(u\right),g_{X+Y}\left(s\right)=g_X\left(s\right)g_Y\left(s\right)$$
  For the moment generating function, $e^{sX}$ and $e^{sY}$ form an independent pair for each value of the parameter s . By the product rule for expectation
  $$E\left[e^{s(X+Y)}\right]=E\left[e^{sX}{e}^{sY}\right]=E\left[e^{sX}\right]E\left[e^{sY}\right]$$
  Similar arguments are used for the other two transforms.
  A partial converse for (T2) is as follows:
• (T3) If $M_{X+Y}\left(s\right)=M_X\left(s\right)M_Y\left(s\right)$ , then the pair $\{X,Y\}$ is uncorrelated. To show this, we obtain two expressions for $E\left[{(X+Y)}^2\right]$ , one by direct expansion and use of linearity, and the other by taking the second derivative of the moment generatingfunction.
  $$E\left[{(X+Y)}^2\right]=E\left[X^2\right]+E\left[Y^2\right]+2E\left[XY\right]$$
  $$M_{X+Y}^{\text{'}\text{'}}\left(s\right)={\left[M_X\left(s\right)M_Y\left(s\right)\right]}^{\text{'}\text{'}}=M_X^{\text{'}\text{'}}\left(s\right)M_Y\left(s\right)+M_X\left(s\right)M_Y^{\text{'}\text{'}}\left(s\right)+2M_X^{\text{'}}\left(s\right)M_Y^{\text{'}}\left(s\right)$$
  On setting $s=0$ and using the fact that $M_X\left(0\right)=M_Y\left(0\right)=1$ , we have
  $$E\left[{(X+Y)}^2\right]=E\left[X^2\right]+E\left[Y^2\right]+2E\left[X\right]E\left[Y\right]$$
  which implies the equality $E\left[XY\right]=E\left[X\right]E\left[Y\right]$ .

Note that we have not shown that being uncorrelated implies the product rule.

We utilize these properties in determining the moment generating and generating functions for several of our common distributions.

Some discrete distributions

1. Indicator function $X=I_{E}P\left(E\right)=p$
   $$g_X\left(s\right)=s^{0}q+s^{1}p=q+ps{M}_X\left(s\right)=g_X\left(e^s\right)=q+p{e}^s$$
2. Simple random variable ${\displaystyle X=\sum _{i=1}^n{t}_i{I}_{A_i}}$ (primitive form) $P\left(A_i\right)=p_i$
   $$M_X\left(s\right)=\sum _{i=1}^n{e}^{s{t}_i}{p}_i$$
3. Binomial $(n,p)$ . ${\displaystyle X=\sum _{i=1}^n{I}_{E_i}\text{with}\{I_{E_i}:1\le i\le n\}\text{iid}P\left(E_i\right)=p}$
   We use the product rule for sums of independent random variables and the generating function for the indicator function.
   $$g_X\left(s\right)=\prod _{i=1}^n(q+ps)={(q+ps)}^n{M}_X\left(s\right)={(q+p{e}^s)}^n$$
4. Geometric $\left(p\right)$ . $P(X=k)=p{q}^k\forall k\ge 0$ $E\left[X\right]=q/p$ We use the formula for the geometric series to get
   $$g_X\left(s\right)=\sum _{k=0}^{\infty }p{q}^k{s}^k=p\sum _{k=0}^{\infty }{\left(qs\right)}^k=\frac{p}{1-qs}{M}_X\left(s\right)=\frac{p}{1-q{e}^s}$$
5. Negative binomial $(m,p)$ If Y _m is the number of the trial in a Bernoulli sequence on which the m th success occurs, and $X_m=Y_m-m$ is the number of failures before the m th success, then
   $$P(X_m=k)=P(Y_m-m=k)=C(-m,k){(-q)}^k{p}^m$$
   $$\text{where}C(-m,k)=\frac{-m(-m-1)(-m-2)\cdots (-m-k+1)}{k!}$$
   The power series expansion about $t=0$ shows that
   $${(1+t)}^{-m}=1+C(-m,1)t+C(-m,2)t^2+\cdots \text{for}-1<t<1$$
   Hence
   $$M_{X_m}\left(s\right)=p^m\sum _{k=0}^{\infty }C(-m,k){(-q)}^k{e}^{sk}={\left[\frac{p}{1-q{e}^s}\right]}^m$$
   Comparison with the moment generating function for the geometric distribution shows that $X_m=Y_m-m$ has the same distribution as the sum of m iid random variables, each geometric $\left(p\right)$ . This suggests that the sequence is characterized by independent, successive waiting times to success. This also shows that the expectation and variance of X _m are m times the expectation and variance for the geometric. Thus
   $$E\left[X_m\right]=mq/p\text{and}\mathrm{Var}\left[X_m\right]=mq/p^2$$
6. Poisson $\left(\mu \right)$ ${\displaystyle P(X=k)=e^{-\mu }\frac{{\mu }^k}{k!}\forall k\ge 0}$ In [link] , above, we establish $g_X\left(s\right)=e^{\mu (s-1)}$ and $M_X\left(s\right)=e^{\mu (e^s-1)}$ . If $\{X,Y\}$ is an independent pair, with $X\sim$ Poisson $\left(\lambda \right)$ and $Y\sim$ Poisson $\left(\mu \right)$ , then $Z=X+Y\sim$ Poisson $(\lambda +\mu )$ . Follows from (T1) and product of exponentials.