3.3 Characteristic functions

You have already encountered the Moment Generating Function of a pdf in the Part IB probability course. This function was closely related to the Laplace Transform of the pdf.

Now we introduce the Characteristic Function for a random variable, which is closely related to the Fourier Transform of the pdf.

In the same way that Fourier Transforms allow easy manipulation of signals when they are convolved with linear system impulseresponses, Characteristic Functions allow easy manipulation of convolved pdfs when they represent sums of random processes.

The Characteristic Function of a pdf is defined as:

Note that whenever $f_X$ is a valid pdf, $\Phi (0)=\int f_X(x)\,d x=1$

Properties of Fourier Transforms apply with $-u$ substituted for $\omega$ . In particular:

• Convolution - (sums of independent rv's)
  $(Y=\sum_{i=1}^N X_i)\implies (f_Y=(f_{X_1}, f_{X_2}, \dots , f_{X_N}))\implies ({\Phi }_Y(u)=\prod_{i=1}^N {\Phi }_{X_i}(u))$
• Inversion
  $f_X(x)=\frac{1}{2\pi }\int e^{-(iux)}{\Phi }_X(u)\,d u$
• Moments
  $(\frac{d^n{\Phi }_X(u)}{du^n}=\int (ix)^{n}e^{iux}{f}_X(x)\,d x)\implies ((X^n)=\int x^n{f}_X(x)\,d x=\frac{1}{i^n}(u, , \frac{d^n{\Phi }_X(u)}{du^n}))$
• Scaling If $Y=aX$ , $f_Y(y)=\frac{f_X(x)}{a}$ from this equation in our previous discussion of functions of random variables, then
  ${\Phi }_Y(u)=\int e^{iuy}{f}_Y(y)\,d y=\int e^{iuax}{f}_X(x)\,d x={\Phi }_X(au)$

Characteristic function of a gaussian pdf

The Gaussian or normal distribution is very important, largely because of the Central Limit Theorem which we shall prove below. Because of this (and as part of the proofof this theorem) we shall show here that a Gaussian pdf has a Gaussian characteristic function too.

A Gaussian distribution with mean $\mu$ and variance $\sigma ^2$ has pdf:

Thus the characteristic function of a Gaussian pdf is also Gaussian in magnitude, $e^{-\left(\frac{u^2\sigma ^2}{2}\right)}$ , with standard deviation $\frac{1}{\sigma }$ , and with a linear phase rotation term, $e^{iu\mu }$ , whose rate of rotation equals the mean $\mu$ of the pdf. This coincides with standard results from Fourier analysis of Gaussianwaveforms and their spectra (e.g. Fourier transform of a Gaussian waveform with time shift).

Summation of two or more gaussian random variables

If two variables, $X_1$ and $X_2$ , with Gaussian pdfs are summed to produce $X$ , their characteristic functions will be multiplied together (equivalent toconvolving their pdfs) to give

Further Gaussian variables can be added and the pdf will remain Gaussian with further terms added to the aboveexpressions for the combined mean and variance.

Central limit theorem

The central limit theorem states broadly that if a large number $N$ of independent random variables of arbitrary pdf, but with equal variance $\sigma ^2$ and zero mean, are summed together and scaled by $\frac{1}{\sqrt{N}}$ to keep the total energy independent of $N$ , then the pdf of the resulting variable will tend to a zero-mean Gaussian with variance $\sigma ^2$ as $N$ tends to infinity.

This result is obvious from the previous result if the input pdfs are also Gaussian , but it is the fact that it applies for arbitrary input pdfs that is remarkable, and is the reason for the importance of the Gaussian (or normal) pdf. Noise generated in nature isnearly always the result of summing many tiny random processes (e.g. noise from electron energy transitions in a resistor ortransistor, or from distant worldwide thunder storms at a radio antenna) and hence tends to a Gaussian pdf.

Although for simplicity, we shall prove the result only forthe case when all the summed processes have the same variance and pdfs, the central limit result is more general than this and applies in many caseseven when the variance and pdfs are not all the same.

Proof:

Let $X_i$ ( $i=1$ to $N$ ) be the $N$ independent random processes, each will zero mean and variance $\sigma ^2$ , which are combined to give

• valid pdf $$\Phi (0)=(X_i^0)=1$$
• zero mean $$\frac{d \Phi (0)}{d }=i(X_i)=0$$
• variance $$\frac{d \Phi (0)}{d }=i^2(X_i^2)=-\sigma ^2$$
• scaled skewness $$\frac{d \Phi (0)}{d }=i^3(X_i^3)=-(i\gamma \sigma ^3)$$
• scaled kurtosis $$\frac{d \Phi (0)}{d }=i^4(X_i^4)=(\kappa +3)\sigma ^4$$

Substituting these moments into and and using the series expansion, $\lg (1+x)=x$ + (terms of order $x^2$ or smaller), gives

Hence we may now infer from , and that the pdf of $X$ as $N\to$∞ will be given by

shows an example of convergence when the input pdfs are uniform, and $N$ is gradually increased from $1$ to $50$ . By $N=12$ , convergence is good, and this is how some 'Gaussian' random generator functions operate - by summingtypically $12$ uncorrelated random numbers with uniform pdfs.

For some less smooth or more skewed pdfs, convergence can be slower, as shown for a highly skewed triangular pdf in ; and pdfs of discrete processes are particularly problematic in this respect, asillustrated in .