0.10 Models of noise

Models of noise

White Noise In a nutshell, a sequence $({\epsilon }_1,{\epsilon }_2,{\epsilon }_3,...)$ is called white noise if its entries ${\epsilon }_k$ are independent identically distributed random numbers.

• “Independence” means that there is no information or relation between the members ${\epsilon }_k$ ; in other words, knowing or observing the sequence until time $t$ , i.e., $({\epsilon }_1,...,{\epsilon }_t)$ does not allow to predict the next member ${\epsilon }_{t+1}$ any better than if nothing had been observed.
• “Random” means that in principle, an entry can take any value in some given set with certain chance, and that it is are notin advance which value it will take.
• “Identically distributed” means that each entry has the same chances to assume a possible value.

White noise is an example of an stationary, ergodic series. Stationarity means that the statistics don't change over time. Ergodicity means that statistical measures such as mean and variance can be estimated by observing enough samples of one single sequence $({\epsilon }_1,{\epsilon }_2,{\epsilon }_3,...)$ ; the mean or expectation Recall that $\mathrm{I}\mathrm{E}\left[X\right]$ denotes the expectation of the random variable $X$ . of an entry $\mathrm{I}\mathrm{E}\left[{\epsilon }_1\right]$ can be estimated as the sample mean $(1/N)({\epsilon }_1+...+{\epsilon }_N)$ .

An example of a stationary, non ergodic series is the one where ${\epsilon }_1$ equals 1 or $-1$ with equal probability, and all other ${\epsilon }_k$ are equal to ${\epsilon }_1$ . Clearly, the sample mean of one single sequence is then either 1 or $-1$ , but not 0 as it should.

An example of white noise is the error ${\epsilon }_k$ introduced by quantization, i.e., ${\epsilon }_k=y_k-x_k$ where $x_k$ is a “typical” signal before and $y_k$ the signal after quantization. Check of Randomness and Independence: as we are observing the first $t$ samples $({\epsilon }_1,...,{\epsilon }_t)$ we have no indication whatsoever on the quantization error of the $t+1$ st sample — unless the signal is very special. [An example of an atypical signal would be one that isalready quantized: after observing 500 times an error 0 we start to suspect that the future errors will also be 0.]Check of the distribution: a quantization done by rounding to the third decimal, e.g., will result in errors that lie between $[-0.00049999...,0.0005]$ where all values in this interval are equally likely. This means, e.g., that ${\epsilon }_k$ is negative with chance 1/2 and that, e.g., ${\epsilon }_k$ is within $[0.0002,0.0003]$ with chance $1/10$ . Since this is the same of all entries ${\epsilon }_k$ , they are identically distributed.

Spectral analysis of stationary signals and series

By their own nature, similarly to periodic signals, stationary signals and series have no finiteenergy. As a simple example consider the sequence ${\epsilon }_k=\pm 1$ with random sign. Since ${\epsilon }_k^2=1$ for all $k$ , the energy of the sequence is clearly infinite. In order to arrive at a meaningful spectral analysisone defines the power of a stationary signal $x\left(t\right)$ as the time average of the energy:

Note that for a periodic signal definition [link] gives the same value as [link] .

While periodic signals possess a natural Fourier expansion into a series, we need to take time-averaged, windowed Fourier transforms for stationary signals; also, due to randomness,one needs to take averages over different realizations in order to obtain a deterministic non-random spectral descriptor. Most useful is the power spectrum $S\left(f\right)$ which corresponds to the square of the absolute value of the time-averaged windowed Fourier transform: