7.7 Shannon's noisy channel coding theorem

It is highly recommended that the information presented in Mutual Information and in Typical Sequences be reviewed before proceeding with this document. An introductory module on thetheorem is available at Noisy Channel Theorems .

Shannon's noisy channel coding

The capacity of a discrete-memoryless channel is given by

Before we consider continuous-time additive white Gaussian channels, let's concentrate on discrete-time Gaussian channels

Consider an output block of size $n$

On the other hand since $X_i$ 's are power constrained and ${}_i$ and $X_i$ 's are independent

How many $X$ 's can we transmit to have nonoverlapping $Y$ spheres in the output domain? The question is how many spheres of radius $\sqrt{n{}_{}^2}$ fit in a sphere of radius $\sqrt{n(P+{}_{}^2)}$ .

The capacity of a discrete-time Gaussian channel $C=\frac{1}{2}\log_2(1+\frac{P}{{}_{}^2})$ bits per channel use.

When the channel is a continuous-time, bandlimited, additive white Gaussian with noise power spectral density $\frac{N_0}{2}$ and input power constraint $P$ and bandwidth $W$ . The system can be sampled at the Nyquist rate to provide power per sample $P$ and noise power