0.13 Coding  (Page 11/22)

Channel capacity

"What Is Information?" showed how much information (measured in bits) is contained in a given symbol,and "Entropy" generalized this to the average amount of information containedin a sequence or set of symbols (measured in bits per symbol). In order to be useful in a communication system, however,the data must move from one place to another. What is the maximum amount of information that canpass through a channel in a given amount of time? The main result of this section is that thecapacity of the channel defines the maximum possible flow of information through the channel.The capacity is a function of the bandwidth of the channel and of the amount of noise inthe system, and it is measured in bits per second.

If the data is encoded using $N=2$ equally probable bits per symbol, and if the symbols are independent, theentropy is $H_2=0.5log\left(2\right)+0.5log\left(2\right)=1$ bit per symbol. Why not increase the number of bits per symbol? This would allow representing moreinformation. Doubling to $N=4$ , the entropy increases to $H_4=2$ . In general, when using $N$ bits, the entropy is $H_N=log\left(N\right)$ . By increasing $N$ without bound, the entropy can be increased without bound! But is it really possibleto send an infinite amount of information?

When doubling the size of $N$ , one of two things must happen. Either the distance between the levels must decrease,or the power must increase. For instance, it is common to represent the binary signal as $\pm 1$ and the four-level signal as $\pm 1,\pm 3$ . In this representation, the distance between neighboringvalues is constant, but the power in the signal has increased. Recall that the power in a discrete signal $x\left[k\right]$ is

For a binary signal with equal probabilities, this is $P_2=\frac{1}{2}(1^2+{(-1)}^2)=1$ . The four-level signal has power $P_4=\frac{1}{4}(1^2+{(-1)}^2+3^2+{(-3)}^2)=5$ . To normalize the power to unity for the four-level signal,calculate the value $x$ such that $\frac{1}{4}(x^2+{(-x)}^2+{\left(3x\right)}^2+{(-3x)}^2)=1$ , which is $x=\sqrt{1/5}$ . [link] shows how the values of the $N$ -level signal become closer together as $N$ increases, when the power is held constant.

Now it will be clearer why it is not really possible to send an infinite amount of information in a single symbol.For a given transmitter power, the amplitudes become closer together for large $N$ , and the sensitivity to noise increases. Thus, when there isnoise (and some is inevitable), the four-level signal is more prone to errors than the two-level signal.Said another way, a higher signal-to-noise ratio As the term suggests, SNR is the ratio of the energy (or power)in the signal to the energy (or power) in the noise. (SNR) is needed to maintain the same probability of error in the four-level signal as in the two-level signal.

Consider the situation in terms of the bandwidth required to transmit a given set of data containing $M$ bits of information.From the Nyquist sampling theorem of [link] , data can be sent through a channel of bandwidth $B$ at a maximum rate of $2B$ symbols per second. If these symbols are coded into two levels, then $M$ symbols must be sent. If the data are transmitted with four levels (by assigning pairs of binary digits to each four-level symbol),then only $M/2$ symbols are required. Thus the multilevel signal can operate at half the symbol rate ofthe binary signal. Said another way, the four-level signal requires only half the bandwidth of the two-level signal.