1. Home
  2. An introduction to source-coding
  3. Dpcm
  4. Analysis of dpcm using rate

2.3 Analysis of dpcm using rate-distortion theory

<< Chapter < Page
  An introduction to source-coding   Page 1 / 1
Chapter >> Page >
Using rate-distortion theory, the optimal SNR attainable for rate-R source coding is related to R and the spectral flatness measure of the source. The SNR of rate-R DPCM is then analyzed and compared to the optimal, and shown to suffer by only 1.53 dB.
  • The rate-distortion function R ( D ) specifies the minimum average rate R required to transmit the source process at a mean distortion of D , while the distortion-rate function D ( R ) specifies the minimum mean distortion D resulting from transmission of the source at average rate R . These bounds are theoretical in the sense that coding techniques whichattain these minimum rates or distortions are in general unknown and thought to be infinitely complex as well as require infinite memory.Still, these bounds form a reference against which any specific coding system can be compared.For a continuous-amplitude white (i.e., “memoryless”) Gaussian source x ( n ) (see Berger and Jayant&Noll),
    R ( D ) = 1 2 log 2 σ x 2 D 0 ≤ D ≤ σ x 2 0 D ≥ σ x 2 D ( R ) = 2 - 2 R σ x 2 .
    The sources we are interested in, however, are non-white. It turns out that when distortion D is “small,” non-white Gaussian x ( n ) have the following distortion-rate function: (see page 644 of Jayant&Noll)
    D ( R ) = 2 - 2 R exp 1 2 π ∫ - π π ln S x ( e j ω ) d ω = 2 - 2 R σ x 2 exp 1 2 π ∫ - π π ln S x ( e j ω ) d ω 1 2 π ∫ - π π S x ( e j ω ) d ω ︸ spectral flatness measure .
    Note the ratio of geometric to arithmetic PSD means, called the spectral flatness measure . Thus optimal coding of x ( n ) yields
    SNR ( R ) = 10 log 10 σ x 2 D ( R ) ≈ 6 . 02 R - 10 log 10 ( SFM x ) .
    To summarize, [link] (lower equation) gives the best possible SNR for any arbitrarily-complex coding system that transmits/stores information at an average rate of R bits/sample.
  • Let's compare the SNR-versus-rate performance acheivable by DPCM to the optimal given by [link] (lower equation). The structure we consider is shown in [link] , where quantized DPCM outputs e ˜ ( n ) are coded into binary bits using an entropy coder.Assuming that e ˜ ( n ) is white (which is a good assumption for well-designed predictors), optimal entropy coding/decoding is ableto transmit and recover e ˜ ( n ) at R = H e ˜ bits/sample without any distortion. H e ˜ is the entropy of e ˜ ( n ) , for which we derived the following expression assuming large- L uniform quantizer:
    H e ˜ = h e - 1 2 log 2 12 var ( e ( n ) - e ˜ ( n ) ) .
    Since var ( e ( n ) - e ˜ ( n ) ) = σ r 2 in DPCM, H e ˜ can be rewritten:
    H e ˜ = h e - 1 2 log 2 12 σ r 2 .
    If e ( n ) is Gaussian, it can be shown that the differential entropy h e takes on the value
    h e = 1 2 log 2 2 π e σ e 2 ,
    so that
    H e ˜ = 1 2 log 2 π e σ e 2 6 σ r 2 .
    Using R = H e ˜ and rearranging the previous expression, we find
    σ r 2 = π e 6 2 - 2 R σ e 2 .
    With the optimal infinite length predictor, σ e 2 equals σ e 2 | min given by equation 10 from Performance of DPCM . Plugging equation 10 from Performance of DPCM into the previous expression and writing the result in terms of the spectral flatness measure,
    σ r 2 = π e 6 2 - 2 R σ x 2 SFM x .
    Translating into SNR, we obtain
    SNR = 10 log 10 σ x 2 σ r 2 ≈ 6 . 02 R - 1 . 53 - 10 log 10 SFM x [dB] .
    To summarize, a DPCM system using a MSE-optimal infinite-length predictor and optimal entropy coding of e ˜ ( n ) could operate at an average of R bits/sample with the SNR in [link] (lower equation).
    This figure is generally similar to the flow charts in figure 5, titled A Typical Differential PCM System. The labels and structure is all identical, except that in between the two arrows labeled e-tilde(n) are two boxes labeled entropy encoder, with arrows to the left and right of them that continuing the flow-movement to the right. This figure is generally similar to the flow charts in figure 5, titled A Typical Differential PCM System. The labels and structure is all identical, except that in between the two arrows labeled e-tilde(n) are two boxes labeled entropy encoder, with arrows to the left and right of them that continuing the flow-movement to the right.
    Entropy-Encoded DPCM System.
  • Comparing [link] (lower equation) and [link] (lower equation), we see that DPCM incurs a 1.5 dB penalty in SNR when compared to the optimal.From our previous discussion on optimal quantization, we recognize that this 1.5 dB penalty comes from the fact that thequantizer in the DPCM system is memoryless. (Note that the DPCM quantizer must be memoryless since the predictor input must not be delayed.)
  • Though we have identified a 1.5 dB DPCM penalty with respect to optimal, a key point to keep in mind is that the design of near-optimalcoders for non-white signals is extremely difficult. When the signal statistics are rapidly changing, such a design taskbecomes nearly impossible. Though still non-trivial to design, near-optimal entropy coders for white signals exist and are widely used in practice. Thus, DPCM can be thought of as a way of pre-processing a colored signalthat makes near-optimal coding possible. From this viewpoint, 1.5 dB might not be considered a high price to pay.
<< Chapter < Page Page > Chapter >>

Read also:

Get Jobilize Job Search Mobile App in your pocket Now!

Get it on Google Play Download on the App Store Now




Source:  OpenStax, An introduction to source-coding: quantization, dpcm, transform coding, and sub-band coding. OpenStax CNX. Sep 25, 2009 Download for free at http://cnx.org/content/col11121/1.2
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'An introduction to source-coding: quantization, dpcm, transform coding, and sub-band coding' conversation and receive update notifications?

Ask