Skip to content Skip to navigation Skip to collection information

Connexions

You are here: Home » Content » An Introduction to Source-Coding: Quantization, DPCM, Transform Coding, and Sub-band Coding » Analysis of DPCM using Rate-Distortion Theory

Navigation

Lenses

What is a lens?

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

This content is ...

Affiliated with (What does "Affiliated with" mean?)

This content is either by members of the organizations listed or about topics related to the organizations listed. Click each link to see a list of all content affiliated with the organization.

Also in these lenses

  • UniqU content

    This collection is included inLens: UniqU's lens
    By: UniqU, LLC

    Click the "UniqU content" link to see all content selected in this lens.

  • Lens for Engineering

    This module and collection are included inLens: Lens for Engineering
    By: Sidney Burrus

    Click the "Lens for Engineering" link to see all content selected in this lens.

Recently Viewed

This feature requires Javascript to be enabled.

Tags

(What is a tag?)

These tags come from the endorsement, affiliation, and other lenses that include this content.
 

Analysis of DPCM using Rate-Distortion Theory

Module by: Phil Schniter. E-mail the author

Summary: Using rate-distortion theory, the optimal SNR attainable for rate-R source coding is related to R and the spectral flatness measure of the source. The SNR of rate-R DPCM is then analyzed and compared to the optimal, and shown to suffer by only 1.53 dB.

  • The rate-distortion functionR(D)R(D) specifies the minimum average rate R required to transmit the source process at a mean distortion of D, while the distortion-rate functionD(R)D(R) specifies the minimum mean distortion D resulting from transmission of the source at average rate R. These bounds are theoretical in the sense that coding techniques which attain these minimum rates or distortions are in general unknown and thought to be infinitely complex as well as require infinite memory. Still, these bounds form a reference against which any specific coding system can be compared. For a continuous-amplitude white (i.e., “memoryless”) Gaussian source x(n)x(n) (see Berger and Jayant & Noll),
    R(D)=12log2σx2D0≤D≤σx20D≥σx2D(R)=2-2Rσx2.R(D)=12log2σx2D0≤D≤σx20D≥σx2D(R)=2-2Rσx2.
    (1)
    The sources we are interested in, however, are non-white. It turns out that when distortion D is “small,” non-white Gaussian x(n)x(n) have the following distortion-rate function: (see page 644 of Jayant & Noll)
    D(R)=2-2Rexp12π∫-ππlnSx(ejω)dω=2-2Rσx2exp12π∫-ππlnSx(ejω)dω12π∫-ππSx(ejω)dω︸spectralflatnessmeasure.D(R)=2-2Rexp12π∫-ππlnSx(ejω)dω=2-2Rσx2exp12π∫-ππlnSx(ejω)dω12π∫-ππSx(ejω)dω︸spectralflatnessmeasure.
    (2)
    Note the ratio of geometric to arithmetic PSD means, called the spectral flatness measure. Thus optimal coding of x(n)x(n) yields
    SNR (R)=10log10σx2D(R)≈6.02R-10log10(SFMx). SNR (R)=10log10σx2D(R)≈6.02R-10log10(SFMx).
    (3)
    To summarize, Equation 3 (lower equation) gives the best possible SNR for any arbitrarily-complex coding system that transmits/stores information at an average rate of R bits/sample.
  • Let's compare the SNR-versus-rate performance acheivable by DPCM to the optimal given by Equation 3 (lower equation). The structure we consider is shown in Figure 1, where quantized DPCM outputs e˜(n)e˜(n) are coded into binary bits using an entropy coder. Assuming that e˜(n)e˜(n) is white (which is a good assumption for well-designed predictors), optimal entropy coding/decoding is able to transmit and recover e˜(n)e˜(n) at R=He˜R=He˜ bits/sample without any distortion. He˜He˜ is the entropy of e˜(n)e˜(n), for which we derived the following expression assuming large-L uniform quantizer:
    He˜=he-12log212var(e(n)-e˜(n)).He˜=he-12log212var(e(n)-e˜(n)).
    (4)
    Since var(e(n)-e˜(n))=σr2var(e(n)-e˜(n))=σr2 in DPCM, He˜He˜ can be rewritten:
    He˜=he-12log212σr2.He˜=he-12log212σr2.
    (5)
    If e(n)e(n) is Gaussian, it can be shown that the differential entropy he takes on the value
    he=12log22πeσe2,he=12log22πeσe2,
    (6)
    so that
    He˜=12log2πeσe26σr2.He˜=12log2πeσe26σr2.
    (7)
    Using R=He˜R=He˜ and rearranging the previous expression, we find
    σr2=πe62-2Rσe2.σr2=πe62-2Rσe2.
    (8)
    With the optimal infinite length predictor, σe2 equals σe2|minσe2|min given by equation 10 from Performance of DPCM. Plugging equation 10 from Performance of DPCM into the previous expression and writing the result in terms of the spectral flatness measure,
    σr2=πe62-2Rσx2SFMx.σr2=πe62-2Rσx2SFMx.
    (9)
    Translating into SNR, we obtain
    SNR =10log10σx2σr2≈6.02R-1.53-10log10SFMx[dB]. SNR =10log10σx2σr2≈6.02R-1.53-10log10SFMx[dB].
    (10)
    To summarize, a DPCM system using a MSE-optimal infinite-length predictor and optimal entropy coding of e˜(n)e˜(n) could operate at an average of R bits/sample with the SNR in Equation 10 (lower equation).
    Figure 1: Entropy-Encoded DPCM System.
    This figure is generally similar to the flow charts in figure 5, titled A Typical Differential PCM System. The labels and structure is all identical, except that in between the two arrows labeled e-tilde(n) are two boxes labeled entropy encoder, with arrows to the left and right of them that continuing the flow-movement to the right.
  • Comparing Equation 3 (lower equation) and Equation 10 (lower equation), we see that DPCM incurs a 1.5 dB penalty in SNR when compared to the optimal. From our previous discussion on optimal quantization, we recognize that this 1.5 dB penalty comes from the fact that the quantizer in the DPCM system is memoryless. (Note that the DPCM quantizer must be memoryless since the predictor input must not be delayed.)
  • Though we have identified a 1.5 dB DPCM penalty with respect to optimal, a key point to keep in mind is that the design of near-optimal coders for non-white signals is extremely difficult. When the signal statistics are rapidly changing, such a design task becomes nearly impossible. Though still non-trivial to design, near-optimal entropy coders for white signals exist and are widely used in practice. Thus, DPCM can be thought of as a way of pre-processing a colored signal that makes near-optimal coding possible. From this viewpoint, 1.5 dB might not be considered a high price to pay.

Collection Navigation

Content actions

Download:

Collection as:

EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Module as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Add:

Collection to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks

Module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks