Finite-Precision Error Analysis 1.1 2003/07/16 12:12:37 GMT-5 2004/12/29 11:39:26.777 US/Central Douglas L. Jones dl-jones@uiuc.edu Douglas L. Jones dl-jones@uiuc.edu Kyle Clarkson kclarks@rice.edu correlation function dithering power spectral density quantization roundoff error truncation error Exact analysis of quantization errors is difficult because quantization is highly nonlinear. Approximating quantization errors as independent, additive white Gaussian noise processes makes analysis tractable and generally leads to fairly accurate results. Dithering can be used to make these approximations more accurate.

Fundamental Assumptions in finite-precision error analysis Quantization is a highly nonlinear process and is very difficult to analyze precisely. Approximations and assumptions are made to make analysis tractable.

Assumption #1 The roundoff or truncation errors at any point in a system at each time are random, stationary, and statistically independent (white and independent of all other quantizers in a system). That is, the error autocorrelation function is r e k e n e n + k σ q 2 δ k . Intuitively, and confirmed experimentally in some (but not all!) cases, one expects the quantization error to have a uniform distribution over the interval Δ 2 Δ 2 for rounding, or Δ 0 for truncation. In this case, rounding has zero mean and variance Q x n x n 0 σ Q 2 e n 2 Δ B 2 12 and truncation has the statistics Q x n x n Δ 2 σ Q 2 Δ B 2 12 Please note that the independence assumption may be very bad (for example, when quantizing a sinusoid with an integer period N). There is another quantizing scheme called dithering, in which the values are randomly assigned to nearby quantization levels. This can be (and often is) implemented by adding a small (one- or two-bit) random input to the signal before a truncation or rounding quantizer.

This is used extensively in practice. Altough the overall error is somewhat higher, it is spread evenly over all frequencies, rather than being concentrated in spectral lines. This is very important when quantizing sinusoidal or other periodic signals, for example.

Assumption #2 Pretend that the quantization error is really additive Gaussian noise with the same mean and variance as the uniform quantizer. That is, model

as

This model is a linear system, which our standard theory can handle easily. We model the noise as Gaussian because it remains Gaussian after passing through filters, so analysis in a system context is tractable.

Summary of Useful Statistical Facts correlation function ≐ r x k x n x n + k power spectral density ≐ S x w DTFT r x n Note r x 0 σ x 2 1 2 w S x w ≐ r xy k x * n y n k cross-spectral density S xy w DTFT r xy n For y h x : S yx w H w S x w S yy w H w 2 S x w Note that the output noise level after filtering a noise sequence is σ y 2 r yy 0 1 w H w 2 S x w so postfiltering quantization noise alters the noise power spectrum and may change its variance! For x 1 , x 2 statistically independent r x 1 + x 2 k r x 1 k r x 2 k S x 1 + x 2 w S x 1 w S x 2 w For independent random variables σ x 1 + x 2 2 σ x 1 2 σ x 2 2