Fundamental Assumptions in finite-precision error analysis

Assumption #1

The roundoff or truncation errors at any point in a system at each time are random, stationary, and statistically independent (white and independent of all other quantizers in a system).

That is, the error autocorrelation function is r e k=E e n e n + k = σ q 2δk r e k e n e n + k σ q 2 δ k . Intuitively, and confirmed experimentally in some (but not all!) cases, one expects the quantization error to have a uniform distribution over the interval -Δ2Δ2 Δ 2 Δ 2 for rounding, or -Δ0 Δ 0 for truncation.

In this case, rounding has zero mean and variance EQ x n - x n =0 Q x n x n 0 σ Q 2=E e n 2= Δ B 212 σ Q 2 e n 2 Δ B 2 12 and truncation has the statistics EQ x n - x n =-Δ2 Q x n x n Δ 2 σ Q 2= Δ B 212 σ Q 2 Δ B 2 12

Please note that the independence assumption may be very bad (for example, when quantizing a sinusoid with an integer period NN). There is another quantizing scheme called dithering, in which the values are randomly assigned to nearby quantization levels. This can be (and often is) implemented by adding a small (one- or two-bit) random input to the signal before a truncation or rounding quantizer.

Figure 1

This is used extensively in practice. Altough the overall error is somewhat higher, it is spread evenly over all frequencies, rather than being concentrated in spectral lines. This is very important when quantizing sinusoidal or other periodic signals, for example.

Assumption #2

Pretend that the quantization error is really additive Gaussian noise with the same mean and variance as the uniform quantizer. That is, model

Subfigure 2.1 as Subfigure 2.2 Figure 2

This model is a linear system, which our standard theory can handle easily. We model the noise as Gaussian because it remains Gaussian after passing through filters, so analysis in a system context is tractable.

Summary of Useful Statistical Facts

• correlation function - r x k≐E x n x n + k ≐ r x k x n x n + k
• power spectral density - S x w≐DTFT r x n ≐ S x w DTFT r x n
• Note r x 0= σ x 2=12π∫-ππ S x wdw r x 0 σ x 2 1 2 w S x w
• r xy k≐E x * nyn+k ≐ r xy k x * n y n k
• cross-spectral density - S xy w=DTFT r xy n S xy w DTFT r xy n
• For y=h*x y h x : S yx w=Hw S x w S yx w H w S x w S yy w=|Hw|2 S x w S yy w H w 2 S x w
• Note that the output noise level after filtering a noise sequence is σ y 2= r yy 0=1π∫-ππ|Hw|2 S x wdw σ y 2 r yy 0 1 w H w 2 S x w so postfiltering quantization noise alters the noise power spectrum and may change its variance!
• For x 1 x 1 , x 2 x 2 statistically independent r x 1 + x 2 k= r x 1 k+ r x 2 k r x 1 + x 2 k r x 1 k r x 2 k S x 1 + x 2 w= S x 1 w+ S x 2 w S x 1 + x 2 w S x 1 w S x 2 w
• For independent random variables σ x 1 + x 2 2= σ x 1 2+ σ x 2 2 σ x 1 + x 2 2 σ x 1 2 σ x 2 2