What value of ββ is required so that the output of an FIR filter cannot overflow ( ∀n:|yn|≤1 n y n 1 , ∀n:|xn|≤1 n x n 1 )? |yn|=|∑k=0M-1hkβxn-k|≤∑k=0M-1|hk||β||xn-k|≤β∑k=0M-1|hk|1 y n k 0 M 1 h k β x n k k 0 M 1 h k β x n k β k M 1 0 h k 1 ⇓ ⇓ β<∑k=0M-1|hk| β k M 1 0 h k Alternatively, we can incorporate the scaling directly into the filter, and require that ∑k=0M-1|hk|<1 k M 1 0 h k 1 to prevent overflow.

To prevent the output from overflowing in an IIR filter, the condition above still holds: ( M=∞ M ) |yn|<∑k=0∞|hk| y n k 0 h k so an initial scaling factor β<1∑k=0∞|hk| β 1 k 0 h k can be used, or the filter itself can be scaled.

However, it is also necessary to prevent the states from overflowing, and to prevent overflow at any point in the signal flow graph where the arithmetic hardware would thereby produce errors. To prevent the states from overflowing, we determine the transfer function from the input to all states ii, and scale the filter such that ∀i:∑k=0∞| h i k|≤1 i k 0 h i k 1

Although this method of scaling guarantees no overflows, it is often too conservative. Note that a worst-case signal is xn=signh-n x n sign h n ; this input may be extremely unlikely. In the relatively common situation in which the input is expected to be mainly a single-frequency sinusoid of unknown frequency and amplitude less than 1, a scaling condition of ∀w:|Hw|≤1 w H w 1 is sufficient to guarantee no overflow. This scaling condition is often used. If there are several potential overflow locations ii in the digital filter structure, the scaling conditions are ∀i,w:| H i w|≤1 i w H i w 1 where H i w H i w is the frequency response from the input to location ii in the filter.

Even this condition may be excessively conservative, for example if the input is more-or-less random, or if occasional overflow can be tolerated. In practice, experimentation and simulation are often the best ways to optimize the scaling factors in a given application.

For filters implemented in the cascade form, rather than scaling for the entire filter at the beginning, (which introduces lots of quantization of the input) the filter is usually scaled so that each stage is just prevented from overflowing. This is best in terms of reducing the quantization noise. The scaling factors are incorporated either into the previous or the next stage, whichever is most convenient.

Some heurisitc rules for grouping poles and zeros in a cascade implementation are:

1. Order the poles in terms of decreasing radius. Take the pole pair closest to the unit circle and group it with the zero pair closest to that pole pair (to minimize the gain in that section). Keep doing this with all remaining poles and zeros.
2. Order the section with those with highest gain ( argmax| H i w| H i w ) in the middle, and those with lower gain on the ends.

Leland B. Jackson has an excellent intuitive discussion of finite-precision problems in digital filters. The book by Roberts and Mullis is one of the most thorough in terms of detail.