Fast Convolution by the FFT

One of the main applications of the FFT is to do convolution more efficiently than the direct calculation from the definition which is:

which can also be written as:

This is often used to filter a signal x(n)x(n) with a filter whose impulse response is h(n)h(n). Each output value y(n)y(n) requires NN multiplications and N-1N-1 additions if y(n)y(n) has NN terms. So, for NN output values, on the order of N2N2 arithmetic operations are required.

Because the DFT converts convolution to multiplication:

can be calculated with the FFT and bring the order of arithmetic operations down to Nlog(N)Nlog(N) which can be significate with large NN.

This approach, which is called “fast convolutions", is a form of block processing since a whole block of x(n)x(n) must be available to calculate even one output value, y(n)y(n). So, a time delay of one block length is always required. Another problem is the filtering use of convolution is usually non-cyclic and the convolution implemented with the DFT is cyclic. This is dealt with by appending zeros to x(n)x(n) and h(n)h(n) such that the output of the cyclic convolution gives one block of the output of the desired non-cyclic convolution.

For filtering and some other applications, one want “on going" convolution where the filter response h(n)h(n) may be finite in length or duration, but the input x(n)x(n) is of arbitrary length. Two methods have traditionally used to break the input into blocks and use the FFT to convolve the block so that the output that would have been calculated by directly implementing Equation 1 or Equation 3 can be constructed efficiently. These are called “overlap-add" and “over-lap save".

Block Processing, a Generalization of Overlap Methods

Convolution is intimately related to the DFT. It was shown in The DFT as Convolution or Filtering that a prime length DFT could be converted to cyclic convolution. It has been long known Entry 48 that convolution can be calculated by multiplying the DFTs of signals.

An important question is what is the fastest method for calculating digital convolution. There are several methods that each have some advantage. The earliest method for fast convolution was the use of sectioning with overlap-add or overlap-save and the FFT Entry 48, Entry 53, Entry 10. In most cases the convolution is of real data and, therefore, real-data FFTs should be used. That approach is still probably the fastest method for longer convolution on a general purpose computer or microprocessor. The shorter convolutions should simply be calculated directly.

Introduction

The partitioning of long or infinite strings of data into shorter sections or blocks has been used to allow application of the FFT to realize on-going or continuous convolution Entry 57, Entry 30. This section develops the idea of block processing and shows that it is a generalization of the overlap-add and overlap-save methods Entry 57, Entry 26. They further generalize the idea to a multidimensional formulation of convolution Entry 3, Entry 15. Moving in the opposite direction, it is shown that, rather than partitioning a string of scalars into blocks and then into blocks of blocks, one can partition a scalar number into blocks of bits and then include the operation of multiplication in the signal processing formulation. This is called distributed arithmetic Entry 14 and, since it describes operations at the bit level, is completely general. These notes try to present a coherent development of these ideas.

Block Signal Processing

In this section the usual convolution and recursion that implements FIR and IIR discrete-time filters are reformulated in terms of vectors and matrices. Because the same data is partitioned and grouped in a variety of ways, it is important to have a consistent notation in order to be clear. The nthnth element of a data sequence is expressed h(n)h(n) or, in some cases to simplify, hnhn. A block or finite length column vector is denoted h̲nh̲n with nn indicating the nthnth block or section of a longer vector. A matrix, square or rectangular, is indicated by an upper case letter such as HH with a subscript if appropriate.