A typical baseband digital system is described in Figure 1(a). At the transmitter, the modulated pulses are filtered to comply with some bandwidth constraint. These pulses are distorted by the reactances of the cable or by fading in the wireless systems. Figure 1(b) illustrates a convenient model, lumping all the filtering into one overall equivalent system transfer function.

H
(
f
)
=
H
t
(
f
)
.
H
c
(
f
)
.
H
r
(
f
)
H
(
f
)
=
H
t
(
f
)
.
H
c
(
f
)
.
H
r
(
f
)
size 12{H \( f \) =H rSub { size 8{t} } \( f \) "." H rSub { size 8{c} } \( f \) "." H rSub { size 8{r} } \( f \) } {}

Due to the effects of system filtering, the received pulses can overlap one another as shown in Figure 1(b). Such interference is termed InterSymbol Interfernce (ISI). Even in the absence of noise, the effects of filtering and channel-induced distortion lead to ISI.

Nyquist investigated and showed that theoretical minimum system bandwidth needed in order to detect
RsRs size 12{R rSub { size 8{s} } } {} symbols/s, without ISI, is
Rs/2Rs/2 size 12{R rSub { size 8{s} } /2} {} or
1/2T1/2T size 12{1/2T} {} hertz. For baseband systems, when
H(f)H(f) size 12{H \( f \) } {} is such a filter with single-sided bandwidth
1/2T1/2T size 12{1/2T} {} (the ideal Nyquist filter) as shown in figure 2a, its impulse response is of the form
h(t)=sinc(t/T)h(t)=sinc(t/T) size 12{h \( t \) ="sin"c \( t/T \) } {}, shown in figure 2b. This
sinc(t/T)sinc(t/T) size 12{"sin"c \( t/T \) } {}-shaped pulse is called the ideal Nyquist pulse. Even though two successive pulses
h(t)h(t) size 12{h \( t \) } {} and
h(t−T)h(t−T) size 12{h \( t - T \) } {} with long tail, the figure shows all tail of
h(t)h(t) size 12{h \( t \) } {} passing through zero amplitude at the instant when
h(t−T)h(t−T) size 12{h \( t - T \) } {} is to be sampled. Therefore, assuming that the synchronization is perfect, there will be no ISI.

Figure 2 Nyquist channels for zero ISI. (a) Rectangular system transfer function H(f). (b) Received pulse shape
h(t)=sinc(t/T)h(t)=sinc(t/T) size 12{h \( t \) ="sin"c \( t/T \) } {}

The names "Nyquist filter" and "Nyquist pulse" are often used to describe the general class of filtering and pulse-shaping that satisfy zero ISI at the sampling points. Among the class of Nyquist filters, the most popular ones are the raised cosine and root-raised cosine.

A fundamental parameter for communication system is bandwidth efficiency,
R/WR/W size 12{R/W} {} bits/s/Hz. For ideal Nyquist filtering, the theoretical maximum symbol-rate packing without ISI is
2symbols/s/Hz2symbols/s/Hz size 12{2 ital "symbols"/s/ ital "Hz"} {}. For example, with 64-ary PAM,
M=64=26M=64=26 size 12{M="64"=2 rSup { size 8{6} } } {} amplitudes, the theoretical maximum bandwidth efficiency is possible without ISI is
6bits/symbol.2symbols/s/Hz=12bits/s/Hz6bits/symbol.2symbols/s/Hz=12bits/s/Hz size 12{6 ital "bits"/ ital "symbol" "." 2 ital "symbols"/s/ ital "Hz"="12" ital "bits"/s/ ital "Hz"} {}.