Summary: A discussion of channels and how much information can be sent through a channel reliably.

In the previous section, we discussed information sources and quantified information. We also discussed how to represent (and compress) information sources in binary symbols in an efficient manner. In this section, we consider channels and will find out how much information can be sent through the channel reliably.

We will first consider simple channels where the input is a discrete random variable and the output is also a discrete random variable. These discrete channels could represent analog channels with modulation and demodulation and detection.

Let us denote the input sequence to the channel as

The channel output

The statistical properties of a channel are determined if one finds

A binary symmetric channel (BSC) is a discrete memoryless channel with
binary input and binary output and

It is interesting to note that every time a BSC is used one bit
is sent across the channel with probability of error of