Paper 2, Section II, I

Coding and Cryptography
Part II, 2014

What is the information capacity of a memoryless, time-independent channel? Compute the information capacity of a binary symmetric channel with probability pp of error. Show the steps in your computation.

Binary digits are transmitted through a noisy channel, which is memoryless and time-independent. With probability α(0<α<1)\alpha(0<\alpha<1) the digit is corrupted and noise is received, otherwise the digit is transmitted unchanged. So, if we denote the input by 0 and 1 and the output as 0,0, * and 1 with * denoting the noise, the transition matrix is

(1α0αα01α)\left(\begin{array}{cc} 1-\alpha & 0 \\ \alpha & \alpha \\ 0 & 1-\alpha \end{array}\right)

Compute the information capacity of this channel.

Explain how to code a message for transmission through the channel described above, and how to decode it, so that the probability of error for each bit is arbitrarily small.