Paper 2, Section II, I
What is the information capacity of a memoryless, time-independent channel? Compute the information capacity of a binary symmetric channel with probability of error. Show the steps in your computation.
Binary digits are transmitted through a noisy channel, which is memoryless and time-independent. With probability the digit is corrupted and noise is received, otherwise the digit is transmitted unchanged. So, if we denote the input by 0 and 1 and the output as and 1 with denoting the noise, the transition matrix is
Compute the information capacity of this channel.
Explain how to code a message for transmission through the channel described above, and how to decode it, so that the probability of error for each bit is arbitrarily small.