B1.14

Information Theory
Part II, 2004

State the formula for the capacity of a memoryless channel.

(a) Consider a memoryless channel where there are two input symbols, AA and BB, and three output symbols, A,BA, B, *. Suppose each input symbol is left intact with probability 1/21 / 2, and transformed into a * with probability 1/21 / 2. Write down the channel matrix, and calculate the capacity.

(b) Now suppose the output is further processed by someone who cannot distinguish AA and *, so that the matrix becomes

(101/21/2)\left(\begin{array}{cc} 1 & 0 \\ 1 / 2 & 1 / 2 \end{array}\right)

Calculate the new capacity.