Paper 1, Section I, G

Coding and Cryptography
Part II, 2019

Let XX and YY be discrete random variables taking finitely many values. Define the conditional entropy H(XY)H(X \mid Y). Suppose ZZ is another discrete random variable taking values in a finite alphabet, and prove that

H(XY)H(XY,Z)+H(Z)H(X \mid Y) \leqslant H(X \mid Y, Z)+H(Z)

[You may use the equality H(X,Y)=H(XY)+H(Y)H(X, Y)=H(X \mid Y)+H(Y) and the inequality H(XY)H(X \mid Y) \leqslant H(X).]H(X) .]

State and prove Fano's inequality.