Paper 3, Section I, G
Part II, 2015
Let be a random variable that takes each value in the finite alphabet with probability . Show that, if each is an integer greater than 0 and , then there is a decodable binary code with each codeword having length .
Prove that, for any decodable code , we have
where is the entropy of the random variable . When is there equality in this inequality?