1.I.4G

Coding and Cryptography
Part II, 2007

Let Σ1\Sigma_{1} and Σ2\Sigma_{2} be alphabets of sizes mm and aa. What does it mean to say that f:Σ1Σ2f: \Sigma_{1} \rightarrow \Sigma_{2}^{*} is a decipherable code? State the inequalities of Kraft and Gibbs, and deduce that if letters are drawn from Σ1\Sigma_{1} with probabilities p1,,pmp_{1}, \ldots, p_{m} then the expected word length is at least H(p1,,pm)/logaH\left(p_{1}, \ldots, p_{m}\right) / \log a.