1.I.11H

Markov Chains
Part IB, 2004

Let P=(Pij)P=\left(P_{i j}\right) be a transition matrix. What does it mean to say that PP is (a) irreducible, (b)(b) recurrent?

Suppose that PP is irreducible and recurrent and that the state space contains at least two states. Define a new transition matrix P~\tilde{P} by

P~ij={0 if i=j(1Pii)1Pij if ij\tilde{P}_{i j}=\left\{\begin{array}{lll} 0 & \text { if } & i=j \\ \left(1-P_{i i}\right)^{-1} P_{i j} & \text { if } & i \neq j \end{array}\right.

Prove that P~\tilde{P} is also irreducible and recurrent.