4.I.9H

Markov Chains
Part IB, 2008

A Markov chain on the state-space I={1,2,3,4,5,6,7}I=\{1,2,3,4,5,6,7\} has transition matrix

P=(01/21/401/4001/301/2001/60000100000100000000010000000100001/201/2)P=\left(\begin{array}{ccccccc} 0 & 1 / 2 & 1 / 4 & 0 & 1 / 4 & 0 & 0 \\ 1 / 3 & 0 & 1 / 2 & 0 & 0 & 1 / 6 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 1 / 2 & 0 & 1 / 2 \end{array}\right)

Classify the chain into its communicating classes, deciding for each what the period is, and whether the class is recurrent.

For each i,jIi, j \in I say whether the limit1limnpij(n)\operatorname{limit}^{-1} \lim _{n \rightarrow \infty} p_{i j}^{(n)} exists, and evaluate the limit when it does exist.