Paper 4, Section I, E

Markov Chains
Part IB, 2010

Consider a Markov chain (Xn)n0\left(X_{n}\right)_{n} \geqslant 0 with state space {a,b,c,d}\{a, b, c, d\} and transition probabilities given by the following table.

\begin{tabular}{c|cccc} & aa & bb & cc & dd \ \hlineaa & 1/41 / 4 & 1/41 / 4 & 1/21 / 2 & 0 \ bb & 0 & 1/41 / 4 & 0 & 3/43 / 4 \ cc & 1/21 / 2 & 0 & 1/41 / 4 & 1/41 / 4 \ dd & 0 & 1/21 / 2 & 0 & 1/21 / 2 \end{tabular}

By drawing an appropriate diagram, determine the communicating classes of the chain, and classify them as either open or closed. Compute the following transition and hitting probabilities:

  • P(Xn=bX0=d)\mathbb{P}\left(X_{n}=b \mid X_{0}=d\right) for a fixed n0n \geqslant 0

  • P(Xn=c\mathbb{P}\left(X_{n}=c\right. for some n1X0=a)\left.n \geqslant 1 \mid X_{0}=a\right).