Paper 4, Section I, E
Part IB, 2010
Consider a Markov chain with state space and transition probabilities given by the following table.
\begin{tabular}{c|cccc} & & & & \ \hline & & & & 0 \ & 0 & & 0 & \ & & 0 & & \ & 0 & & 0 & \end{tabular}
By drawing an appropriate diagram, determine the communicating classes of the chain, and classify them as either open or closed. Compute the following transition and hitting probabilities:
for a fixed
for some .