2.II.22H

Markov Chains
Part IB, 2004

Consider a Markov chain on the state space S={0,1,2,}{1,2,3,}S=\{0,1,2, \ldots\} \cup\left\{1^{\prime}, 2^{\prime}, 3^{\prime}, \ldots\right\} with transition probabilities as illustrated in the diagram below, where 0<q<10<q<1 and p=1qp=1-q.

For each value of q,0<q<1q, 0<q<1, determine whether the chain is transient, null recurrent or positive recurrent.

When the chain is positive recurrent, calculate the invariant distribution.