1.II.19C

Markov Chains
Part IB, 2007

Consider a Markov chain (Xn)n0\left(X_{n}\right)_{n \geqslant 0} on states {0,1,,r}\{0,1, \ldots, r\} with transition matrix (Pij)\left(P_{i j}\right), where P0,0=1=Pr,rP_{0,0}=1=P_{r, r}, so that 0 and rr are absorbing states. Let

A=(Xn=0, for some n0)A=\left(X_{n}=0, \text { for some } n \geqslant 0\right) \text {, }

be the event that the chain is absorbed in 0 . Assume that hi=P(AX0=i)>0h_{i}=\mathbb{P}\left(A \mid X_{0}=i\right)>0 for 1i<r1 \leqslant i<r.

Show carefully that, conditional on the event A,(Xn)n0A,\left(X_{n}\right)_{n \geqslant 0} is a Markov chain and determine its transition matrix.

Now consider the case where Pi,i+1=12=Pi,i1P_{i, i+1}=\frac{1}{2}=P_{i, i-1}, for 1i<r1 \leqslant i<r. Suppose that X0=i,1i<rX_{0}=i, 1 \leqslant i<r, and that the event AA occurs; calculate the expected number of transitions until the chain is first in the state 0 .