Paper 3, Section I, H

Markov Chains
Part IB, 2011

Let (Xn)n0\left(X_{n}\right)_{n \geqslant 0} be a Markov chain with state space SS.

(i) What does it mean to say that (Xn)n0\left(X_{n}\right)_{n} \geqslant 0 has the strong Markov property? Your answer should include the definition of the term stopping time.

(ii) Show that

P(Xn=i at least k times X0=i)=[P(Xn=i at least once X0=i)]k\mathbb{P}\left(X_{n}=i \text { at least } k \text { times } \mid X_{0}=i\right)=\left[\mathbb{P}\left(X_{n}=i \text { at least once } \mid X_{0}=i\right)\right]^{k}

for a state iSi \in S. You may use without proof the fact that (Xn)n0\left(X_{n}\right)_{n \geqslant 0} has the strong Markov property.