Let (Xn)n⩾0 be a Markov chain with state space S.
(i) What does it mean to say that (Xn)n⩾0 has the strong Markov property? Your answer should include the definition of the term stopping time.
(ii) Show that
P(Xn=i at least k times ∣X0=i)=[P(Xn=i at least once ∣X0=i)]k
for a state i∈S. You may use without proof the fact that (Xn)n⩾0 has the strong Markov property.