Paper 3, Section I, H

Markov Chains
Part IB, 2016

Let (Xn)n0\left(X_{n}\right)_{n \geqslant 0} be a Markov chain such that X0=iX_{0}=i. Prove that

n=0Pi(Xn=i)=1Pi(Xni for all n1)\sum_{n=0}^{\infty} \mathbb{P}_{i}\left(X_{n}=i\right)=\frac{1}{\mathbb{P}_{i}\left(X_{n} \neq i \text { for all } n \geqslant 1\right)}

where 1/0=+1 / 0=+\infty. [You may use the strong Markov property without proof.]