Paper 4, Section I, H

Markov Chains
Part IB, 2019

For a Markov chain XX on a state space SS with u,vSu, v \in S, we let puv(n)p_{u v}(n) for n{0,1,}n \in\{0,1, \ldots\} be the probability that Xn=vX_{n}=v when X0=uX_{0}=u.

(a) Let XX be a Markov chain. Prove that if XX is recurrent at a state vv, then n=0pvv(n)=\sum_{n=0}^{\infty} p_{v v}(n)=\infty. [You may use without proof that the number of returns of a Markov chain to a state vv when starting from vv has the geometric distribution.]

(b) Let XX and YY be independent simple symmetric random walks on Z2\mathbb{Z}^{2} starting from the origin 0 . Let Z=n=01{Xn=Yn}Z=\sum_{n=0}^{\infty} \mathbf{1}_{\left\{X_{n}=Y_{n}\right\}}. Prove that E[Z]=n=0p00(2n)\mathbb{E}[Z]=\sum_{n=0}^{\infty} p_{00}(2 n) and deduce that E[Z]=\mathbb{E}[Z]=\infty. [You may use without proof that pxy(n)=pyx(n)p_{x y}(n)=p_{y x}(n) for all x,yZ2x, y \in \mathbb{Z}^{2} and nNn \in \mathbb{N}, and that XX is recurrent at 0.]