A1.1 B1.1

Markov Chains
Part II, 2003

(i) Let (Xn,Yn)n0\left(X_{n}, Y_{n}\right)_{n \geqslant 0} be a simple symmetric random walk in Z2\mathbb{Z}^{2}, starting from (0,0)(0,0), and set T=inf{n0:max{Xn,Yn}=2}T=\inf \left\{n \geqslant 0: \max \left\{\left|X_{n}\right|,\left|Y_{n}\right|\right\}=2\right\}. Determine the quantities E(T)\mathbb{E}(T) and P(XT=2\mathbb{P}\left(X_{T}=2\right. and YT=0)\left.Y_{T}=0\right).

(ii) Let (Xn)n0\left(X_{n}\right)_{n \geqslant 0} be a discrete-time Markov chain with state-space II and transition matrix PP. What does it mean to say that a state iIi \in I is recurrent? Prove that ii is recurrent if and only if n=0pii(n)=\sum_{n=0}^{\infty} p_{i i}^{(n)}=\infty, where pii(n)p_{i i}^{(n)} denotes the (i,i)(i, i) entry in PnP^{n}.

Show that the simple symmetric random walk in Z2\mathbb{Z}^{2} is recurrent.