Paper 2, Section II, E

Markov Chains
Part IB, 2010

Let (Xn)n0\left(X_{n}\right)_{n \geqslant 0} be a simple, symmetric random walk on the integers {,1,0,1,}\{\ldots,-1,0,1, \ldots\}, with X0=0X_{0}=0 and P(Xn+1=i±1Xn=i)=1/2\mathbb{P}\left(X_{n+1}=i \pm 1 \mid X_{n}=i\right)=1 / 2. For each integer a1a \geqslant 1, let Ta=inf{n0:Xn=a}T_{a}=\inf \left\{n \geqslant 0: X_{n}=a\right\}. Show that TaT_{a} is a stopping time.

Define a random variable YnY_{n} by the rule

Yn={Xn if n<Ta2aXn if nTaY_{n}= \begin{cases}X_{n} & \text { if } n<T_{a} \\ 2 a-X_{n} & \text { if } n \geqslant T_{a}\end{cases}

Show that (Yn)n0\left(Y_{n}\right)_{n} \geqslant 0 is also a simple, symmetric random walk.

Let Mn=max0inXnM_{n}=\max _{0 \leqslant i \leqslant n} X_{n}. Explain why {Mna}={Tan}\left\{M_{n} \geqslant a\right\}=\left\{T_{a} \leqslant n\right\} for a0a \geqslant 0. By using the process (Yn)n0\left(Y_{n}\right)_{n \geqslant 0} constructed above, show that, for a0a \geqslant 0,

P(Mna,Xna1)=P(Xna+1),\mathbb{P}\left(M_{n} \geqslant a, X_{n} \leqslant a-1\right)=\mathbb{P}\left(X_{n} \geqslant a+1\right),

and thus

P(Mna)=P(Xna)+P(Xna+1)\mathbb{P}\left(M_{n} \geqslant a\right)=\mathbb{P}\left(X_{n} \geqslant a\right)+\mathbb{P}\left(X_{n} \geqslant a+1\right)

Hence compute

P(Mn=a)\mathbb{P}\left(M_{n}=a\right)

when aa and nn are positive integers with nan \geqslant a. [Hint: if nn is even, then XnX_{n} must be even, and if nn is odd, then XnX_{n} must be odd.]