Paper 2, Section II, H

Markov Chains
Part IB, 2019

Fix n1n \geqslant 1 and let GG be the graph consisting of a copy of {0,,n}\{0, \ldots, n\} joining vertices AA and BB, a copy of {0,,n}\{0, \ldots, n\} joining vertices BB and CC, and a copy of {0,,n}\{0, \ldots, n\} joining vertices BB and DD. Let EE be the vertex adjacent to BB on the segment from BB to CC. Shown below is an illustration of GG in the case n=5n=5. The vertices are solid squares and edges are indicated by straight lines.

Let (Xk)\left(X_{k}\right) be a simple random walk on GG. In other words, in each time step, XX moves to one of its neighbours with equal probability. Assume that X0=AX_{0}=A.

(a) Compute the expected amount of time for XX to hit BB.

(b) Compute the expected amount of time for XX to hit EE. [Hint: first show that the expected amount of time xx for XX to go from BB to EE satisfies x=13+23(L+x)x=\frac{1}{3}+\frac{2}{3}(L+x) where LL is the expected return time of XX to BB when starting from BB.]

(c) Compute the expected amount of time for XX to hit CC. [Hint: for each ii, let viv_{i} be the vertex which is ii places to the right of BB on the segment from BB to CC. Derive an equation for the expected amount of time xix_{i} for XX to go from viv_{i} to vi+1v_{i+1}.]

Justify all of your answers.