Paper 2, Section II, H

Markov Chains
Part IB, 2014

Let (Xn:n0)\left(X_{n}: n \geqslant 0\right) be a homogeneous Markov chain with state space S\mathrm{S} and transition matrix P=(pi,j:i,jS)P=\left(p_{i, j}: i, j \in S\right). For ASA \subseteq S, let

HA=inf{n0:XnA} and hiA=P(HA<X0=i),iSH^{A}=\inf \left\{n \geqslant 0: X_{n} \in A\right\} \text { and } h_{i}^{A}=\mathbb{P}\left(H^{A}<\infty \mid X_{0}=i\right), i \in S

Prove that hA=(hiA:iS)h^{A}=\left(h_{i}^{A}: i \in S\right) is the minimal non-negative solution to the equations

hiA={1 for iAjSpi,jhjA otherwise. h_{i}^{A}= \begin{cases}1 & \text { for } i \in A \\ \sum_{j \in S} p_{i, j} h_{j}^{A} & \text { otherwise. }\end{cases}

Three people A,BA, B and CC play a series of two-player games. In the first game, two people play and the third person sits out. Any subsequent game is played between the winner of the previous game and the person sitting out the previous game. The overall winner of the series is the first person to win two consecutive games. The players are evenly matched so that in any game each of the two players has probability 12\frac{1}{2} of winning the game, independently of all other games. For n=1,2,n=1,2, \ldots, let XnX_{n} be the ordered pair consisting of the winners of games nn and n+1n+1. Thus the state space is {AA,AB,AC,BA,BB,BC,CA,CB,CC}\{A A, A B, A C, B A, B B, B C, C A, C B, C C\}, and, for example, X1=ACX_{1}=A C if AA wins the first game and CC wins the second.

The first game is between AA and BB. Treating AA,BBA A, B B and CCC C as absorbing states, or otherwise, find the probability of winning the series for each of the three players.