Paper 1, Section II, H

Markov Chains
Part IB, 2018

A coin-tossing game is played by two players, A1A_{1} and A2A_{2}. Each player has a coin and the probability that the coin tossed by player AiA_{i} comes up heads is pip_{i}, where 0<pi<1,i=1,20<p_{i}<1, i=1,2. The players toss their coins according to the following scheme: A1A_{1} tosses first and then after each head, A2A_{2} pays A1A_{1} one pound and A1A_{1} has the next toss, while after each tail, A1A_{1} pays A2A_{2} one pound and A2A_{2} has the next toss.

Define a Markov chain to describe the state of the game. Find the probability that the game ever returns to a state where neither player has lost money.