Paper 3, Section I, H

Markov Chains
Part IB, 2013

Prove that if a distribution π\pi is in detailed balance with a transition matrix PP then it is an invariant distribution for PP.

Consider the following model with 2 urns. At each time, t=0,1,t=0,1, \ldots one of the following happens:

  • with probability β\beta a ball is chosen at random and moved to the other urn (but nothing happens if both urns are empty);

  • with probability γ\gamma a ball is chosen at random and removed (but nothing happens if both urns are empty);

  • with probability α\alpha a new ball is added to a randomly chosen urn,

where α+β+γ=1\alpha+\beta+\gamma=1 and α<γ\alpha<\gamma. State (i,j)(i, j) denotes that urns 1,2 contain ii and jj balls respectively. Prove that there is an invariant measure

λi,j=(i+j)!i!j!(α/2γ)i+j\lambda_{i, j}=\frac{(i+j) !}{i ! j !}(\alpha / 2 \gamma)^{i+j}

Find the proportion of time for which there are nn balls in the system.