A2.1

Markov Chains
Part II, 2002

(i) In each of the following cases, the state-space II and non-zero transition rates qijq_{i j} (ij)(i \neq j) of a continuous-time Markov chain are given. Determine in which cases the chain is explosive.

(ii) Children arrive at a see-saw according to a Poisson process of rate 1 . Initially there are no children. The first child to arrive waits at the see-saw. When the second child arrives, they play on the see-saw. When the third child arrives, they all decide to go and play on the merry-go-round. The cycle then repeats. Show that the number of children at the see-saw evolves as a Markov Chain and determine its generator matrix. Find the probability that there are no children at the see-saw at time tt.

Hence obtain the identity

n=0ett3n(3n)!=13+23e32tcos32t\sum_{n=0}^{\infty} e^{-t} \frac{t^{3 n}}{(3 n) !}=\frac{1}{3}+\frac{2}{3} e^{-\frac{3}{2} t} \cos \frac{\sqrt{3}}{2} t

 (a) I={1,2,3,},qi,i+1=i2,iI (b) I=Z,qi,i1=qi,i+1=2i,iI\begin{aligned} & \text { (a) } I=\{1,2,3, \ldots\}, \quad q_{i, i+1}=i^{2}, \quad i \in I \text {, } \\ & \text { (b) } \quad I=\mathbb{Z}, \quad q_{i, i-1}=q_{i, i+1}=2^{i}, \quad i \in I \text {. } \end{aligned}