A3.1 B3.1

Markov Chains
Part II, 2003

(i) Consider the continuous-time Markov chain (Xt)t0\left(X_{t}\right)_{t \geqslant 0} with state-space {1,2,3,4}\{1,2,3,4\} and QQ-matrix

Q=(2002132002201528)Q=\left(\begin{array}{rrrr} -2 & 0 & 0 & 2 \\ 1 & -3 & 2 & 0 \\ 0 & 2 & -2 & 0 \\ 1 & 5 & 2 & -8 \end{array}\right)

Set

Yt={Xt if Xt{1,2,3}2 if Xt=4Y_{t}= \begin{cases}X_{t} & \text { if } X_{t} \in\{1,2,3\} \\ 2 & \text { if } X_{t}=4\end{cases}

and

Zt={Xt if Xt{1,2,3}1 if Xt=4Z_{t}= \begin{cases}X_{t} & \text { if } X_{t} \in\{1,2,3\} \\ 1 & \text { if } X_{t}=4\end{cases}

Determine which, if any, of the processes (Yt)t0\left(Y_{t}\right)_{t \geqslant 0} and (Zt)t0\left(Z_{t}\right)_{t \geqslant 0} are Markov chains.

(ii) Find an invariant distribution for the chain (Xt)t0\left(X_{t}\right)_{t \geqslant 0} given in Part (i). Suppose X0=1X_{0}=1. Find, for all t0t \geqslant 0, the probability that Xt=1X_{t}=1.