(i) Consider the continuous-time Markov chain (Xt)t⩾0 with state-space {1,2,3,4} and Q-matrix
Q=⎝⎜⎜⎜⎛−21010−32502−22200−8⎠⎟⎟⎟⎞
Set
Yt={Xt2 if Xt∈{1,2,3} if Xt=4
and
Zt={Xt1 if Xt∈{1,2,3} if Xt=4
Determine which, if any, of the processes (Yt)t⩾0 and (Zt)t⩾0 are Markov chains.
(ii) Find an invariant distribution for the chain (Xt)t⩾0 given in Part (i). Suppose X0=1. Find, for all t⩾0, the probability that Xt=1.