Paper 2, Section II, J

Applied Probability
Part II, 2018

Let X=(Xt:t0)X=\left(X_{t}: t \geqslant 0\right) be a continuous-time Markov chain on the finite state space SS. Define the terms generator (or Q-matrix) and invariant distribution, and derive an equation that links the generator GG and any invariant distribution π\pi. Comment on the possible non-uniqueness of invariant distributions.

Suppose XX is irreducible, and let N=(Nt:t0)N=\left(N_{t}: t \geqslant 0\right) be a Poisson process with intensity λ\lambda, that is independent of XX. Let YnY_{n} be the value of XX immediately after the nnth arrival-time of NN (and Y0=X0)\left.Y_{0}=X_{0}\right). Show that (Yn:n=0,1,)\left(Y_{n}: n=0,1, \ldots\right) is a discrete-time Markov chain, state its transition matrix and prove that it has the same invariant distribution as XX.