Paper 4, Section I, H

Markov Chains
Part IB, 2014

Let (Xn:n0)\left(X_{n}: n \geqslant 0\right) be a homogeneous Markov chain with state space SS and transition matrixP=(pi,j:i,jS)\operatorname{matrix} P=\left(p_{i, j}: i, j \in S\right).

(a) Let Wn=X2n,n=0,1,2,W_{n}=X_{2 n}, n=0,1,2, \ldots Show that (Wn:n0)\left(W_{n}: n \geqslant 0\right) is a Markov chain and give its transition matrix. If λi=P(X0=i),iS\lambda_{i}=\mathbb{P}\left(X_{0}=i\right), i \in S, find P(W1=0)\mathbb{P}\left(W_{1}=0\right) in terms of the λi\lambda_{i} and the pi,jp_{i, j}.

[Results from the course may be quoted without proof, provided they are clearly stated.]

(b) Suppose that S={1,0,1},p0,1=p1,1=0S=\{-1,0,1\}, p_{0,1}=p_{-1,-1}=0 and p1,0p1,0p_{-1,0} \neq p_{1,0}. Let Yn=XnY_{n}=\left|X_{n}\right|, n=0,1,2,n=0,1,2, \ldots In terms of the pi,jp_{i, j}, find

(i) P(Yn+1=0Yn=1,Yn1=0)\mathbb{P}\left(Y_{n+1}=0 \mid Y_{n}=1, Y_{n-1}=0\right) and

(ii) P(Yn+1=0Yn=1,Yn1=1,Yn2=0)\mathbb{P}\left(Y_{n+1}=0 \mid Y_{n}=1, Y_{n-1}=1, Y_{n-2}=0\right).

What can you conclude about whether or not (Yn:n0)\left(Y_{n}: n \geqslant 0\right) is a Markov chain?