Paper 4, Section I, H

Markov Chains
Part IB, 2015

Let X0,X1,X2,X_{0}, X_{1}, X_{2}, \ldots be independent identically distributed random variables with P(Xi=1)=1P(Xi=0)=p,0<p<1\mathbb{P}\left(X_{i}=1\right)=1-\mathbb{P}\left(X_{i}=0\right)=p, 0<p<1. Let Zn=Xn1+cXn,n=1,2,Z_{n}=X_{n-1}+c X_{n}, n=1,2, \ldots, where cc is a constant. For each of the following cases, determine whether or not (Zn:n1)\left(Z_{n}: n \geqslant 1\right) is a Markov chain: (a) c=0c=0; (b) c=1c=1; (c) c=2c=2.

In each case, if (Zn:n1)\left(Z_{n}: n \geqslant 1\right) is a Markov chain, explain why, and give its state space and transition matrix; if it is not a Markov chain, give an example to demonstrate that it is not.