Paper 2, Section II, H

Markov Chains
Part IB, 2017

Let Y1,Y2,Y_{1}, Y_{2}, \ldots be i.i.d. random variables with values in {1,2,}\{1,2, \ldots\} and E[Y1]=μ<\mathbb{E}\left[Y_{1}\right]=\mu<\infty. Moreover, suppose that the greatest common divisor of {n:P(Y1=n)>0}\left\{n: \mathbb{P}\left(Y_{1}=n\right)>0\right\} is 1 . Consider the following process

Xn=inf{mn:Y1++Yk=m, for some k0}n.X_{n}=\inf \left\{m \geqslant n: Y_{1}+\ldots+Y_{k}=m, \text { for some } k \geqslant 0\right\}-n .

(a) Show that XX is a Markov chain and find its transition probabilities.

(b) Let T0=inf{n1:Xn=0}T_{0}=\inf \left\{n \geqslant 1: X_{n}=0\right\}. Find E0[T0]\mathbb{E}_{0}\left[T_{0}\right].

(c) Find the limit as nn \rightarrow \infty of P(Xn=0)\mathbb{P}\left(X_{n}=0\right). State carefully any theorems from the course that you are using.