(i) Let $X$ be a Markov chain with finitely many states. Define a stopping time and state the strong Markov property.

(ii) Let $X$ be a Markov chain with state-space $\{-1,0,1\}$ and Q-matrix

$$Q=\left(\begin{array}{ccc} -(q+\lambda) & \lambda & q \\ 0 & 0 & 0 \\ q & \lambda & -(q+\lambda) \end{array}\right), \text { where } q, \lambda>0$$

Consider the integral $\int_{0}^{t} X(s) \mathrm{d} s$, the signed difference between the times spent by the chain at states $+1$ and $-1$ by time $t$, and let

$$\begin{aligned} Y &=\sup \left[\int_{0}^{t} X(s) \mathrm{d} s: t>0\right] \\ \psi_{\pm}(c) &=\mathbb{P}\left(Y>c \mid X_{0}=\pm 1\right), \quad c>0 \end{aligned}$$

Derive the equation

$$\psi_{-}(c)=\int_{0}^{\infty} q e^{-(\lambda+q) u_{+}} \psi_{+}(c+u) \mathrm{d} u$$

(iii) Obtain another equation relating $\psi_{+}$to $\psi_{-}$.

(iv) Assuming that $\psi_{+}(c)=e^{-c A}, c>0$, where $A$ is a non-negative constant, calculate $A$.

(v) Give an intuitive explanation why the function $\psi_{+}$must have the exponential form $\psi_{+}(c)=e^{-c A}$ for some $A$.