Paper 3, Section II, J

Optimization and Control
Part II, 2014

A particle follows a discrete-time trajectory on R\mathbb{R} given by

xt+1=Axt+ξtut+ϵtx_{t+1}=A x_{t}+\xi_{t} u_{t}+\epsilon_{t}

for t=1,2,,Tt=1,2, \ldots, T, where T2T \geqslant 2 is a fixed integer, AA is a real constant, xtx_{t} is the position of the particle and utu_{t} is the control action at time tt, and (ξt,ϵt)t=1T\left(\xi_{t}, \epsilon_{t}\right)_{t=1}^{T} is a sequence of independent random vectors with Eξt=Eϵt=0,var(ξt)=Vξ>0,var(ϵt)=Vϵ>0\mathbb{E} \xi_{t}=\mathbb{E} \epsilon_{t}=0, \operatorname{var}\left(\xi_{t}\right)=V_{\xi}>0, \operatorname{var}\left(\epsilon_{t}\right)=V_{\epsilon}>0 and cov(ξt,ϵt)=0\operatorname{cov}\left(\xi_{t}, \epsilon_{t}\right)=0.

Find the closed-loop control, i.e. the control action utu_{t} defined as a function of (x1,,xt;u1,,ut1)\left(x_{1}, \ldots, x_{t} ; u_{1}, \ldots, u_{t-1}\right), that minimizes

t=1Txt2+ct=1T1ut\sum_{t=1}^{T} x_{t}^{2}+c \sum_{t=1}^{T-1} u_{t}

where c>0c>0 is given. [Note that this function is quadratic in xx, but linear in uu.]

Does the closed-loop control depend on VϵV_{\epsilon} or on VξV_{\xi} ? Deduce the form of the optimal open-loop control.