A particle follows a discrete-time trajectory on R given by
xt+1=Axt+ξtut+ϵt
for t=1,2,…,T, where T⩾2 is a fixed integer, A is a real constant, xt is the position of the particle and ut is the control action at time t, and (ξt,ϵt)t=1T is a sequence of independent random vectors with Eξt=Eϵt=0,var(ξt)=Vξ>0,var(ϵt)=Vϵ>0 and cov(ξt,ϵt)=0.
Find the closed-loop control, i.e. the control action ut defined as a function of (x1,…,xt;u1,…,ut−1), that minimizes
t=1∑Txt2+ct=1∑T−1ut
where c>0 is given. [Note that this function is quadratic in x, but linear in u.]
Does the closed-loop control depend on Vϵ or on Vξ ? Deduce the form of the optimal open-loop control.