4.II.29I

Optimization and Control
Part II, 2007

Consider the scalar controllable linear system, whose state XnX_{n} evolves by

Xn+1=Xn+Un+εn+1X_{n+1}=X_{n}+U_{n}+\varepsilon_{n+1}

with observations YnY_{n} given by

Yn+1=Xn+ηn+1Y_{n+1}=X_{n}+\eta_{n+1}

Here, UnU_{n} is the control variable, which is to be determined on the basis of the observations up to time nn, and εn,ηn\varepsilon_{n}, \eta_{n} are independent N(0,1)N(0,1) random variables. You wish to minimize the long-run average expected cost, where the instantaneous cost at time nn is Xn2+Un2X_{n}^{2}+U_{n}^{2}. You may assume that the optimal control in equilibrium has the form Un=KX^nU_{n}=-K \hat{X}_{n}, where X^n\hat{X}_{n} is given by a recursion of the form

X^n+1=X^n+Un+H(Yn+1X^n)\hat{X}_{n+1}=\hat{X}_{n}+U_{n}+H\left(Y_{n+1}-\hat{X}_{n}\right)

and where HH is chosen so that Δn=XnX^n\Delta_{n}=X_{n}-\hat{X}_{n} is independent of the observations up to time nn. Show that K=H=(51)/2=2/(5+1)K=H=(\sqrt{5}-1) / 2=2 /(\sqrt{5}+1), and determine the minimal long-run average expected cost. You are not expected to simplify the arithmetic form of your answer but should show clearly how you have obtained it.