Paper 1, Section I, H

Optimization
Part IB, 2019

Suppose that ff is an infinitely differentiable function on R\mathbb{R}. Assume that there exist constants 0<C1,C2<0<C_{1}, C_{2}<\infty so that f(x)C1\left|f^{\prime \prime}(x)\right| \geqslant C_{1} and f(x)C2\left|f^{\prime \prime \prime}(x)\right| \leqslant C_{2} for all xRx \in \mathbb{R}. Fix x0Rx_{0} \in \mathbb{R} and for each nNn \in \mathbb{N} set

xn=xn1f(xn1)f(xn1).x_{n}=x_{n-1}-\frac{f^{\prime}\left(x_{n-1}\right)}{f^{\prime \prime}\left(x_{n-1}\right)} .

Let xx^{*} be the unique value of xx where ff attains its minimum. Prove that

xxn+1C22C1xxn2 for all nN.\left|x^{*}-x_{n+1}\right| \leqslant \frac{C_{2}}{2 C_{1}}\left|x^{*}-x_{n}\right|^{2} \quad \text { for all } n \in \mathbb{N} .

[Hint: Express f(x)f^{\prime}\left(x^{*}\right) in terms of the Taylor series for ff^{\prime} at xnx_{n} using the Lagrange form of the remainder: f(x)=f(xn)+f(xn)(xxn)+12f(yn)(xxn)2f^{\prime}\left(x^{*}\right)=f^{\prime}\left(x_{n}\right)+f^{\prime \prime}\left(x_{n}\right)\left(x^{*}-x_{n}\right)+\frac{1}{2} f^{\prime \prime \prime}\left(y_{n}\right)\left(x^{*}-x_{n}\right)^{2} where yny_{n} is between xnx_{n} and x.]\left.x^{*} .\right]