2.II.20A

Numerical Analysis
Part IB, 2004

Given an m×nm \times n matrix AA and bRm\mathbf{b} \in \mathbb{R}^{m}, prove that the vector xRn\mathbf{x} \in \mathbb{R}^{n} is the solution of the least-squares problem for AxbA \mathbf{x} \approx \mathbf{b} if and only if AT(Axb)=0A^{T}(A \mathbf{x}-\mathbf{b})=\mathbf{0}. Let

A=[12311341],b=[3012]A=\left[\begin{array}{cc} 1 & 2 \\ -3 & 1 \\ 1 & 3 \\ 4 & 1 \end{array}\right], \quad \mathbf{b}=\left[\begin{array}{c} 3 \\ 0 \\ -1 \\ 2 \end{array}\right]

Determine the solution of the least-squares problem for AxbA \mathbf{x} \approx \mathbf{b}.