3.I.5I

Statistical Modelling
Part II, 2005

Consider the model Y=Xβ+ϵY=X \beta+\epsilon, where YY is an nn-dimensional observation vector, XX is an n×pn \times p matrix of rank p,ϵp, \epsilon is an nn-dimensional vector with components ϵ1,,ϵn\epsilon_{1}, \ldots, \epsilon_{n}, and ϵ1,,ϵn\epsilon_{1}, \ldots, \epsilon_{n} are independently and normally distributed, each with mean 0 and variance σ2\sigma^{2}

(a) Let β^\hat{\beta} be the least-squares estimator of β\beta. Show that

(XTX)β^=XTY\left(X^{T} X\right) \hat{\beta}=X^{T} Y

and find the distribution of β^\hat{\beta}.

(b) Define Y^=Xβ^\hat{Y}=X \hat{\beta}. Show that Y^\hat{Y} has distribution N(Xβ,σ2H)N\left(X \beta, \sigma^{2} H\right), where HH is a matrix that you should define.

[You may quote without proof any results you require about the multivariate normal distribution.]