Learning From Data – A Short Course: Exercise 3.3

Page 87, Exercise 3.3:

Consider the hat matrix H = X(X^{T}X)^{-1}X^{T}, where  X is an N by d + 1 matrix, and X^{T}X is invertible.

(a) Show that H is symmetric.

(b) Show that H^{K} = H for any positive integer K.

(c) If I is the identity matrix of size N, show that (I - H)^{K} = I - H for any positive integer K.

(d) Show that trace(H) = d + 1, where the trace is the sum of diagonal elements.

(a) and (d) Forgot the linear algebra, will come back to it later (or never, lol).

(b) Base cases:

  • K = 1: H^{1} = H
  • K = 2: H^{2} = H (stated in book, verify later)

Induction step for K > 2:

    \[ H^{K} = H \times H^{K-1} \]

Due to  H^{K-1} = H:

    \[ H \times H^{K-1} = H \times H = H^{2} \]

We also have H^{2} = H, so:

    \[ H^{K} = H \]

(c) We observe that:

 (I - H)^{2} = (I - H)(I - H) = I^{2} - IH - HI + H^{2} = I - H - H + H = I - H

The rest of the proof is similar to (b).


Facebooktwittergoogle_plusredditpinterestlinkedinmail

Leave a Reply

Your email address will not be published. Required fields are marked *