Proof that unless projection matrix P = I, P is singular

We have:  P = A(A^{T}A)^{-1}A^{T}, first we need to prove that rank(P) \leq rank(A): Set M = (A^{T}A)^{-1}A^{T} \Rightarrow P = AM, hence each column vector of P is a linear combination of column vectors of A, so C(P) \subseteq C(A) \Rightarrow rank(P) \leq rank(A).

Here P is an m by m matrix while A is an m by n matrix and column vectors of A must be linearly indepedent (so m \geq n), hence rank(A) \leq n.

  • If m > n then it’s trivial to show that P is singular.
  • If  m = n then  P = A(A^{T}A)^{-1}A^{T} = AA^{-1}(A^{T})^{-1}A^{T} = I.

So the statement follows.


Leave a Reply

Your email address will not be published. Required fields are marked *