(1) Use Gaussian Elimination to address (i). All the rest can be easily inferred.
(2) Proof by contradiction:
Assume and the solution of the simultaneous linear equation does not exist.
If then . However, and , so is guaranteed to have linearly dependent columns. Therefore, we can always represent as a linear combination of other column vectors in . As the solution of the simultaneous linear equation is guaranteed to exist, this contradicts the assumption.
So the statement follows.
(3) There are various solutions to this problem. But I am not sure if it is appropriate to apply the normal equation directly.
Let . We set the partial derivative of to be zero to find critical points:
Given , if then (proof). So is invertible. Therefore, we can write:
Now we need to prove that critical point is the local minimum (and also global minimum as we only have one critical point):
is a positive-semidefinite matrix (as ), so the found critical point with is the local minimum and also the global minimum.
(4) Let’s formulate the problem:
Let’s find :
So: . Given , if then . Therefore, is invertible and we can write .
Prove that the found critical point is also the constrained local minimum:
So, the critical point with is the constrained local minimum and also the constrained global minimum.
(5) How can we compute Pseudoinverse for any Matrix? is the pseudoinverse of .
(6) for (3) and for (4). I really don’t know what this question is for.