Math 443 — Projections and Orthonormal Frames

Projections
Definition.  A projection matrix is a square matrix $P$ that satisfies $P^2=P$.
Theorem. If $P$ is a projection and $\vx\in\cR(P)$ then $P\vx = \vx$      (proof).
Since $\vx\in\cR(P)$ there is a vector $\vy$ such that $\vx=P\vy$. It then follows that
$P\vx = P(P\vy) = P^2\vy = P\vy = \vx$.
Theorem. Let $P$ be a projection. Then any vector $\vx\in\R^n$ can be written as the sum $\vx=\vy+\vz$ of a vector $\vy\in \cR(P)$ and a vector $\vz\in\cN(P)$.      (proof).
If $\vx\in \R^n$ then we can write $\vx = \vy + \vz$ where by definition $\vy = P\vx$ and $\vz = \vx-P\vx$. The vector $\vy = P\vx$ belongs to the range of $P$. The vector $\vz = \vx - P\vx$ belong to the nullspace of $P$ because
$P\vz = P(\vx-P\vx) = P\vx - P^2\vx = P\vx -P\vx = 0$.
To show that the decomposition $\vx=\vy+\vz$ is unique, we assume $\vx = \vy+\vz$ with $\vy\in\cR(P)$, $\vz\in\cN(P)$. Since $\vy\in\cR(P)$ we have $P\vy=\vy$; we also assume $\vz\in\cN(P)$, so $P\vz=0$. This implies that
$P\vx = P\vy + P\vz = \vy + 0 = \vy$
In other words, if $\vx=\vy+\vz$ with $\vy\in\cR(P)$ and $\vz\in\cN(P)$, then $\vy$ must be given by $\vy=P\vx$, and thus $\vz$ can only be $\vz = \vx-\vy = \vx-P\vx$.
Definition. A projection is orthogonal if $\cR(P)$ and $\cN(P)$ are orthogonal, i.e. if $\vx\perp\vy$ for all $\vx\in\cR(P)$ and $\vy\in\cN(P)$.          
Theorem. If a projection $P$ is symmetric then it is an orthogonal projection.       (proof)
Assuming $P$ is symmetric, i.e. if $P^T=P$, we will show that $\vy^T\vz=0$ for any $\vy\in\cR(P)$ and $\vz\in\cN(P)$. If $\vy\in\cR(P)$ then by definition $\vy=P\vw$ for some vector $\vw$. It then follows that
$\vy^T\vz = (P\vw)^T\vz = \vw^T P^T \vz = \vw^T P \vz = \vw^T{\mathbf0} = 0$.
Here we have used that $P^T=P$, and $P\vz=\mathbf0$.
Orthonormal frames
Definition.  The vectors $(\vv_1, \vv_2, \cdots ,\vv_k)$ form an orthonormal frame if $\vv_i\perp\vv_j$ for all $i\neq j$, and if $\|\vv_i\|=1$ for all $i$.
Let $\vv_1, \dots, \vv_k$ be a given set of vectors, and consider the matrix
$V = \begin{bmatrix} | & & | \\ \vv_1 & \cdots & \vv_k \\ | & & | \end{bmatrix}$.
Theorem.  The vectors $\vv_1, \dots, \vv_k$ form an orthonormal frame if and only if $V^TV = I_{k\times k}$      (proof)
The matrix $V^TV$ is given by
$V^TV = \begin{bmatrix} \vv_1^T\vv_1 & \cdots & \vv_1^T\vv_k \\ \vdots & & \vdots \\ \vv_k^T\vv_1 & \cdots & \vv_k^T\vv_k \end{bmatrix}$.
Theorem. If the vectors $\vv_1, \dots,\vv_k$ form an orthonormal frame, then the matrix $VV^T$ is the orthogonal projection on the subspace spanned by $\vv_1, \dots, \vv_k$.     (proof)
We have shown that $V^TV=I$. Therefore we have
$(VV^T)^2 = (VV^T)(VV^T) =V(V^TV)V^T = VV^T$
so that $VV^T$ is a projection.

Since $(VV^T)^T = (V^T)^TV^T = VV^T$ the projection matrix $VV^T$ is symmetric, and the projection $VV^T$ is orthogonal.

If $\vx$ is any vector then $VV^T\vx$ is a linear combination of the columns of $V$. Therefore range of $VV^T$ is contained in the span of $\vv_1, \dots, \vv_k$. On the other hand $VV^T\vv_i = \vv_i$, so the range of $VV^T$ contains the vectors $\vv_1, \dots, \vv_k$. Therefore $\cR(VV^T) = \mathrm{span}\{\vv_1, \dots, \vv_k\}$