## Subsection2.2.3Orthonormal vectors and matrices

A lot of the formulae in the last unit become simpler if the length of the vector equals one: If $\| u \|_2 = 1$ then

• the component of $v$ in the direction of $u$ equals

\begin{equation*} \frac{u^H v}{u^H u} u = u^H v u. \end{equation*}
• the matrix that projects a vector onto the vector $u$ is given by

\begin{equation*} \frac{u u^H}{u^H u} = u u^H. \end{equation*}
• the component of $v$ orthogonal to $u$ equals

\begin{equation*} v - \frac{u^H v}{u^H u} u = v - u^H v u. \end{equation*}
• the matrix that projects a vector onto the space orthogonal to $u$ is given by

\begin{equation*} I - \frac{u u^H}{u^H u} = I - u u^H. \end{equation*}
###### Homework2.2.3.1.

Let $u \neq 0 \in \Cm \text{.}$

ALWAYS/SOMETIMES/NEVER $u / \| u \|_2$ has unit length.

ALWAYS.

Now prove it.

Solution
\begin{equation*} \begin{array}{l} \left\| \frac{u}{\| u \|_2} \right\|_2 \\ ~~~ = ~~~~ \lt \mbox{ homogenuity of norms } \gt \\ \frac{ \| u \|_2} {\| u \|_2} \\ ~~~ = ~~~~ \lt \mbox{ algebra } \gt \\ 1 \end{array} \end{equation*}

This last exercise shows that any nonzero vector can be scaled (normalized) to have unit length.

###### Definition2.2.3.1. Orthonormal vectors.

Let $u_0, u_1, \ldots, u_{n-1} \in \C^m \text{.}$ These vectors are said to be mutually orthonormal if for all $0 \leq i,j \lt n$

\begin{equation*} u_i^H u_j = \left\{ \begin{array}{c l} 1 \amp {\rm if~} i = j \\ 0 \amp {\rm otherwise} \end{array} \right. . \end{equation*}

The definition implies that $\| u_i \|_2 = \sqrt{ u_i^H u_i } = 1$ and hence each of the vectors is of unit length in addition to being orthogonal to each other.

The standard basis vectors (Definition 1.3.1.3)

\begin{equation*} \{ e_j \}_{i=0}^{m-1} \subset \Cm, \end{equation*}

where

\begin{equation*} e_j = \left( \begin{array}{c} 0 \\ \vdots \\ 0 \\ 1 \\ 0 \\ \vdots \\ 0 \end{array} \right) \longleftarrow \mbox{ entry indexed with } j \end{equation*}

are mutually orthonormal since, clearly,

\begin{equation*} e_i^H e_j = \left\{ \begin{array}{c l} 1 \amp {\rm if~} i = j \\ 0 \amp {\rm otherwise.} \end{array} \right. \end{equation*}

Naturally, any subset of the standard basis vectors is a set of mutually orthonormal vectors.

###### Remark2.2.3.2.

For $n$ vectors of size $m$ to be mutually orthonormal, $n$ must be less than or equal to $m \text{.}$ This is because $n$ mutually orthonormal vectors are linearly independent and there can be at most $m$ linearly independent vectors of size $m \text{.}$

A very concise way of indicating that a set of vectors are mutually orthonormal is to view them as the columns of a matrix, which then has a very special property:

###### Definition2.2.3.3. Orthonormal matrix.

Let $Q \in \C^{m \times n}$ (with $n \leq m$). Then $Q$ is said to be an orthonormal matrix iff $Q^H Q = I \text{.}$

The subsequent exercise makes the connection between mutually orthonormal vectors and an orthonormal matrix.

###### Homework2.2.3.2.

Let $Q \in \C^{m \times n}$ (with $n \leq m$). Partition $Q = \left( \begin{array}{c | c | c | c} q_0 \amp q_1 \amp \cdots \amp q_{n-1} \end{array} \right) \text{.}$

TRUE/FALSE: $Q$ is an orthonormal matrix if and only if $q_0, q_1, \ldots , q_{n-1}$ are mutually orthonormal.

TRUE

Now prove it!

Solution

Let $Q \in \C^{m \times n}$ (with $n \leq m$). Partition $Q = \left( \begin{array}{c | c | c | c} q_0 \amp q_1 \amp \cdots \amp q_{n-1} \end{array} \right)\text{.}$ Then

\begin{equation*} \begin{array}{rcl} Q^H Q \amp=\amp \left( \begin{array}{c | c | c | c} q_0 \amp q_1 \amp \cdots \amp q_{n-1} \end{array} \right)^H \left( \begin{array}{c | c | c | c} q_0 \amp q_1 \amp \cdots \amp q_{n-1} \end{array} \right) \\ \amp=\amp \left( \begin{array}{c} q_0^H \\ q_1^H \\ \vdots \\ q_{n-1}^H \end{array} \right) \left( \begin{array}{c | c | c | c} q_0 \amp q_1 \amp \cdots \amp q_{n-1} \end{array} \right) \\ \amp=\amp \left( \begin{array}{c | c | c | c} q_0^H q_0 \amp q_0^H q_1 \amp \cdots \amp q_0^H q_{n-1} \\ \hline q_1^H q_0 \amp q_1^H q_1 \amp \cdots \amp q_1^H q_{n-1} \\ \hline \vdots \amp \vdots \amp \amp \vdots \\ \hline q_{n-1}^H q_0 \amp q_{n-1}^H q_1 \amp \cdots \amp q_{n-1}^H q_{n-1} \end{array} \right). \end{array} \end{equation*}

Now consider that $Q^H Q = I \text{:}$

\begin{equation*} \begin{array}{rcl} \left( \begin{array}{c | c | c | c} q_0^H q_0 \amp q_0^H q_1 \amp \cdots \amp q_0^H q_{n-1} \\ \hline q_1^H q_0 \amp q_1^H q_1 \amp \cdots \amp q_1^H q_{n-1} \\ \hline \vdots \amp \vdots \amp \amp \vdots \\ \hline q_{n-1}^H q_0 \amp q_{n-1}^H q_1 \amp \cdots \amp q_{n-1}^H q_{n-1} \end{array} \right) = \left( \begin{array}{c | c | c | c} 1 \amp 0 \amp \cdots \amp 0 \\ \hline 0 \amp 1 \amp \cdots \amp 0 \\ \hline \vdots \amp \vdots \amp \amp \vdots \\ \hline 0 \amp 0 \amp \cdots \amp 1 \end{array} \right). \end{array} \end{equation*}

Clearly $Q$ is orthonormal if and only if $q_0, q_1, \ldots, q_{n-1}$ are mutually orthonormal.

###### Homework2.2.3.3.

Let $Q \in \Cmxn \text{.}$

ALWAYS/SOMETIMES/NEVER: If $Q^H Q = I$ then $Q Q^H = I \text{.}$

• If $Q$ is a square matrix ($m = n$) then $Q^H Q = I$ means $Q^{-1} = Q^H \text{.}$ But then $Q Q^{-1} = I$ and hence $Q Q^H = I \text{.}$
• If $Q$ is not square, then $Q^H Q = I$ means $m \gt n \text{.}$ Hence $Q$ has rank equal to $n$ which in turn means $Q Q^H$ is a matrix with rank at most equal to $n \text{.}$ (Actually, its rank equals $n \text{.}$). Since $I$ has rank equal to $m$ (it is an $m \times m$ matrix with linearly independent columns), $Q Q^H$ cannot equal $I \text{.}$
More concretely: let $m \gt 1$ and $n = 1 \text{.}$ Choose $Q = \left( \begin{array}{c} e_0 \end{array} \right) \text{.}$ Then $Q^H Q = e_0^H e_0 = 1 = I \text{.}$ But