Skip to main content

Subsection 2.3.5 SVD of nonsingular matrices

Homework 2.3.5.1.

Let \(A \in \Cmxm \) and \(A = U \Sigma V^H \) be its SVD.

TRUE/FALSE: \(A \) is nonsingular if and only if \(\Sigma \) is nonsingular.

Answer

TRUE

Solution

\(\Sigma = U^H A V \text{.}\) The product of square matrices is nonsingular if and only if each individual matrix is nonsingular. Since \(U \) and \(V \) are unitary, they are nonsingular.

Homework 2.3.5.2.

Let \(A \in \Cmxm \) and \(A = U \Sigma V^H \) be its SVD with

\begin{equation*} \Sigma = \left( \begin{array}{ c c c c} \sigma_0 \amp 0 \amp \cdots \amp 0 \\ 0 \amp \sigma_1 \amp \cdots \amp 0 \\ \vdots \amp \vdots \amp \ddots \amp \vdots \\ 0 \amp 0 \amp \cdots \amp \sigma_{m-1} \end{array} \right) \end{equation*}

TRUE/FALSE: \(A \) is nonsingular if and only if \(\sigma_{m-1} \neq 0 \text{.}\)

Answer

TRUE

Solution

By the last homework, \(A \) is nonsingular if and only if \(\Sigma \) is nonsingular. A diagonal matrix is nonsingular if and only if its diagonal elements are all nonzero. \(\sigma_{0} \geq \cdots \geq \sigma_{m-1} \gt 0 \text{.}\) Hence the diagonal elements of \(\Sigma \) are nonzero if and only if \(\sigma_{m-1} \neq 0 \text{.}\)

Homework 2.3.5.3.

Let \(A \in \Cmxm \) be nonsingular and \(A = U \Sigma V^H \) be its SVD.

ALWAYS/SOMETIMES/NEVER: The SVD of \(A^{-1} \) equals \(V \Sigma^{-1} U^H \text{.}\)

Answer

SOMETIMES

Explain it!

Solution

It would seem that the answer is ALWAYS: \(A^{-1} = ( U \Sigma V^H )^{-1} = (V^H)^{-1} \Sigma^{-1} U^{-1} = V \Sigma^{-1} U^H \) with

\begin{equation*} \begin{array}{l} \Sigma^{-1} \\ ~~~=~~~~ \lt \gt \\ \left( \begin{array}{ c | c | c | c} \sigma_0 \amp 0 \amp \cdots \amp 0 \\ \hline 0 \amp \sigma_1 \amp \cdots \amp 0 \\ \hline \vdots \amp \vdots \amp \ddots \amp \vdots \\ \hline 0 \amp 0 \amp \cdots \amp \sigma_{m-1} \end{array} \right)^{-1} \\ ~~~=~~~~ \lt \gt \\ \left( \begin{array}{ c | c | c | c} 1/\sigma_0 \amp 0 \amp \cdots \amp 0 \\ \hline 0 \amp 1/\sigma_1 \amp \cdots \amp 0 \\ \hline \vdots \amp \vdots \amp \ddots \amp \vdots \\ \hline 0 \amp 0 \amp \cdots \amp 1/\sigma_{m-1} \end{array} \right). \end{array} \end{equation*}

However, the SVD requires the diagonal elements to be positive and ordered from largest to smallest.

So, only if \(\sigma_0 = \sigma_1 = \cdots = \sigma_{m-1}\) is it the case that \(V \Sigma^{-1} U^H \) is the SVD of \(A^{-1} \text{.}\) In other words, when \(\Sigma = \sigma_0 I \text{.}\)

Homework 2.3.5.4.

Let \(A \in \Cmxm \) be nonsingular and

\begin{equation*} \begin{array}{rcl} A \amp = \amp U \Sigma V^H \\ \amp = \amp \left( \begin{array}{c | c | c } u_0 \amp \cdots \amp u_{m-1} \end{array} \right) \left( \begin{array}{c | c | c} \sigma_0 \amp \cdots \amp 0 \\ \hline \vdots \amp \ddots \amp \vdots \\ \hline 0 \amp \cdots \amp \sigma_{m-1} \end{array} \right) \left( \begin{array}{c | c | c } v_0 \amp \cdots \amp v_{m-1} \end{array} \right)^H \end{array} \end{equation*}

be its SVD.

The SVD of \(A^{-1} \) is given by (indicate all correct answers):

  1. \(V \Sigma^{-1} U^H \text{.}\)
  2. \(\displaystyle \left( \begin{array}{c | c | c } v_0 \amp \cdots \amp v_{m-1} \end{array} \right) \left( \begin{array}{c | c | c} 1/\sigma_{0} \amp \cdots \amp 0 \\ \hline \vdots \amp \ddots \amp \vdots \\ \hline 0 \amp \cdots \amp 1/\sigma_{m-1} \end{array} \right) \left( \begin{array}{c | c | c } u_0 \amp \cdots \amp u_{m-1} \end{array} \right)^H \)
  3. \(\displaystyle \left( \begin{array}{c | c | c } v_{m-1} \amp \cdots \amp v_{0} \end{array} \right) \left( \begin{array}{c | c | c} 1/\sigma_{m-1} \amp \cdots \amp 0 \\ \hline \vdots \amp \ddots \amp \vdots \\ \hline 0 \amp \cdots \amp 1/\sigma_{0} \end{array} \right) \left( \begin{array}{c | c| c } u_{m-1} \amp \cdots \amp u_{0} \end{array} \right)^H .\)

  4. \(( V P^H ) ( P \Sigma^{-1} P^H ) ( U P^H )^H \) where \(P = \left( \begin{array}{c c c c} 0 \amp \cdots \amp 0 \amp 1 \\ 0 \amp \cdots \amp 1 \amp 0 \\ \vdots \amp \amp \vdots \amp \vdots \\ 1 \amp \cdots \amp 0 \amp 0 \end{array} \right) \)

Answer

3. and 4.

Explain it!

Solution

This question is a bit tricky.

  1. It is the case that \(A^{-1} = V \Sigma^{-1} U^H \text{.}\) However, the diagonal elements of \(\Sigma^{-1} \) are ordered from smallest to largest, and hence this is not its SVD.

  2. This is just Answer 1. but with the columns of \(U\) and \(V \text{,}\) and the elements of \(\Sigma \text{,}\) exposed.

  3. This answer corrects the problems with the previous two answers: it reorders colums of \(U \) and \(V \) so that the diagonal elements of \(\Sigma\) end up ordered from largest to smallest.

  4. This answer is just a reformulation of the last answer.

Homework 2.3.5.5.

Let \(A \in \Cmxm \) be nonsingular.

TRUE/FALSE: \(\| A^{-1} \|_2 = 1 / \min_{\| x \|_2 = 1} \| A x \|_2 \text{.}\)

Answer

TRUE

Solution
\begin{equation*} \begin{array}{l} \| A^{-1} \|_2 \\ ~~~=~~~~ \lt {\rm definition} \gt \\ \max_{x \neq 0} \frac{\| A^{-1} x \|_2}{ \| x \|_2} \\ ~~~=~~~~ \lt {\rm algebra} \gt \\ \max_{x \neq 0} \frac{1}{\frac{\| x \|_2}{ \| A^{-1} x \|_2}} \\ ~~~=~~~~ \lt {\rm algebra~} \gt \\ \frac{1}{\min_{x \neq 0} \frac{\| x \|_2}{ \| A^{-1} x \|_2}} \\ ~~~=~~~~ \lt {\rm substitute~} z = A^{-1} x \gt \\ \frac{1}{\min_{A z \neq 0} \frac{\| A z \|_2}{ \| z \|_2}} \\ ~~~=~~~~ \lt A {\rm ~is~nonsingular} \gt \\ \frac{1}{\min_{z \neq 0} \frac{\| A z \|_2}{ \| z \|_2}} \\ ~~~=~~~~ \lt x = z/\| z \|_2 \gt \\ \frac{1}{\min_{\| x \|_2 =1} \| A x \|_2} \\ \end{array} \end{equation*}

In Subsection 2.3.2, we discussed the case where \(A \in \R^{2 \times 2} \text{.}\) Letting \(A = U \Sigma V^T \) and partitioning

\begin{equation*} A = \left( \begin{array}{c | c} u_0 \amp u_1 \end{array} \right) \left( \begin{array}{c | c} \sigma_0 \amp 0 \\ \hline 0 \amp \sigma_1 \end{array} \right) \left( \begin{array}{c | c} v_0 \amp v_1 \end{array} \right)^T \end{equation*}

yielded the pictures

\(\R^{2} \text{:}\) Domain of \(A \text{:}\)

\(\R^{2} \text{:}\) Range (codomain) of \(A \text{:}\)

This captures what the condition number \(\kappa_2( A ) = \sigma_0 / \sigma_{n-1} \) captures: how elongated the oval that equals the image of the unit ball is. The more elongated, the greater the ratio \(\sigma_0 / \sigma_{n-1} \text{,}\) and the worse the condition number of the matrix. In the limit, when \(\sigma_{n-1} = 0 \text{,}\) the unit ball is mapped to a lower dimensional set, meaning that the transformation cannot be "undone."

Ponder This 2.3.5.6.

For the 2D problem discussed in this unit, what would the image of the unit ball look like as \(\kappa_2( A ) \rightarrow \infty \text{?}\) When is \(\kappa_2( A ) = \infty \text{?}\)

login