ALAFF The Reduced Singular Value Decomposition

Subsection 2.3.4 The Reduced Singular Value Decomposition

Corollary 2.3.4.1. Reduced Singular Value Decomposition.

Let \(A \in \Cmxn \) and \(r = \rank( A )\text{.}\) There exist orthonormal matrix \(U_L \in \C^{m \times r} \text{,}\) orthonormal marix \(V_L \in \C^{n \times r} \text{,}\) and matrix \(\Sigma_{TL}\in \R^{r \times r} \) with \(\Sigma_{TL} = \diag{ \sigma_0, \ldots , \sigma_{r-1} }\) and \(\sigma_0 \geq \sigma_1 \geq \cdots \geq \sigma_{r-1} > 0 \) such that \(A = U_L \Sigma_{TL} V_L^H \text{.}\)

Homework 2.3.4.1.

Prove the above corollary.

Solution

Let \(A = U \Sigma V^H = \FlaOneByTwo{ U_L }{ U_R } \FlaTwoByTwo{\Sigma_{TL}}{0}{0}{0} \FlaOneByTwo{ V_L }{ V_R }^H \) be the SVD of \(A \text{,}\) where \(U_L \in \C^{m \times r} \text{,}\) \(V_L \in \C^{n \times r} \) and \(\Sigma_{TL} \in \R^{r \times r} \) with \(\Sigma_{TL} = \diag{ \sigma_0, \sigma_1, \cdots, \sigma_{r-1}} \) and \(\sigma_0 \geq \sigma_1 \geq \cdots \geq \sigma_{r-1} \gt 0 \text{.}\) Then

\begin{equation*} \begin{array}{l} A \\ ~~~ = ~~~~ \lt {\rm SVD~of~}A\gt \\ U \Sigma V^T \\ ~~~ = ~~~~ \lt {\rm Partitioning}\gt \\ \FlaOneByTwo{ U_L }{ U_R } \FlaTwoByTwo{\Sigma_{TL}}{0}{0}{0} \FlaOneByTwo{ V_L }{ V_R }^H \\ ~~~ = ~~~~ \lt {\rm partitioned~matrix-matrix~multiplication} \gt \\ U_L \Sigma_{TL} V_L^H. \end{array} \end{equation*}

Corollary 2.3.4.2.

Let \(A = U_L \Sigma_{TL} V_L^H \) be the Reduced SVD with \(U_L = \left( \begin{array}{ c | c | c } u_0 \amp \cdots \amp u_{r-1} \end{array} \right) \text{,}\) \(V_L = \left( \begin{array}{ c | c | c } v_0 \amp \cdots \amp v_{r-1} \end{array} \right) \text{,}\) and \(\Sigma_{TL} = \left( \begin{array}{ c | c | c } \sigma_0 \amp \amp \\ \hline \amp \ddots \amp \\ \hline \amp \amp \sigma_{r-1} \end{array} \right) \text{.}\) Then

\begin{equation*} A = \sigma_0 u_0 v_0^H + \cdots + \sigma_{r-1} u_{r-1} v_{r-1}^H. \end{equation*}

Remark 2.3.4.3.

This last result establishes that any matrix \(A \) with rank \(r \) can be written as a linear combination of \(r\) outer products:

\begin{equation*} A = \begin{array}[t]{c} \underbrace{ ~~~~~~~~~\sigma_0 u_0 v_0^H ~~~~~~~~~ } \\ \sigma_0 \!\!\!\! \begin{array}[t]{c|c} ~\amp~\\ ~\amp~\\ \end{array} \!\!\!\! \begin{array}[t]{c}\hline ~~~~~~~~~ \\ ~ \end{array} \end{array} + \begin{array}[t]{c} \underbrace{ ~~~~~~~~~\sigma_1 u_1 v_1^H ~~~~~~~~~ } \\ \sigma_1 \!\!\!\! \begin{array}[t]{c|c} ~\amp~\\ ~\amp~\\ \end{array} \!\!\!\! \begin{array}[t]{c} \hline ~~~~~~~~~ \\ ~ \end{array} \end{array} + \cdots + \begin{array}[t]{c} \underbrace{ ~~~~~~~~~\sigma_{r-1} u_{r-1} v_{r-1}^H ~~~~~~~~~ } \\ \sigma_{r-1} \!\!\!\! \begin{array}[t]{c|c} ~\amp~\\ ~\amp~\\ \end{array} \!\!\!\! \begin{array}[t]{c} \hline ~~~~~~~~~ \\ ~ \end{array} \end{array} . \end{equation*}