Skip to main content

Unit 2.3.4 The Reduced Singular Value Decomposition

Homework 2.3.4.1.
Prove the above corollary.
Solution

Let \(A = U \Sigma V^H = \FlaOneByTwo{ U_L }{ U_R } \FlaTwoByTwo{\Sigma_{TL}}{0}{0}{0} \FlaOneByTwo{ V_L }{ V_R }^H \) be the SVD of \(A \text{,}\) where \(U_L \in \C^{m \times r} \text{,}\) \(V_L \in \C^{n \times r} \) and \(\Sigma_{TL} \in \R^{r \times r} \) with \(\Sigma_{TL} = \diag{ \sigma_0, \sigma_1, \cdots, \sigma_{r-1}} \) and \(\sigma_0 \geq \sigma_1 \geq \cdots \geq \sigma_{r-1} \gt 0 \text{.}\) Then

\begin{equation*} \begin{array}{l} A \\ ~~~ = ~~~~ \lt {\rm SVD~of~}A\gt \\ U \Sigma V^T \\ ~~~ = ~~~~ \lt {\rm Partitioning}\gt \\ \FlaOneByTwo{ U_L }{ U_R } \FlaTwoByTwo{\Sigma_{TL}}{0}{0}{0} \FlaOneByTwo{ V_L }{ V_R }^H \\ ~~~ = ~~~~ \lt {\rm partitioned~matrix-matrix~multiplication} \gt \\ U_L \Sigma_{TL} V_L^H. \end{array} \end{equation*}
Remark 2.3.4.3.

This last result establishes that any matrix \(A \) with rank \(r \) can be written as a linear combination of \(r\) outer products:

\begin{equation*} A = \begin{array}[t]{c} \underbrace{ ~~~~~~~~~\sigma_0 u_0 v_0^H ~~~~~~~~~ } \\ \sigma_0 \!\!\!\! \begin{array}[t]{c|c} ~\amp~\\ ~\amp~\\ \end{array} \!\!\!\! \begin{array}[t]{c}\hline ~~~~~~~~~ \\ ~ \end{array} \end{array} + \begin{array}[t]{c} \underbrace{ ~~~~~~~~~\sigma_1 u_1 v_1^H ~~~~~~~~~ } \\ \sigma_1 \!\!\!\! \begin{array}[t]{c|c} ~\amp~\\ ~\amp~\\ \end{array} \!\!\!\! \begin{array}[t]{c} \hline ~~~~~~~~~ \\ ~ \end{array} \end{array} + \cdots + \begin{array}[t]{c} \underbrace{ ~~~~~~~~~\sigma_{r-1} u_{r-1} v_{r-1}^H ~~~~~~~~~ } \\ \sigma_{r-1} \!\!\!\! \begin{array}[t]{c|c} ~\amp~\\ ~\amp~\\ \end{array} \!\!\!\! \begin{array}[t]{c} \hline ~~~~~~~~~ \\ ~ \end{array} \end{array} . \end{equation*}