Advanced Linear Algebra: Foundations to Frontiers

Subsection 1.6.2 Summary

If \(\alpha, \beta \in \mathbb C \) with \(\alpha = \alpha_r + \alpha_c i \) and \(\beta = \beta_r + i \beta_c \text{,}\) where \(\alpha_r, \alpha_c, \beta_r, \beta_c \in \mathbb R \text{,}\) then

• Conjugate: \(\overline \alpha = \alpha_r - \alpha_c i \text{.}\)

• Product: \(\alpha \beta = (\alpha_r \beta_r - \alpha_c \beta_c ) + ( \alpha_r \beta_c + \alpha_c \beta_r ) i \text{.}\)

• Absolute value: \(\vert \alpha \vert = \sqrt{ \alpha_r^2 + \alpha_c^2 } = \sqrt{ \overline \alpha \alpha } \text{.}\)

Let \(x, y \in \Cm \) with \(x = \left( \begin{array}{c} \chi_0 \\ \vdots \\ \chi_{m-1} \end{array} \right) \mbox{ and } y = \left( \begin{array}{c} \psi_0 \\ \vdots \\ \psi_{m-1} \end{array} \right). \) Then

• Conjugate:

  \begin{equation*} \overline x = \left( \begin{array}{c} \overline \chi_0 \\ \vdots \\ \overline \chi_{m-1} \end{array} \right). \end{equation*}

• Transpose of vector:

  \begin{equation*} x^T = \left( \begin{array}{c c c} \chi_0 \amp \cdots \amp \chi_{m-1} \end{array} \right) \end{equation*}

• Hermitian transpose (conjugate transpose) of vector:

  \begin{equation*} x^H = \overline x^T = \overline {x^T} = \left( \begin{array}{c c c} \overline \chi_0 \amp \cdots \amp \overline \chi_{m-1} \end{array} \right). \end{equation*}

• Dot product (inner product): \(x^H y = \overline x^T y = \overline {x^T} y = \overline \chi_0 \psi_0 + \cdots + \overline \chi_{m-1} \psi_{m-1} = \sum_{i=0}^{m-1} \overline \chi_i \psi_i \text{.}\)

• 2-norm (Euclidean length): \(\| x \|_2 = \sqrt{ x^H x } = \sqrt{ \vert \chi_0 \vert^2 + \cdots + \vert \chi_{m-1} \vert^2 } = \sqrt{ \overline \chi_0 \chi_0 + \cdots + \overline \chi_{m-1} \chi_{m-1} } \) \(= \sqrt{ \sum_{i=0}^{m-1} \vert \chi_i \vert^2 } \text{.}\)

• \(p \)-norm: \(\| x \|_p = \sqrt[p]{ \vert \chi_0 \vert^p + \cdots + \vert \chi_{m-1} \vert^p } = \sqrt[p]{\sum_{i=0}^{m-1} \vert \chi_i \vert^p } \text{.}\)

• 1-norm: \(\| x \|_1 = \vert \chi_0 \vert + \cdots + \vert \chi_{m-1} \vert = \sum_{i=0}^{m-1} \vert \chi_i \vert \text{.}\)

• \(\infty \)-norm: \(\| x \|_\infty = \max( \vert \chi_0 \vert, \ldots , \chi_{m-1} \vert ) = \max_{i=0}^{m-1} \vert \chi_i \vert = \lim_{p \rightarrow \infty} \| x \|_p \text{.}\)

• Unit ball: Set of all vectors with norm equal to one. Notation: \(\| x \| = 1. \)

If \(L \) is a linear transformation and we let \(a_j = L( e_j ) \) then

is the matrix that represents \(L \) in the sense that \(A x = L( x ) \text{.}\)

Partition \(C \text{,}\) \(A \text{,}\) and \(B \) by rows and columns

and

then \(C := A B \) can be computed in the following ways:

1. By columns:

   \begin{equation*} \left( \begin{array}{c | c | c} c_0 \amp \cdots \amp c_{n-1} \end{array} \right) := A \left( \begin{array}{c | c | c} b_0 \amp \cdots \amp b_{n-1} \end{array} \right) = \left( \begin{array}{c | c | c} A b_0 \amp \cdots \amp A b_{n-1} \end{array} \right). \end{equation*}

   In other words, \(c_j := A b_j \) for all columns of \(C \text{.}\)

2. By rows:

   \begin{equation*} \left( \begin{array}{c} \widetilde c_0^T \\ \hline \vdots \\ \hline \widetilde c_{m-1}^T \end{array} \right) := \left( \begin{array}{c} \widetilde a_0^T \\ \hline \vdots \\ \hline \widetilde a_{m-1}^T \end{array} \right) B = \left( \begin{array}{c} \widetilde a_0^T B \\ \hline \vdots \\ \hline \widetilde a_{m-1}^T B \end{array} \right). \end{equation*}

   In other words, \(\widetilde c_i^T = \widetilde a_i^T B\) for all rows of \(C \text{.}\)

3. As the sum of outer products:

   \begin{equation*} C := \left( \begin{array}{c | c | c} a_0 \amp \cdots \amp a_{k-1} \end{array} \right) \left( \begin{array}{c} \widetilde b_0^T \\ \hline \vdots \\ \hline \widetilde b_{k-1}^T \end{array} \right) = a_0 \widetilde b_0^T + \cdots + a_{k-1} \widetilde b_{k-1} ^T, \end{equation*}

   which should be thought of as a sequence of rank-1 updates, since each term is an outer product and an outer product has rank of at most one.

Partition \(C \text{,}\) \(A \text{,}\) and \(B \) by blocks (submatrices),

and

where the partitionings are "conformal." Then

Let \(A \in \mathbb C^{m \times n} \) and

Then

• Conjugate of matrix:

  \begin{equation*} \overline A = \left( \begin{array}{cc c } \overline \alpha_{0,0} \amp \cdots \amp \overline \alpha_{0,n-1} \\ \vdots \amp \vdots \\ \overline \alpha_{m-1,0} \amp \cdots \amp \overline \alpha_{m-1,n-1} \end{array} \right). \end{equation*}

• Transpose of matrix:

  \begin{equation*} A^T = \left( \begin{array}{c c c} \alpha_{0,0} \amp \cdots \amp \alpha_{m-1,0} \\ \vdots \amp \vdots \\ \alpha_{0,n-1} \amp \cdots \amp \alpha_{m-1,n-1} \\ \end{array} \right). \end{equation*}

• Conjugate transpose (Hermitian transpose) of matrix:

  \begin{equation*} A^H = \overline A^T = {\overline A^T} = \left( \begin{array}{c c c} \overline \alpha_{0,0} \amp \cdots \amp \overline \alpha_{m-1,0} \\ \vdots \amp \vdots \\ \overline \alpha_{0,n-1} \amp \cdots \amp \overline \alpha_{m-1,n-1} \\ \end{array} \right). \end{equation*}

• Frobenius norm: \(\| A \|_F = \sqrt{\sum_{i=0}^{m-1} \sum_{j=0}^{n-1} \vert \alpha_{i,j} \vert^2} = \sqrt{ \sum_{j=0}^{n-1} \| a_j \|_2^2 } = \sqrt{ \sum_{i=0}^{m-1} \| \widetilde a_i \|_2^2 } \)

• matrix p-norm: \(\| A \|_p = \max_{x \neq 0} \frac{\| A x \|_p}{\| x \|_p} = \max_{\| x \|_p =1} \| A x \|_p .\)

• matrix 2-norm: \(\| A \|_2 = \max_{x \neq 0} \frac{\| A x \|_2}{\| x \|_2} = \max_{\| x \|_2 =1} \| A x \|_2 = \| A^H \|_2.\)

• matrix 1-norm: \(\| A \|_1 = \max_{x \neq 0} \frac{\| A x \|_1}{\| x \|_1} = \max_{\| x \|_1 =1} \| A x \|_1 = \max_{0 \leq j \lt n} \| a_j \|_1 = \| A^H \|_\infty. \)

• matrix \(\infty \)-norm: \(\| A \|_\infty = \max_{x \neq 0} \frac{\| A x \|_\infty}{\| x \|_\infty} = \max_{\| x \|_\infty =1} \| A x \|_\infty = \max_{0 \leq i \lt m} \| \widetilde a_i \|_1 = \| A^H \|_1.\)

Let \(A, \Delta\!A \in \mathbb C^{m \times m} \text{,}\) \(x, \delta\!x, b, \delta\!b \in \Cm \text{,}\) \(A \) be nonsingular, and \(\| \cdot \| \) be a vector norm and corresponding subordinate matrix norm. Then