## Subsection8.2.4Method of Steepest Descent

For a function $f: \R^n \rightarrow \R$ that we are trying to minimize, for a given $x \text{,}$ the direction in which the function most rapidly increases in value at $x$ is given by its gradient,

\begin{equation*} \nabla f( x ) . \end{equation*}

Thus, the direction in which it decreases most rapidly is

\begin{equation*} - \nabla f( x ) . \end{equation*}

For our function

\begin{equation*} f( x ) = \frac{1}{2} x^T A x - x^T b \end{equation*}

this direction of steepest descent is given by

\begin{equation*} - \nabla f( x ) = - ( A x - b ) = b - A x, \end{equation*}

which we recognize as the residual. Thus, recalling that $r^{(k)} = b - A x^{(k)} \text{,}$ the direction of steepest descent at $x^{(k)}$ is given by $p^{(k)} = r^{(k)} = b - A x^{(k)} \text{.}$ These insights motivate the algorithms in Figure 8.2.4.1.