The reader will recall that
is a *descent direction* for
*f* at *x*^{(k)} if

This condition implies that

Indeed, if represents the one-dimensional ``slice'' of

then the descent condition implies that and hence that for all sufficiently small (see Figure 1).

Given *x*^{(k)} and a descent direction *p*, it is possible to reduce *f* by
moving in the direction of *p*, that is, by choosing an appropriate and defining
.
A procedure for choosing
is referred to
as a *line search* (since *x*^{(k+1)} is found on the (half-)line parametrized
as
). I will discuss line searches (a solution to the third
difficulty described above in the Introduction) later. For now, I want to
concentrate on methods for producing descent directions.

Newton's method produces the direction

This is a descent direction if

that is, if

that is, if

Condition (1) will hold if is positive definite. The reader may recall that the eigenvalues of

The following observation is essential: If *H* is *any* symmetric
positive definite matrix, then
is a descent direction
for *f* at *x*^{(k)}. This suggests the following modification of Newton's
method:

where

I will now describe two specific quasi-Newton methods.