Descent direction

In optimization, a descent direction is a vector $\backslash mathbf\{p\}\backslash in\backslash mathbb\; R^n$ that points towards a local minimum $\backslash mathbf\{x\}^*$ of an objective function $f:\backslash mathbb\; R^n\backslash to\backslash mathbb\; R$.

Computing $\backslash mathbf\{x\}^*$ by an iterative method, such as line search defines a descent direction $\backslash mathbf\{p\}\_k\backslash in\backslash mathbb\; R^n$ at the $k$th iterate to be any $\backslash mathbf\{p\}\_k$ such that , where $\backslash langle\; ,\; \backslash rangle$ denotes the inner product. The motivation for such an approach is that small steps along $\backslash mathbf\{p\}\_k$ guarantee that $\backslash displaystyle\; f$ is reduced, by Taylor's theorem.

Using this definition, the negative of a non-zero gradient is always a

descent direction, as .

Numerous methods exist to compute descent directions, all with differing merits, such as gradient descent or the conjugate gradient method.

More generally, if $P$ is a positive definite matrix, then

$p\_k\; =\; -P\; \backslash nabla\; f(x\_k)$ is a descent direction at $x\_k$.{{cite book | author = J. M. Ortega and W. C. Rheinbold | title = Iterative Solution of Nonlinear Equations in Several Variables | pages = 243 | year = 1970 | doi = 10.1137/1.9780898719468

| isbn = 978-0-89871-461-6 }} This generality is used in preconditioned gradient descent methods.