Uzawa iteration

In numerical mathematics, the Uzawa iteration is an algorithm for solving saddle point problems. It is named after Hirofumi Uzawa and was originally introduced in the context of concave programming.{{cite book |first=H. |last=Uzawa |chapter=Iterative methods for concave programming |editor1-first=K. J. |editor1-last=Arrow |editor2-first=L. |editor2-last=Hurwicz |editor3-first=H. |editor3-last=Uzawa |title=Studies in linear and nonlinear programming |url=https://archive.org/details/studiesinlinearn0000arro |url-access=registration |publisher=Stanford University Press |year=1958 }}

Basic idea

We consider a saddle point problem of the form

: $\begin{pmatrix} A & B\\ B^* & \end{pmatrix} \begin{pmatrix} x_1\\ x_2 \end{pmatrix}
= \begin{pmatrix} b_1\\ b_2 \end{pmatrix},$

where $A$ is a symmetric positive-definite matrix.

Multiplying the first row by $B^* A^{-1}$ and subtracting from the second row yields the upper-triangular system

: $\begin{pmatrix} A & B\\ & -S \end{pmatrix} \begin{pmatrix} x_1\\ x_2 \end{pmatrix}
= \begin{pmatrix} b_1\\ b_2 - B^* A^{-1} b_1 \end{pmatrix},$

where $S := B^* A^{-1} B$ denotes the Schur complement.

Since $S$ is symmetric positive-definite, we can apply standard iterative methods like the gradient descent

method or the conjugate gradient method to solve

: $S x_2 = B^* A^{-1} b_1 - b_2$

in order to compute $x_2$ .

The vector $x_1$ can be reconstructed by solving

: $A x_1 = b_1 - B x_2. \,$

It is possible to update $x_1$ alongside $x_2$ during the iteration for the Schur complement system and thus obtain an efficient algorithm.

Implementation

We start the conjugate gradient iteration by computing the residual

: $r_2 := B^* A^{-1} b_1 - b_2 - S x_2 = B^* A^{-1} (b_1 - B x_2) - b_2 = B^* x_1 - b_2,$

of the Schur complement system, where

: $x_1 := A^{-1} (b_1 - B x_2)$

denotes the upper half of the solution vector matching the initial guess $x_2$ for its lower half. We complete the initialization by choosing the first search direction

: $p_2 := r_2.\,$

In each step, we compute

: $a_2 := S p_2 = B^* A^{-1} B p_2 = B^* p_1$

and keep the intermediate result

: $p_1 := A^{-1} B p_2$

for later.

The scaling factor is given by

: $\alpha := p_2^* a_2 /p_2^* r_2$

and leads to the updates

: $x_2 := x_2 + \alpha p_2, \quad r_2 := r_2 - \alpha a_2.$

Using the intermediate result $p_1$ saved earlier, we can also update the upper part of the solution vector

: $x_1 := x_1 - \alpha p_1.\,$

Now we only have to construct the new search direction by the Gram–Schmidt process, i.e.,

: $\beta := r_2^* a_2 / p_2^* a_2,\quad p_2 := r_2 - \beta p_2.$

The iteration terminates if the residual $r_2$ has become sufficiently small or if the norm of $p_2$ is significantly smaller than $r_2$ indicating that the Krylov subspace has been almost exhausted.

Modifications and extensions

If solving the linear system $A x=b$ exactly is not feasible, inexact solvers can be applied.{{cite journal |first=H. C. |last=Elman |first2=G. H. |last2=Golub |author-link2=Gene H. Golub |title=Inexact and preconditioned Uzawa algorithms for saddle point problems |journal=SIAM J. Numer. Anal. |volume=31 |issue=6 |pages=1645–1661 |year=1994 |doi=10.1137/0731085 |citeseerx=10.1.1.307.8178 }}{{cite journal |first=J. H. |last=Bramble |author-link=James H. Bramble |first2=J. E. |last2=Pasciak |first3=A. T. |last3=Vassilev |title=Analysis of the inexact Uzawa algorithm for saddle point problems |journal=SIAM J. Numer. Anal. |volume=34 |issue=3 |pages=1072–1982 |year=1997 |doi=10.1137/S0036142994273343 |citeseerx=10.1.1.52.9559 }}{{cite journal |first=W. |last=Zulehner |title=Analysis of iterative methods for saddle point problems. A unified approach |journal=Math. Comp. |volume=71 |issue= 238|pages=479–505 |year=1998 |doi=10.1090/S0025-5718-01-01324-2 |doi-access=free }}

If the Schur complement system is ill-conditioned, preconditioners can be employed to improve the speed of convergence of the underlying gradient method.{{cite book |first=C. |last=Gräser |first2=R. |last2=Kornhuber |chapter=On Preconditioned Uzawa-type Iterations for a Saddle Point Problem with Inequality Constraints |title=Domain Decomposition Methods in Science and Engineering XVI |series=Lec. Not. Comp. Sci. Eng. |volume=55 |pages=91–102 |year=2007 |doi=10.1007/978-3-540-34469-8_8 |isbn=978-3-540-34468-1 |citeseerx=10.1.1.72.9238 }}

Inequality constraints can be incorporated, e.g., in order to handle obstacle problems.

Uzawa iteration

Basic idea

Implementation

Modifications and extensions

References

Further reading