next up previous
Next: The logarithmic barrier method Up: Introduction to inequality-constrained optimization: Previous: Introduction to inequality-constrained optimization:

Introduction

I now turn to the inequality-constrained nonlinear program

  
$\displaystyle \min$   f(x) (1)
s.t.   $\displaystyle h(x)\ge 0,$ (2)

where $f:{\bf {\rm R}}^n\rightarrow{\bf {\rm R}}$ and $h:{\bf {\rm R}}^n\rightarrow{\bf {\rm R}}^p$. The reader should notice that $h(x)\ge 0$ is to be interpreted component-wise; that is, $h(x)\ge 0$ if and only if $h_i(x)\ge 0$ for each $i=1,2,\ldots,p$. Inequality constraints are common in optimization problems; indeed, almost every optimization problems does or could have inequality constraints, although it is sometimes safe to ignore them.

The most common type of inequality constraints are simple bounds that the variables must satisfy. For example, if every variable must satisfy both lower and upper bound, then the constraints could be written as

\begin{displaymath}a\le x\le b,
\end{displaymath}

where $a,b\in{\bf {\rm R}}^n$ with $a\le b$. To incorporate these constraints into the standard form (2), one would write them as

\begin{eqnarray*}x-a\ge 0,\\
b-x\ge 0.
\end{eqnarray*}


Therefore, if a problem contained only these simple bounds, the constraint function $h:{\bf {\rm R}}^n\rightarrow{\bf {\rm R}}^p$ (p=2n) would be defined as

\begin{displaymath}h(x)=\left[\begin{array}{c}x-a\\ b-x\end{array}\right]=
\left...
...\\
b_1-x_1\\
b_2-x_2\\
\vdots\\
b_n-x_n\end{array}\right].
\end{displaymath}

Simple bounds are common because variables typically have a physical interpretation and some real numbers do not make physical sense for a given variable. For example, physical parameters (density, elasticity, thermal conductivity, etc.) typically cannot take negative values, so nonnegativity constraints of the form

\begin{displaymath}x\ge 0
\end{displaymath}

are common. The same constraint appears when the variables represents quantities (for example, the number of barrels of petroleum) that cannot be negative. Upper bounds often represent limited resources.

In some cases, it may seem that the appropriate constraint is a strict inequality, as when the variables represent physical parameters that must be strictly positive. However, a strict inequality constraint may lead to an optimization problem that is ill-posed in that a minimizer is infeasible but on the boundary of the feasible set. In such as case, there may not be a solution to the optimization problem. A simple example of this is

\begin{eqnarray*}&\min&x^2\\
&s.t.&x>0.
\end{eqnarray*}


For this reason, strict inequality constraints are not used in nonlinear programming.

When the appropriate constraint seems to be a strict inequality, one of the following is usually true:

1.
The problem is expected to have a solution that easily satisfies the strict inequality. In this case, as I will show below, the constraint plays little part in the theory or algorithm other than as a ``sanity check'' on the variable. Therefore, the nonstrict inequality constraint is just as useful.
2.
Due to noise in the data or other complications, the solution to the optimization problem may lie on the boundary of the feasible set, even though such a solution is not physically meaningful. In this case, the inequality must be perturbed slightly and written as a nonstrict inequality. For example, a constraint of the form xi>0 should be replaced with $x_i\ge a_i$, where ai>0 is the smallest value of xi that is physically meaningful.
It may not be clear how to distinguish the first case from the second; if it is not, the second approach is always valid. The first case can usually be identified by the fact that the solution is not expected to be close to satisfying the inequality as an equation.

Inequality constraints that are not simple bounds are usually bounds on derived quantities. For example, if x represents design parameters for a certain object, the mass m of the object may be represented as a function of x: m=q(x). In this case, constraints of the form $q(x)\ge M_1$ or $q(x)\le M_2$ (or both) may be appropriate.

I now present a simple example of an inequality-constrained nonlinear program.

Example 1.1   I define $f:{\bf {\rm R}}^2\rightarrow{\bf {\rm R}}$ and $h:{\bf {\rm R}}^2\rightarrow{\bf {\rm R}}^2$ by

\begin{displaymath}f(x)=(x_1-1)^2+2(x_2-2)^2,\
h(x)=\left[\begin{array}{c}
1-x_1^2-x_2^2\\
x_1+x_2\end{array}\right]
\end{displaymath}

The feasible set for

\begin{eqnarray*}&\min&f(x)\\
&s.t.&h(x)\ge 0
\end{eqnarray*}


is shown in Figure 1, together with the contours of the objective function f.
  
Figure: The contours of f and the feasible set determined by $h(x)\ge 0$(see Example 1.1). The feasible set is half of the unit disk (the shaded region). The minimizer $x^*\doteq (0.3116,0.9502)$ is indicated by an asterisk.
\includegraphics[height=3in,width=4in]{{barex1.eps}}

An important aspect of this example is that the second constraint, $x_1+x_2\ge 0$, does not affect the solution. If the second constraint were changed to $x_1+x_2\ge u$ for some value of u that is not too large, or if the second constraint were simply omitted, the optimization problem would have the same solution. Both the theory of and algorithms for inequality-constrained problems must address the issue of ``inactive'' constraints.

The first step in analyzing NLP (1-2) should be to derive the optimality conditions. However, since the optimality conditions are somewhat more complicated than in the case of equality constraints, I will begin by presenting an algorithm for solving (1-2), namely, the logarithmic barrier method. From this algorithm, I will deduce the optimality conditions for the inequality-constrained NLP. A rigorous derivation of these optimality conditions will be given later.


next up previous
Next: The logarithmic barrier method Up: Introduction to inequality-constrained optimization: Previous: Introduction to inequality-constrained optimization:
Mark S. Gockenbach
2003-03-30