The Maximum Principle: General Inequality Constraints. Chapter 4. 4.1 Pure State Variable Inequality Constraints: Indirect Method. It is common to require state variable to remain nonnegative, i.e., i.e., x i ( t ) 0, i =1,2,…, n . Constraints exhibiting (4.1) are
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
The Maximum Principle: General Inequality Constraints
It is common to require state variable to remain
i.e., xi(t) 0, i=1,2,…,n. Constraints exhibiting (4.1) are
called pure state variable inequality constraints. The
general form is
With (4.1): at any point where a component xi(t)>0, the
corresponding constraint xi(t) 0is not binding and can
In any interval where xi(t)=0,we must have so
that xi does not become negative.
The control must be constrained to satisfy
making fi 0, as a constraint of the mixed type (3.3) over
the interval. We can add the constraint
We associate multipliers ηiwith (4.3) whenever (4.3)
must be imposed, i.e., whenever xi(t)=0. A convenient
way to do this is to impose an “either or” condition
ηi xi=0. This will make ηi=0 whenever xi>0.
We can now form the Lagrangian
where H is as defined in (3.7) and η=(η1,η2 ,…,ηn).
We apply the maximum principle in (3.11) with
additional necessary conditions satisfied by η
and the modified transversality condition
where is a constant vector satisfying
Since the constraints are adjoined indirectly (in this
case via their first time derivative) to form the
Lagrangian, the method is called the indirect adjoining
approach. If on the other hand the Lagrangian L is
formed by adjoining directly the constraints (4.1), i.e.,
where is a multiplier associated with (4.1), then the
method is referred to as the direct adjoining approach.
Remark 4.1 The first two conditions in (4.5) are
complementary slackness conditions on the multiplier
η. The last condition is difficult to motivate. The
direct maximum principle multiplier is related to η as
The complementary slackness conditions for
the direct multiplier are 0 and x*=0. Since 0,
it follows that
Example 4.1 Consider the problem:
which implies the optimal control to be
When x=0, we impose , in order to insure
that (4.10) holds. Therefore, the optimal control on the
state constraint boundary is
Now we form the Lagrangian
where 1,2, and satisfy the complementary
Furthermore, the optimal trajectory must satisfy
From the Lagrangian we also get
Let us first try (2)= =0. Then the solution for is the
same as in Example 2.2, namely,
Since (t)-1 on [0,1] and x(0)=1>0, the original
optimal control given by (4.11) is u*(t)=-1. Substituting
this into (4.8) we get x(t)=1-t, which is positive for t<1.
In the time interval [0,1) by (4.14), 2=0 since *<1,
and by (4.15) =0 because x>0. Therefore, 1(t)=-(t)
=2-t >0 for 0t<1, and this with u=-1 satisfies (4.13).
At t=1 we have x(1)=0 so the optimal control is given
by (4.12), which is u*(1)=0. Now assume that we
continue to use the control u*(t)=0 in the interval
1 t 2.
With this control, we can solve (4.8) beginning with
x(1)=0, and obtain x(t)=0 for 1t2. Since (t) 0 in the
same interval, we see that u*(t)=0 satisfies (4.12)
throughout this interval.To complete the solution, we
calculate the Lagrange multipliers.since u*=0, we have
1=2=0 throughout 1t2. Then from (4.16) we obtain
=-=2-t0 which, with x=0, satisfies (4.15). This
completes the solution.
Remark 4.2 In instances where the initial state or the
final state or both are on the constraint boundary, the
maximum principle may degenerate I.e., there is no nontrivial solution of the necessary conditions,i.e., (t)0, t[0,T], where T is the terminal time.
There may be a piecewise continuous (t) satisfying a
at a time at which the state trajectory hits its
boundary value zero.
Example 4.2 Consider Example 4.1 with T=3 and the
terminal state constraint
Clearly, the optimal control u* will be the one that
keeps x as small as possible, subject to the state
constraint (4.1) and the boundary condition
We only compute the adjoint function and multipliers
that satisfy the optimality conditions. These are
Mixed constraints as in Chapter 3 and the pure state
where h: En x E1 Ep. By the definition of function h,
(4.26) represents a set of p constraints hi(x,t)0,
i=1,2,…,p.It is noted that the constraint hi0 is called
a constraint of rth order if the rth time derivative of
hiis the first time a term in control u appears in the
expression by putting f(x,u,t) for after each
In case of first order constraints, h1(x,u,t) is as follows:
With respect to the ith constraint hi(x,t)0, a sub
Interval (1,2) [0,T] with 1< 2is called an interior
interval if hi(x(t),t)>0 for all t(1,2). If the optimal
trajectory satisfies hi(x(t),t)=0 for for
some i, then is called a boundary interval. An
instant is called an entry time if there is an interior
interval ending at t= and a boundary interval
starting at . Correspondingly, is called an exit
time if a boundary interval ends and an interior interval
starts at .
If the trajectory just touches the boundary at time ,
i.e., and if the trajectory is in
the interior just before and just after ,then is called
a contact time. Entry, exit and contact times are called
Full rank condition on any boundary interval is
To formulate the maximum principle for the problem
with mixed constraints as well as first-order pure state
constraints, we form the Lagrangian as
H is defined in (3.7), u satisfies the complementary
slackness conditions stated in (3.9), and Eqsatisfies
The maximum principle sates that the necessary
conditions for u* to be an optimal control are that there
exists multipliers ,,,,,and , and the jump
parameters , which satisfy (4.29) that follows.
Note that the jump conditions on the adjoint variables in (4.29) generalize the jump condition on H in (4.29)requires that the Hamiltonian should be continuous at if ht = 0.Example 4.3 Consider the following problem with the discount rate 0:
Figure 4.2: Feasible State Space and Optimal State Trajectory
for Example 4.3
Note that at the entry time t=1 to the state constraint (4.34), the control u* and, therefore, h1*=u*+2(t-2) is discontinuous, i.e., the entry is non-tangential. on the other hand, u* and h1* are continuous at t=2 so that the exit is tangential. With u* and x* thus obtained, we must obtain , 1 ,2, , and so that the necessary optimality conditions (4.29) holds, i.e.,
which, along with u* and x*, satisfy (4.29).
Note, furthermore, that is continuous at the exit
timet=2. At the entry time so that (4.30) also holds.
With the Hamiltonian H as defined in (3.33), we can
write The Lagrangian
We can now state the current-value form of the
maximum principle as given in (4.42)
The sufficiency results can be stated in the indirect
adjoining framework. In order to do so, let us define the
Hamiltonian H and the Lagrangian Ld in the direct
where d, d, and dare multipliers in the direct
formulation, corresponding to , , and in the direct
It can be shown that:
Theorem 4.1Let satisfy the necessary
conditions in (4.29) and let
If is concave in (x,u) at each t [0,T], S in
(3.2) is concave in x,g in (3.3) is quasiconcave in (x,u)
h in (4.26) and a in (3.4) are quasiconcave in x, and b
in (3.5) is linear in x, then (x*,u*) is optimal.
Theorem 4.2 Theorem 4.1 remains valid if the concavity of in (x,u) at each t is replaced by the concavity of the maximized Hamiltonian
in x, where
Theorem 4.1 and Theorem 4.2 are written for finitehorizon problems and remains valid if the transversality conditions on the adjoint variables (4.29) is replaced by the following limiting transversality condition
Example 4.1 (Continued) First we obtain the direct
It is easy to see that
is linear and hence concave in (x,u) at each t[0,2].
are linear and hence quasiconcave in (x,u) and x,
respectively. Functions S 0, a 0 and b 0 satisfy
The conditions of Theorem 4.1 trivially.