Schulz M. Control Theory in Physics and Other Fields of Science: Concepts, Tools, and Applications
Подождите немного. Документ загружается.
2.5 The Hamilton–Jacobi Equation 55
where C>0 is a free parameter. That means we are interested in small
fluctuations of the kinetic energy around their thermodynamically expected
average 3/2MT and simultaneously in small friction coefficients. From here,
we obtain the generalized Hamiltonian (2.94)
H =
M
i=1
P
i
(F
i
− up
i
)+Q
i
p
i
m
−
1
2
M
i=1
p
2
i
2m
−
3
2
MT
2
−
C
2
u
2
(2.123)
with the generalized momenta Q
i
(corresponding to ˙x
i
)andP
i
(corresponding
to ˙p
i
). From here, we obtain the preoptimized control
u
(∗)
= −
1
C
M
i=1
P
i
p
i
. (2.124)
Hence, we get the evolution equations
˙x
i
=
p
i
m
and ˙p
i
= F
i
+
p
i
C
M
j=1
P
j
p
j
(2.125)
and the corresponding set of adjoint evolution equations
˙
Q
i
= −
M
j=1
P
j
∂F
j
∂x
i
(2.126)
and
˙
P
i
= −
P
i
C
M
j=1
P
j
p
j
−
Q
i
m
+
M
j=1
p
2
j
2m
−
3
2
MT
p
i
m
. (2.127)
The numerical solution of the extended system of evolution equations
8
(2.125),
(2.126), and (2.127) now yields a deterministic substitute process for the evo-
lution of a many-particle system in a thermodynamic bath.
2.5 The Hamilton–Jacobi Equation
Up to now, we have considered the generalized action (2.62) or the cost func-
tional (2.52) as the starting point for the application of the variational cal-
culus. The central aim of our previous analysis was the determination of the
optimum trajectory and the optimum control. But sometimes it is necessary
to know the value of the functional along the optimum trajectory. Of course,
one can compute functional (2.52) directly from the optimal curve X
∗
(t)and
the optimum control u
∗
(t). We will derive an alternative way which allows us
to determine the performance functional without the knowledge of X
∗
and
8
Note that we now have 4dM insteadof2dM differential equations for the evolu-
tion of the model system in a d-dimensional space.
56 2 Deterministic Control Theory
u
∗
. First, the performance functional (2.52) and the generalized action (2.62)
are identical at the optimum curve,
S[X
∗
,P
∗
,u
∗
,T,Λ
∗
, Λ
∗
]=R[X
∗
,u
∗
,T] , (2.128)
because the optimal solution satisfies both the boundary conditions (2.55)and
constraints (2.53). Especially the Lagrangian (2.63) simply becomes
L
t, X
∗
(t),
˙
X
∗
(t),P
∗
(t),u
∗
(t)
= φ(t, X
∗
(t),u
∗
(t)) . (2.129)
On the other hand, S[X
∗
,P
∗
,u
∗
,T,Λ
∗
, Λ
∗
] is simply a function of the bound-
ary conditions and the time T , i.e., we may write S[X
∗
,P
∗
,u
∗
,T,Λ
∗
, Λ
∗
]=
S(X
0
,X
e
,T). We emphasize again that S(X
0
,X
e
,T) means here the optimum
action. Let us now determine the change in S(X
0
,X
e
,T) for a small change in
the final boundary conditions, X
e
→ X
e
+ δX
e
. The change in the boundary
conditions also changes the optimal trajectory. Formally, we obtain
δS(X
0
,X
e
,T)=S(X
0
,X
e
+ δX
e
,T) − S(X
0
,X
e
,T)
=
T
0
dt
∂L
∗
∂X
∗
δX
∗
+
T
0
dt
∂L
∗
∂
˙
X
∗
δ
˙
X
∗
+
T
0
dt
∂L
∗
∂P
∗
δP
∗
+
T
0
dt
∂L
∗
∂u
∗
δu
∗
(2.130)
with δX
∗
, δ
˙
X
∗
, δP
∗
,andδu
∗
being the changes of the optimal trajectories of
the state, the momenta, and the control due to the change in the boundary
condition. The boundary terms, b(X
0
)Λ and b(X
e
)Λ, does not contribute to
the change in the action because the initial conditions satisfy optimal curves
b(X
0
) = 0 and the change in the final boundary condition (which satisfies
b(X
e
) = 0) implies a change in the functional structure b → b
with b
(X
e
+
δX
e
) = 0. In other words, all boundary terms are separately canceled in
S(X
0
,X
e
,T)aswellasinS(X
0
,X
e
+ δX
e
,T). The second term in (2.130)is
now integrated by parts. Considering (2.75), (2.74), and (2.69), we arrive at
δS(X
0
,X
e
,T)=
∂L
∗
∂
˙
X
∗
δX
∗
T
0
+
T
0
dt
∂L
∗
∂X
∗
−
d
dt
∂L
∗
∂
˙
X
∗
δX
∗
=
∂L
∗
∂
˙
X
∗
t=0
δX
0
+
∂L
∗
∂
˙
X
∗
t=T
δX
e
(2.131)
and therefore with (2.88)andδX
0
=0
δS(X
0
,X
e
,T)=P
∗
e
δX
e
. (2.132)
We conclude from the relation that
P
∗
e
=
∂S(X
0
,X
e
,T)
∂X
e
. (2.133)
2.5 The Hamilton–Jacobi Equation 57
On the other hand, the functional structure (2.62) implies the relation
dS(X
0
,X
e
,T)
dT
= L
T,X
e
,
˙
X
∗
e
,P
∗
e
,u
∗
e
. (2.134)
The total derivative may also be written as
dS(X
0
,X
e
,T)
dT
=
∂S(X
0
,X
e
,T)
∂T
+
∂S(X
0
,X
e
,T)
∂X
e
˙
X
∗
e
=
∂S(X
0
,X
e
,T)
∂T
+ P
∗
e
˙
X
∗
e
.
Thus, we obtain
∂S(X
0
,X
e
,T)
∂T
= L
T,X
e
,
˙
X
∗
e
,P
∗
e
,u
∗
e
− P
∗
e
˙
X
∗
e
= −H (T,X
e
,P
∗
e
,u
∗
e
) . (2.135)
The optimum control u
∗
e
can be substituted by the preoptimized control, u
∗
e
=
u
(∗)
e
(T,X
e
,P
∗
e
). Finally, we replace the momentum P
∗
e
by (2.133)and
∂S
∂T
+ H
T,X
e
,
∂S
∂X
e
,u
(∗)
e
T,X
e
,
∂S
∂X
e
=0. (2.136)
This nonlinear first-order partial differential equation defines the action
S = S(X
0
,X
e
,T). Equation (2.136) is called the Hamilton–Jacobi equation.
In principle, (2.136) solves the above-introduced problem. Unfortunately, the
general solution of a partial differential equation of first-order depends on ar-
bitrary functions. The specific structure of these functions is usually fixed by
suitable boundary conditions.
For many applications in optimal control theory, the knowledge of these
functions is secondary. The leading role is played by so-called complete in-
tegrals. In our case, this is a solution of (2.136) with N + 1 arbitrary, but
independent constants
9
. Since (2.136) contains only derivatives of S,oneof
these constants is additive. The general structure of a complete integral is
given by
S
comp
= f(T,X
e
,
P )+C
0
(2.137)
with the constants
P =(
P
1
,...,
P
N
)andC
0
. The condition that S
comp
con-
tains independent constants is det ∂
2
f/∂X
e
∂
P = 0. We remark that the gen-
eral solution can be obtained from the complete integrals by the construction
of the corresponding envelope. We now use f (T,X
e
,
P ) as the generating func-
tion for the canonical transformation
P =
∂f
∂X
e
X =
∂f
∂
P
H = H +
∂f
∂T
= H +
∂S
comp
∂T
, (2.138)
9
The number of independent constants in a complete integral is equivalent to the
number of independent variables. In the present case, we have N state variables
X
1
,X
2
,...,X
N
and the time T .
58 2 Deterministic Control Theory
where the new
H satisfy the canonical equations (2.92) with the new coordi-
nates
X and the new momenta
P . On the other hand we get
H = 0 because
S
comp
is a solution of (2.136). Thus we obtain d
X/dt =d
P/dt = 0 and there-
fore
X =const. Hence, the solution of
X = ∂f/∂
P with respect to the final
state is a function X
e
(T,
X,
P ) of the time T and 2N independent constants.
On the other hand, the trajectory of X
e
is identical to the optimum path
10
of
the system X
∗
. Therefore, X
e
(t,
X,
P ) is a general solution of the optimum
problem. The constants
X and
P may be used to fix the initial and final
boundaries of this solution.
In this sense we may reformulate the concept of the Hamilton–Jacobi the-
ory: a complete integral S(t, X,
P )+C
0
, which satisfies the Hamilton–Jacobi
equation
∂S
∂T
+ H
t, X,
∂S
∂X
,u
(∗)
t, X,
∂S
∂X
=0, (2.139)
allows us to construct the general solution X
∗
(t,
P,
X) of the system of canon-
ical equations (2.92) by solving the algebraic equations
X = ∂S(t, X,
P )/∂
P
for X and to determine the open parameters
X and
P in such a way that the
boundary conditions are fulfilled.
Let us finally demonstrate this concept for a very simple example which
concerns the performance functional of a free particle
R =
T
0
u
2
2
dt → inf (2.140)
for the simple constraint ˙x = u, u ∈ (−∞, +∞) and the boundary conditions
x(0) = x
0
and x(T )=x
e
. We obtain the Hamiltonian H = qu−u
2
/2 with the
generalized momentum q. The preoptimal control is simply u
(∗)
= q and the
preoptimized Hamiltonian is H = q
2
/2. This leads to the Hamilton–Jacobi
equation
∂S
∂t
+
1
2
∂S
∂x
2
=0, (2.141)
and the separation of variables leads to the complete integral S = −c
2
t/2+cx
and therefore to ∂S/∂c = x − ct = b with the free constants b and c.The
boundary conditions require b = x
0
and c =(x
e
− x
0
)/T and the optimal
solution is x
∗
= x
0
+(x
e
− x
0
)(t/T ).
10
This statement is a direct consequence of the time-local structure of the Hamil-
tonian and the Lagrangian, which means the sequence of the optimum trajectory
from X
0
to X
1
and the subsequent optimum trajectory from X
1
to X
e
yields the
optimum trajectory from X
0
to X
e
.
References 59
References
1. G. Galilei: Dialogues concerning two New Sciences, translated by H. Crew, A.
de Salvio (Prometheus Books, Buffalo, NY, 1998) 17
2. P. Costabel, J. Peiffer: Die Gesammelten Werke der Mathematiker und Physiker
der Familie Bernoulli (Birkh¨auser, Basel, 1988) 17, 18
3. B. Singh, R. Kumar: Indian J. Pure Appl. Math. 19, 575 (1988) 18
4. T. Koetsier: The story of the creation of the calculus of variations : the contri-
butions of Jakob Bernoulli, Johann Bernoulli and Leonhard Euler (Dutch), in
1985 Holiday Course : Calculus of Variations,Eds.A.M.H.Gerards,J.W.
Klop (CWI, Amsterdam, 1985), 1–25.
5. A.D. Ioffe, V.M. Tihomirov: Theory Extremal Problems (North-Holland, Ams-
terdam, 1979) 24
6. R. Bulirsch, A. Miele, J. Stoer, K. Well: Optimal Control (Birkh¨auser, Basel,
1998) 24
7. D.A. Carlson, A.B. Haurie, A. Leizarowitz: Infinite Horizon Optimal Control
(Springer, Berlin Heidelberg New York, 1991) 24
8. P. Whittle: Optimal Control: Basics and Beyond (Wiley, Chichester, 1996) 24
9. J.H. Davis: Foundations of Deterministic and stochastic Control (Birkh¨auser,
Boston, 2002) 24
10. V.I. Arnold: Mathematical Methods of Classical Mechanics (Springer, Berlin
Heidelberg New York, 1989) 24
11. T.W. Kibble: Classical Mechanics (Imperial College Press, London, 2004) 24
12. M.G. Calkin: Lagrangian and Hamiltonian Mechanics (World Scientific Publish-
ing, Singapore, 1997) 24
13. G.R. Fowles: Analytical Mechanics (Brooks/Cole Publishing Co., Pacific Grove,
1998) 30
14. D. Kleppner: An Introduction to Mechanics (McGraw-Hill, New York, 1973) 30
15. T.L. Chow: Classical Mechanics (Wiley, Chichester, 1995) 30
16. K.H. Hoffmann, I. Lasiecka, G. Leugering, J. Sprekels, F. Tr¨oltzsch: Optimal
Control of Complex Structures. International Series of Numerical Mathematics,
vol. 139 (Birkh¨auser, Basel, 2001) 35
17. A. Strauss: An Introduction to Optimal Control Theory (Springer, Berlin Hei-
delberg New York, 1968) 35
18. D.A. Carlson, A.B. Haurie, A. Leizarowitz: Infinite Horizon Optimal Control
(Springer, Berlin Heidelberg New York, 1991) 35
19. J. Borggaard, J. Burkhardt, M. Gunzburger: Optimal Design and Control
(Birkh¨auser, Basel, 1995) 35
20. L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze, E.F. Mishchenko: The
Mathematical Theory of Optimal Processes (Interscience Publishers, New York,
1962) 42
21. A.E. Bryson, Y.C. Ho: Applied Optimal Control (Hemisphere Publishing Co.,
Washington, 1975) 45
22. M. Athans, P.L. Falb: Optimal Control (McGraw-Hill, New York, 1966) 45
23. G. Knowles: An Introduction to Applied Optimal Control (Academic, New York,
1981) 45
24. D.J. Bell, D.H. Jacobson: Singular Optimal Control Problems (Academic, New
York, 1975) 45
25. R. Burlisch, D. Kraft: Computational Optimal Control (Birkh¨auser, Basel, 1994) 45
60 2 Deterministic Control Theory
26. J. Gregory: Constraint Optimization in the Calculus of Variations and Optimal
Control Theory (Van Nostrand Reinhold, New York, 1992) 45
27. A. Locatelli: Optimal Control (Birkh¨auser, Basel, 2001) 50, 52
28. D. Zubarev, D. Zubarev, G. R¨opke: Statistical Mechanics of Nonequilibrium
Processes: Basic Concepts, Kinetic Theory (Akademie-Verlag, Berlin, 1996) 51
29. R. Zwanzig: Nonequilibrium Statistical Mechanics (Oxford University Press, Ox-
ford, 2001) 51
30. G.F. Mazenko: Nonequilibrium Statistical Mechanics (Wiley, Chichester, 2005) 51
31. D.J. Evans, G.P. Morriss: Comp. Phys. Rep. 1, 297 (1984) 54
32. D.J. Evans, W.G. Hoover, B.H. Failor, B. Moran: Phys. Rev. A 28, 1016 (1983) 54
33. S. Nos´e: J. Chem. Phys. 81, 511 (1984) 54
34. W.G. Hoover: Phys. Rev. A 31, 1695 (1985) 54
35. S. Toxvaerd: Mol. Phys. 72, 159 (1991) 54
3
Linear Quadratic Problems
3.1 Introduction to Linear Quadratic Problems
3.1.1 Motivation
Suppose we have a deterministic system under control, described by dynami-
cal equations of motion for an N-dimensional state vector X(t). Usually, these
equations can be written as a set nonlinear first-order differential equations
(2.53) which are essentially influenced by the control function u(t). Further-
more, let us assume that we have obtained the optimum trajectory X
∗
(t)and
the optimum control u
∗
(t) by the methods described in the previous chapter.
We also denote X
∗
(t) as a nominal state of the system and u
∗
(t) as a nominal
input. Unfortunately, we must expect that unavoidable uncertainties in the
system description and disturbances are acting on the system, so that the real
trajectory X(t) shows some deviations from the optimal trajectory X
∗
(t).
The determination X
∗
(t)andu
∗
(t) and the application of these solutions
on a real experiment or a real system may be interpreted as an open loop
control scheme as discussed in Sect. 1.1. This concept is sufficient as far as
the optimal trajectories and controls are stable against disturbances. But it
may be possible that a small deviation Y (t)=X(t) − X
∗
(t) of the system
state from the optimum trajectory decreases rapidly and the system becomes
unstable in comparison to the desired nominal state. In this case, it seems
rather reasonable to steer against a small deviation Y (t) by a small correction
w(t)=u(t) − u
∗
(t). This can be done by a controller which measures the
deviation Y (t) of the current state from the nominal state and supplies the
evolution of the system by the correction w(t) in order to make Y (t) small
(Fig. 3.1). If the deviations are sufficiently small, the effect Y (t)andw(t)
can be evaluated through the linearized equations of motion, following from
a series expansion of the usually nonlinear equations of motion (2.53)
˙
Y (t)=
∂F (X
∗
,u
∗
,t)
∂X
∗
Y (t)+
∂F (X
∗
,u
∗
,t)
∂u
∗
w(t) . (3.1)
M. Schulz: Control Theory in Physics and other Fields of Science
STMP 215, 61–92 (2006)
c
Springer-Verlag Berlin Heidelberg 2006
62 3 Linear Quadratic Problems
X
n
system
u
n
Y
w
u
X
control unit
nominal system
Fig. 3.1. The formal relations between a system under control, the nominal system
and the controller
This system of linear differential equations with possibly time-dependent co-
efficients defines the linear response of the system on an arbitrary but small
change of the control function u(t) against the optimum control u
∗
(t). Al-
though the evolution of the linearized system can be completely determined
from (3.1), the corresponding initial conditions and a certain control aim which
is still to be defined, the general situation described here is rather a closed
loop control than an open loop scheme.
3.1.2 The Performance Functional
The control problem for the above-introduced linear system of equations of
motion must be completed by the performances functional in order to de-
clare the control aim. Here we will give a heuristic motivation for the typical
structure of these important quantity. We start from the original performance
functional (2.52). The minimization of this functional together with the corre-
sponding constraints and boundary conditions supplied the optimal trajectory
X
∗
(t) and the optimum control u
∗
(t). Thus, we obtain
T
0
dtφ(t, X
∗
(t),u
∗
(t)) ≤
T
0
dtφ(t, X
(t),u
(t)) , (3.2)
where X
(t)andu
(t) are trajectories and controls which strictly satisfy
the constraints, i.e., the evolution equations of the underlying system and
the boundary conditions. We substitute X
(t)=X
∗
(t)+Y (t)andu
(t)=
u
∗
(t)+w(t) and consider that Y (t)andw(t) are small deviations which can
be described by the linearized equations of motion (3.1). An expansion of the
performance functional in terms of Y and u up to the second-order leads to
0 ≤
T
0
dt
∂φ
∗
∂X
∗
Y (t)+
T
0
dt
∂φ
∗
∂u
∗
w(t)+
1
2
T
0
dtY (t)
∂
2
φ
∗
∂X
∗2
Y (t)
3.1 Introduction to Linear Quadratic Problems 63
+
T
0
dtY (t)
∂
2
φ
∗
∂X
∗
∂u
∗
w(t)+
1
2
T
0
dtw(t)
∂
2
φ
∗
∂u
∗2
w(t) (3.3)
with φ
∗
= φ(t, X
∗
,u
∗
) . The linear terms disappear
1
and it remains only an
inequality for a squared form. This squared form is often estimated by the
stronger inequality
T
0
dtY (t)
∂
2
φ
∗
∂X
∗2
Y (t)+
T
0
dtw(t)
∂
2
φ
∗
∂u
∗2
w(t) ≥ 0 (3.4)
so that we obtain a performance functional of a quadratic type
J[Y, w]=
1
2
T
0
dt
Y (t)Q(t)Y (t)+
T
0
dtw(t)R(t)w(t)
. (3.5)
The matrix functions Q(t)andR(t) can be identified with the second-order
derivatives in (3.4). We remark that for the most applications and especially
for sufficiently complicated systems the rigorous derivation of Q(t)andR(t)
is often replaced by a more or less empirically chosen quantities. The only
condition one must take into account is that Q(t)andR(t) are symmetric
and positive definite matrices. In this case, functional (3.5) has only one global
minimum which is reached for Y = w = 0 when the constraints are neglected.
However, the minimization of (3.5) under consideration of the equations of
motion (3.1) still guarantees the smallest possible corrections Y (t)andw(t)
to a nominal state X
∗
(t) and a nominate control u
∗
(t).
The minimization of the quadratic performance functional (3.5) under the
constraint of linear evolution equations (3.1) is called a linear quadratic prob-
lem. As we have illustrated, such problems arise in a fairly spontaneous and
natural way. In principle, the linear quadratic problem is again a determinis-
tic optimum control problem not for the optimal trajectory and the optimal
control, but for the deviations from the optimum solution. Linear quadratic
problems have a broad spreading in natural, technological, and economic sci-
ences.
3.1.3 Stability Analysis
Linear Stability Analysis
First of all, we will check whether a further control of a system close to
the nominal state is actually necessary. To this aim, we consider a possible
1
It should be remarked that the linear terms vanish only for such Y (t)andw(t)
which satisfy the constraints. Therefore, we have
dt (∂φ
∗
/∂X
∗
) Y (t)=0and
dt (∂φ
∗
/∂u
∗
) w(t) = 0 but not necessarily ∂φ
∗
/∂X
∗
= 0 and ∂φ
∗
/∂u
∗
=0.
The latter would correspond to the variational derivative of the performance
functional, where all admissible functions, Y (t)andw(t), are taken into account
independently from their constraints.
64 3 Linear Quadratic Problems
deviation Y (t) of the current state X(t) against the optimum state X
∗
(t),
but we allow no additional control as the optimum control u
∗
(t), i.e., we have
w(t) = 0. For the further investigations we may write the original nonlinear
system of evolution equations (2.53) in the form
˙
Y (t)=F (X
∗
(t)+Y (t),u
∗
(t),t) − F (X
∗
(t),u
∗
(t),t)
= H (Y (t),t) . (3.6)
If this equation approaches the optimum state, i.e., Y (t) → 0fort →∞,a
further control is not necessary for stabilizing the system. Such a system is
called self-stabilized. Otherwise, if an initially small deviation becomes suffi-
ciently large or it diverges in the long time limit, the system is unstable and
needs definitely an additional control.
An apparently simple case occurs for autonomous differential equations
which depend not explicitly on time. In this case, we have the explicitly time-
independent version of the evolution equations (3.6):
˙
Y = H (Y ) (3.7)
with a (stable or instable) fixed point Y = 0 in the N-dimensional phase
space. The linear stability analysis requires the linearized evolution equation
˙
Y = AY , (3.8)
where the matrix A has the components
A
αβ
=
∂H
α
(Y )
∂Y
β
Y =0
with α, β =1,...,N . (3.9)
A standard method for the characterization of the stability of (3.7) is analysis
of the linear equation (3.8). In particular, the equation is called stable with
respect to a fixed point Y = 0 if the real part of all eigen-values of A is
negative.
Resonances
The linear stability analysis does not always yield sufficient indications for
the stability or instability of a given system. In order to avoid misleading
conclusions with respect to the results of the linear stability analysis, we
expand H (Y ) in terms of a Taylor series with respect to Y . Then we can
write (3.7)intheform
∂Y
∂t
= AY + ψ
(r)
(Y ) . (3.10)
The rest function is given by ψ
(r)
(Y )=H (Y ) − AY . The leading term of
the function ψ
(r)
(Y ) is of an order of magnitude |Y |
r
with r ≥ 2. Let us now
introduce a transformation z = Y + h(Y ) where h is a vector polynomial with
the leading order 2 so that h(0) = ∂h/∂Y |
Y =0
= 0. Thus we obtain