Haddad W.M. Nonlinear Dynamical Systems and Control: A Lyapunov-Based Approach

Подождите немного. Документ загружается.

NonlinearBook10pt November 20, 2007

Chapter Fourteen

Discrete-Time Optimal Nonlinear

Feedback Control

14.1 Introduction

Since most physical processes evolve naturally in continuous time, it is not

surprising that the bulk of nonlinear control theory has been developed

for continuous-time systems. Nevertheless, it is the overwhelming trend

to implement controllers digitally. Despite this fact, the development of

nonlinear control theory for discrete-time systems has lagged its continuous-

time counterpart. This is in part due to the fact that concepts s uch as

zero dynamics, normal forms, and minimum phase are much more intricate

for discrete-time s ys tems. For example, in contrast to the continuous-time

case, technicalities involving passivity analysis to ols needed to prove global

stability via smooth feedback controllers [282] as well as system relative

degree requirements [78] are more involved in the discrete-time case.

In this chapter, we extend the framework developed in Chapters

8–11 to address the problem of optimal discrete-time nonlinear analysis

and feedback control with nonlinear-nonquadratic performance criteria.

Speciﬁcally, we consider discrete-time autonomous nonlinear regulation

in feedback control problems on an inﬁnite horizon involving nonlinear-

nonquadratic performance functionals. As in the continuous-time case,

the performance functional can be evaluated in closed form as long as the

nonlinear-nonquadratic cost functional considered is related in a speciﬁc way

to an underlying Lyapunov function that guarantees asymptotic stability

of the n on linear closed-loop system. This Lyapunov function is shown to

be the s olution of the discrete-time steady-state Bellman equation arising

from the principle of optimality in dynamic programming and plays a

key role in constructing the optimal non linear control law. The overall

framework provides the foundation for extending discrete-time, linear-

quadratic synthesis to nonlinear-nonquadratic pr oblems.

NonlinearBook10pt November 20, 2007

846 CHAPTER 14

14.2 Optimal Control and the Bellman Equation

In this section, we consider a discrete-time control problem involving

a notion of optimality with respect to a nonlinear-nonquadratic cost

functional. Speciﬁcally, we consider the following optimal control problem.

Optimal Control Problem. Consider the nonlinear controlled

system given by

x(k + 1) = F (x(k), u(k), k), x(k

) = x

, x(k

) = x

, u(k) ∈ U, k ≥ k

(14.1)

where x(k) ∈ D ⊆ R

, k ∈ Z

, is th e state vector, D is an open set with

0 ∈ D, u(k) ∈ U ⊆ R

, k ∈ Z

, is the control input, x(k

) = x

is given,

x(k

) = x

is ﬁxed, and F : D × U × R → R

satisﬁes F (0, 0, ·) = 0. We

assume that u(·) is restricted to the class of admissible controls U consisting

of measurable functions u(·) such that u(k) ∈ U for all k ∈ Z

, where the

constraint set U is given with 0 ∈ U . Furthermore, we assume that F (·, ·, ·)

is continuous. Then determine the control inpu t u(k) ∈ U, k ∈ [k

, k

], such

that the cost functional

J(x

, u(·), k

) =

k=k

L(x(k), u(k), k), (14.2)

is minimized, where L : D × U × R → R is given.

To solve the optimal control problem we present Bellman’s principle

of optimality which provides necessary and suﬃcient conditions, f or a given

control u(k) ∈ U, k ∈ Z

, to minimize the cost functional (14.2).

Lemma 14.1. Let u

∗

(·) ∈ U be an optimal control that generates the

trajectory x(k), k ∈ [k

, k

], with x(k

) = x

. Then the trajectory x(·)

from (k

, x

) to (k

, x

) is optimal if and only if for all k

, k

∈ [k

, k

], the

portion of the trajectory x(·) going from (k

, x(k

)) to (k

, x(k

)) optimizes

the same cost functional over [k

, k

], where x(k

) = x

is a point on the

optimal trajectory generated by u

∗

(·).

Proof. Let u

∗

(·) ∈ U solve the optimal control problem and let x(k),

k ∈ [k

, k

], be the solution to (14.1) generated by u

∗

(·). Next, suppose, ad

absurdum, that there exist k

≥ k

, k

≤ k

, and ˆu(k), k ∈ [k

, k

], such that

k=k

L(ˆx(k), ˆu(k), k) <

k=k

L(x(k), u

∗

(k), k),

where ˆx(k) solves (14.1) for all k ∈ [k

, k

] with u(k) = ˆu(k), ˆx(k

) = x(k

NonlinearBook10pt November 20, 2007

DISCRETE-TIME NONLINEAR CONTROL 847

and ˆx(k

) = x(k

). Now, deﬁne

(k)

△







∗

(k), k ∈ [k

, k

ˆu(k), k ∈ [k

, k

∗

(k), k ∈ (k

, k

Then,

J(x

, u

(·), k

)

k=k

L(x(k), u

(k), k)

k=k

L(x(k), u

∗

(k), k) +

k=k

L(ˆx(k), ˆu(k), k) +

k=k

L(x(k), u

∗

(k), k)

k=k

L(x(k), u

∗

(k), k) +

k=k

L(x(k), u

∗

(k), k) +

k=k

L(x(k), u

∗

(k), k)

= J(x

, u(·), k

which is a contradiction.

Conversely, if u

∗

(·) minimizes J(·, ·, ·) over [k

, k

] for all k

≥ k

and

≤ k

, then it minimizes J(·, ·, ·) over [k

, k

Lemma 14.1 states that u

∗

(·) solves th e optimal control pr oblem over

the time interval [k

, k

] if and only if u

∗

(·) solves the optimal control

problem over every subset of the time interval [k

, k

]. Next, let u

∗

(k),

k ∈ [k

, k

], solve the optimal control problem and deﬁne the optimal cost

∗

, k

)

△

= J(x

, u

∗

(·), k

). Furthermore, deﬁne, for p : R

×Z

→ [0, ∞),

the Hamiltonian H(x, u, p(x, k), k)

△

= L(x, u, k)+p(F (x, u, k), k+1)−p(x, k).

With these deﬁn itions we have the following result.

Theorem 14.1. Let J

∗

(x, k) denote the min imal cost for the optimal

control problem with x

= x and k

= k. T hen

0 = min

u(k)∈U

H(x(k), u(k), J

∗

(x(k), k), k). (14.3)

Furth ermore, if u

∗

(·) solves th e optimal control pr oblem, then

0 = H(x(k), u

∗

(k), J

∗

(x(k), k), k). (14.4)

Proof. It f ollows from Lemma 14.1 that, for all k

≥ k + 1,

∗

(x(k), k) = min

u(·)∈U

i=k

L(x(i), u(i), i)

NonlinearBook10pt November 20, 2007

848 CHAPTER 14

= min

u(·)∈U

(

L(x(k), u(k), k) +

i=k+1

L(x(i), u(i), i)

)

= min

u(k)∈U

{L(x(k), u(k), k) + J

∗

(x(k + 1), k + 1)},

or, equivalently,

0 = min

u(k)∈U

{(J

∗

(x(k + 1), k + 1) −J

∗

(x(k), k)) + L(x(k), u(k), k)},

which yields (14.3). Finally, (14.4) can be proved in a similar manner by

replacing u(·) with u

∗

(·), where u

∗

(·) is the optimal control.

Next, we provide the converse result to Theorem 14.1.

Theorem 14.2. Suppose there exist a continuous function V : D×R →

R and an optimal control u

∗

(·) such that V (x(k

+ 1), k

+ 1) = 0,

0 = H(x, u

∗

(k), V (x(k), k), k), x ∈ D, k ∈ Z

, (14.5)

and

H(x, u

∗

(k), V (x(k), k), k) ≤ H(x, u(k), V (x(k), k), k),

x ∈ D, u(k) ∈ U, k ∈ Z

. (14.6)

Then u

∗

(·) solves the optimal control problem, that is,

∗

, k

) = J(x

, u

∗

(·), k

) ≤ J(x

, u(·), k

), u(·) ∈ U, (14.7)

and

∗

, k

) = V (x

, k

). (14.8)

Proof. Let x(k), k ∈ Z

, satisfy (14.1) and, for all k ∈ [k

, k

+ 1],

deﬁne

∆V (x(k), k)

△

= V (F (x(k), u(k)), k + 1), k) − V (x(k), k). (14.9)

Then, with u(·) = u

∗

(·), it follows from (14.5) that

0 = ∆V (x(k), k) + L(x(k), u

∗

(k), k).

Now, summing over [k

, k

] and noting that V (x(k

+ 1), k

+ 1) = 0 yields

V (x

, k

) =

k=k

L(x(k), u

∗

(k), k) = J(x

, u

∗

(·), k

) = J

∗

, k

Next, for all u(·) ∈ U it follows from (14.5) and (14.9) that

J(x

, u(·), k) =

k=k

L(x(k), u(k), k)

NonlinearBook10pt November 20, 2007

DISCRETE-TIME NONLINEAR CONTROL 849

k=k

{−∆V (x(k), k) + L(x(k), u(k), k)

+V (F (x(k), u(k), k), k) − V (x(k), k)}

k=k

{−∆V (x(k), k) + H(x(k), u(k), V (x(k), k), k)}

≥

k=k

{−∆V (x(k), k) + H(x(k), u

∗

(k), V (x(k), k), k)}

k=k

−∆V (x(k), k)

= V (x

, k

)

= J

∗

, k

which completes the proof.

Note that (14.5) and (14.6) imply

0 = min

u(k)∈U

H(x(k), u(k), V (x(k), k), k), (14.10)

which is known as the Bellman equation. It follows from Theorems 14.1 and

14.2 that the Bellman equation pr ovides necessary and suﬃcient conditions

for characterizing the optimal control for time-varying nonlinear dynamical

systems over a ﬁnite time interval or the inﬁnite horizon. I n the inﬁnite-

horizon, time-invariant case, V (·) is independent of k so that the Bellman

equation reduces to the time-invariant equation

0 = min

u∈U

H(x, u, V (x)), x ∈ D. (14.11)

14.3 Stability Analysis of Discrete-Time No n linear Systems

In this section, we present suﬃcient conditions for stability of nonlinear

discrete-time systems. In particular, we consider the problem of evaluating a

nonlinear-nonquadratic performance functional depending upon a nonlinear

discrete-time diﬀerence equation. As in the continuous-time case, it is shown

that the cost functional can be evaluated in closed form as long as the cost

functional is related in a speciﬁc way to an und erlying Lyapunov function

that guarantees stability. Here, we restrict our attention to time-invariant

inﬁnite horizon systems. For the following result, let D ⊂ R

be an open

set, assume 0 ∈ D, let L : D → R, and let f : D → D be s uch that f (0) = 0.

Theorem 14.3. Consider the nonlinear discrete-time dynamical sys-

NonlinearBook10pt November 20, 2007

850 CHAPTER 14

tem

x(k + 1) = f(x(k)), x(0) = x

, k ∈ Z

, (14.12)

with nonlinear-nonquadratic performance functional

J(x

)

△

∞

k=0

L(x(k)). (14.13)

Furth ermore, assume there exists a continuous function V : D → R such

that

V (0) = 0, (14.14)

V (x) > 0, x ∈ D, x 6= 0, (14.15)

V (f (x)) −V (x) < 0, x ∈ D, x 6= 0, (14.16)

L(x) + V (f(x)) − V (x) = 0, x ∈ D. (14.17)

Then then the zero solution x(k) ≡ 0 to (14.12) is a locally asymptotically

stable and there exists a neighborhood of the origin D

⊆ D such that

J(x

) = V (x

), x

∈ D

. (14.18)

Finally, if D = R

and

V (x) → ∞ as kxk → ∞, (14.19)

then the zero solution x(k) ≡ 0 to (14.12) is globally asymptotically stable.

Proof. Let x(k), k ∈ Z

, satisfy (14.12). Then it follows from (14.16)

that

∆V (x(k)) = V (f(x(k)) − V (x(k)) < 0, k ∈ Z

, x(k) 6= 0. (14.20)

Thus , from (14.14), (14.15), and (14.20) it follows that V (·) is a Lyapunov

function for (14.12), which proves local asymp totic stability of the zero

solution x(k) ≡ 0 to (14.12). Consequently, x(k) → 0 as k → ∞ for all

initial conditions x

∈ D

for some neighborhood of the origin D

⊆ D.

Now, since

0 = −∆V (x(k)) + V (f (x(k))) − V (x(k)), k ∈ Z

it follows fr om (14.17) that

L(x(k)) = −∆V (x(k)) + L(x(k)) + V (f(x(k))) − V (x(k)) = −∆V (x(k)).

Now, summing over [0, N] yields

k=0

L(x(k)) = −V (x(N)) + V (x

Letting N → ∞ and noting that V (x(N)) → 0 for all x

∈ D

yields

J(x

) = V (x

). Finally, for D = R

global asymptotic stability is a direct

consequence of the radially unbounded condition (14.19) on V (x).

NonlinearBook10pt November 20, 2007

DISCRETE-TIME NONLINEAR CONTROL 851

Note that if (14.17) holds, then (14.16) is equivalent to L(x) > 0, x ∈

D, x 6= 0. Theorem 14.3 is the discrete-time analog to Theorem 8.1. The key

feature of Th eorem 14.3 is that it provides suﬃcient conditions for stability of

discrete-time nonlinear systems. Furthermore, th e nonlinear-nonquadratic

performance functional is given in terms of an underlying Lyapunov function

that guarantees asymptotic stability.

The following corollary specializes Theorem 14.3 to discrete-time linear

systems.

Corollary 14.1. Let A ∈ R

n×n

and R ∈ P

. Consider the linear sy s tem

x(k + 1) = Ax(k), x(0) = x

, k ∈ Z

, (14.21)

with performance functional

J(x

)

△

∞

k=0

(k)Rx(k). (14.22)

Furth ermore, assume there exists P ∈ P

such that

P = A

P A + R. (14.23)

Then the zero solution x(k) ≡ 0 to (14.21) is globally asymptotically stable

and

J(x

) = x

P x

. (14.24)

Proof. The result is a direct consequence of Theorem 14.3 with

f(x) = Ax, L(x) = x

Rx, V (x) = x

P x, and D = R

. Speciﬁcally,

conditions (14.14) and (14.15) are trivially satisﬁed. Now, V (f(x))−V (x) =

P A − P )x, and hence, it follows from (14.23) th at L(x) + V (f(x)) −

V (x) = 0 so that all the conditions of Theorem 14.3 are satisﬁed. Finally,

since V (x) is radially unbounded, the zero solution x(k) ≡ 0 to (14.21) is

globally asymptotically stable.

It follows from Corollary 14.1 that T heorem 14.3 is an extension of the

discrete-time H

analysis framework to nonlinear systems. Recall that the

Hardy space consists of complex matrix-valued functions G(z) ∈ C

l×m

that are analytic outside the unit disk and satisfy

sup

η>0

2π

−π

kG(e

η+θ

dθ < ∞. (14.25)

The norm of an H

function G(z) is deﬁned by

|||G|||

△



2π

−π

kG(e

θ

dθ



1/2

. (14.26)

NonlinearBook10pt November 20, 2007

852 CHAPTER 14

Alternatively, usin g Parseval’s theorem (see Problem 13.24) we can ex-

press the H

norm as an ℓ

norm of the impulse response H(k) =

2π

−π

G(e

θ

θk

dθ. In particular, the ℓ

norm of the matrix-valued impulse

response function H(k) ∈ R

l×m

, k ∈ Z

, is deﬁned by

|||H|||

△

∞

k=0

kH(k)k

1/2

. (14.27)

Now, letting R = E

E and deﬁning the free response z(k)

△

= Ex(k) =

, k ∈ Z

, it follows that the performance functional (14.22) can be

written as

J(x

) =

∞

k=0

(k)z(k)

∞

k=0

= x

P x

∞

k=0

kH(k)k

2π

−π

kG(e

θ

dθ

= |||G|||

, (14.28)

where H(k) = EA

and

G(z) ∼



A x

E 0



Alternatively, assuming x

has an expected value V , that is,

E[x

] = V , where E denotes expectation, and letting V = DD

, it follows

that the averaged performance fun ctional is given by

E[J(x

)] = E[x

P x

] = tr D

P D = |||G|||

, (14.29)

where P satisﬁes (14.23) and

G(z) ∼



A D

E 0



Next, we specialize Theorem 14.3 to linear and n onlinear systems with

multilinear cost functionals. The following lemma is needed. For this result

deﬁne the spectral radius of A ∈ R

n×n

ρ(A)

△

= max{|λ| : λ ∈ spec A}.

NonlinearBook10pt November 20, 2007

DISCRETE-TIME NONLINEAR CONTROL 853

We say that th e matrix A ∈ R

n×n

is Schur if and only if ρ(A) < 1.

Lemma 14.2. Let A ∈ R

n×n

be Schur and let h : R

→ R be a

q-multilinear function. Then there exists a unique q-multilinear function

g : R

→ R such that

0 = g(Ax) − g(x) + h(x), x ∈ R

. (14.30)

Furth ermore, if h(x), x ∈ R

, is nonnegative (respectively, positive) deﬁnite,

then g(x), x ∈ R

, is nonnegative (respectively, positive) deﬁnite.

Proof. Let h(x) = Ψx

[q]

and deﬁne g(x)

△

= Γx

[q]

, where Γ

△

= −Ψ(A

[q]

−

[q]

)

−1

. Note that A

[q]

− I

[q]

is invertible s ince A, and hence, A

[q]

is Schur.

Now, note that for all x ∈ R

g(Ax) − g(x) = Γ((Ax)

[q]

− x

[q]

)

= Γ(A

[k]

[q]

− x

[q]

)

= Γ(A

[q]

−I

[q]

= −Ψx

[q]

= −h(x).

To prove uniqueness, suppose that ˆg(x) =

Γx

[q]

also s atisﬁes (14.30).

Then it follows that

Γ(A

[q]

− I

[q]

Γ(A

[q]

− I

[q]

, x ∈ R

Since A

[q]

is Schur and (A

[q]

)

= (A

)

[q]

, it follows that, for all x ∈ R

Γx

[q]

= Γ(A

[q]

− I

[q]

)(A

[q]

− I

[q]

)

−1

[q]

= −Γ(A

[q]

− I

[q]

)

∞

j=1

[q]

)

[q]

= −Γ(A

[q]

− I

[q]

)

∞

j=1

)

[q]

= −

∞

j=1

Γ(A

[q]

−I

[q]

)(A

[q]

= −

∞

j=1

Γ(A

[q]

−I

[q]

)(A

[q]

Γx

[q]

which shows that g(x) = ˆg(x), x ∈ R

Finally, if h(x), x ∈ R

, is nonnegative d eﬁ nite, then it follows that,

NonlinearBook10pt November 20, 2007

854 CHAPTER 14

for all x ∈ R

g(x) = −Ψ(A

[q]

− I

[q]

)

−1

[q]

= Ψ

∞

j=1

[q]

)

[q]

= Ψ

∞

j=1

)

[q]

∞

j=1

Ψ(A

[q]

≥ 0.

If, in addition, h(x), x ∈ R

, is positive deﬁnite, then g(x), x ∈ R

, is

positive deﬁnite.

Next, assume A is Schur, let P be given by (14.23), and consider the

case in which L(·), f(·), and V (·) are given by

L(x) = x

Rx + h(x), (14.31)

f(x) = Ax + N(x), (14.32)

V (x) = x

P x + g(x), (14.33)

where h : D → R and g : D → R are nonquadratic and N : D → R

nonlinear. In this case, (14.17) holds if and only if

0 = x

Rx + h(x) + x

P Ax + N

(x)P N(x) + 2x

P N(x) − x

P x

+ g(Ax + N(x)) −g(x), x ∈ D, (14.34)

or, equivalently,

0 = x

P A −P + R)x + g(Ax + N(x)) − g(x) + h(x) + N

(x)P N(x)

+2x

P N(x), x ∈ D. (14.35)

If A is Schur, then we can choose P to satisfy (14.23) as in the linear-

quadratic case. Now, suppose N(x) ≡ 0 and let P satisfy (14.23). Then

(14.35) specializes to

0 = g(Ax) −g(x) + h(x), x ∈ D. (14.36)

Next, given h(·), we determine th e existence of a function g(·) satisfying

(14.36). To this end, we focus ou r attention on multilinear functions for

which (14.36) holds with D = R

Consider the linear system (14.21) and let h(x), x ∈ R

, be a positive-

deﬁnite q-multilinear function, where q is necessarily even. Furthermore, let

g(x), x ∈ R

, be the positive-deﬁnite q-multilinear function given by Lemma

14.2. Then, since g(Ax) −g(x) < 0, x ∈ R

, x 6= 0, it follows th at g(x) is a