Bhattacharya R., Majumdar M. Random Dynamical Systems: Theory and Applications
Подождите немного. Документ загружается.
6.5 Applications 393
Given any optimal policy ζ
∗
(stationary or not) and initial y ≥ 0
we refer to (x
˜
ζ
∗
, c
˜
ζ
∗
, y
˜
ζ
∗
) as an optimal input, consumption, and stock
process (from y), respectively. To simplify notation, we often drop the
superscript ζ
∗
.
A general monotonicity property is first noted. The proof uses the
strict concavity of u crucially.
Theorem 5.2 Assume [T.1]–[T.3] and [U.1]–[U.3]. Let i : S → Sbean
optimal investment policy function. Then i is non-decreasing, i.e., “y >
y
” implies “i(y) ≥ i (y
)”.
Proof. We prove the following:
Suppose y > y
and (x
t
) and (x
t
) are optimal input processes from y, y
,
respectively. Then x
0
≥ x
0
.
The theorem follows by taking x
0
= i(y) and x
0
= i(y
). Suppose,
to the contrary, that x
0
< x
0
. Define new input processes from y and
y
as follows. Let
¯
x
t
= x
t
for t ≥ 0. Then
¯
x
0
= x
0
≤ y
< y, and for
t ≥ 1,
¯
x
t
= x
t
≤ f
k
(x
t−1
) = f
k
(
¯
x
t−1
). Hence, (
¯
x
t
) is a feasible input pro-
cess from y. Next define
¯
x
t
= x
t
for t ≥ 0. Then
¯
x
0
= x
0
< x
0
≤ y
and
for t ≥ 1,
¯
x
t
= x
t
≤ f
k
(x
t−1
) = f
k
(
¯
x
t−1
). Hence (
¯
x
t
) is a feasible input
process from y
. Let (
¯
c
t
) and (
¯
c
t
) be consumption processes generated
by (
¯
x
t
) and (
¯
x
t
), respectively, (i.e.,
¯
c
0
= y −
¯
x
0
,
¯
c
t
= f
k
(
¯
x
t−1
) −
¯
x
t
for
t ≥ 1; similarly,
¯
c
0
= y
−
¯
x
0
,
¯
c
t
= f
k
(
¯
x
t−1
) −
¯
x
t
for t ≥ 1)). Using the
functional equation, we obtain
u(c
0
) + δ
N
k=1
V [ f
k
(x
0
)]q
k
= V (y) ≥ u(
¯
c
0
) + δ
N
k=1
V [ f
k
(
¯
x
0
)]q
k
.
(5.8)
u(c
0
) + δ
N
k=1
V [ f
k
(x
0
)]q
k
= V (y
) ≥ u(
¯
c
0
) + δ
N
k=1
V [ f
k
(
¯
x
0
)]q
k
.
(5.9)
Adding these two inequalities and noting that x
0
=
¯
x
0
, x
0
=
¯
x
0
, we obtain
u(c
0
) + u(c
0
) ≥ u(
¯
c
0
) + u(
¯
c
0
). (5.10)
Now,
¯
c
0
= y − x
0
> y
− x
0
= c
0
and
¯
c
0
= y − x
0
< y − x
0
= c
0
.
Hence, there is some θ ,0<θ<1 such that
¯
c
0
= θ c
0
+ (1 −θ)c
0
.
394 Discounted Dynamic Programming Under Uncertainty
Then,
¯
c
0
=y
− x
0
= (y − x
0
) + (y
− x
0
) − (y − x
0
)=c
0
+ c
0
−
¯
c
0
=
(1 − θ )c
0
+ θc
0
. This gives, using the strict concavity of u,
u(
¯
c
0
) >θu(c
0
) + (1 − θ)u(c
0
) (5.11)
and
u(
¯
c
0
) > (1 − θ)u(c
0
) + θ u(c
0
). (5.12)
So, by addition,
u(
¯
c
0
) + u(
¯
c
0
) > u(c
0
) + u(c
0
). (5.13)
Thus, (5.10) and (5.13) lead to a contradiction. Hence x
0
≥ x
0
.
Let us go back to the functional equation (5.7):
V (y) = I (ζ
∗
)(y) = max
a∈A
u[(1 −a)y] +δ
N
k=1
V [ f
k
(ay)]q
k
= u[y − ˆη(y)y] +δ
N
k=1
V [ f
k
(ˆη(y)y)]q
k
. (5.14)
For any y ≥ 0, let ψ (y) be the set of all actions a, where the right-hand
side of the first line in (5.14) attains its maximum. Then ψ (y) is an upper
semicontinuous correspondence from S into A, and ˆη is a selection from
ψ. Hence, as we remarked earlier (Remark 3.1), if ψ(y) is unique for
each y ≥ 0, then ˆη(y) = ψ(y) is a continuous function.
We now make the following assumption:
[T.4] Each f
k
∈ is strictly concave on R
+
.
Theorem 5.3 Assume [T.1]–[T.4] and [U.1]–[U.4]. Then
(a) If (x
˜
, c
˜
, y
˜
) and (x
˜
, c
˜
, y
˜
) are optimal (resp. input, consumption,
and stock) processes from the initial stock y ≥ 0, one has for t ≥ 0,
x
t
= x
t
c
t
= c
t
y
t
= y
t
a.s.
(b) The value function V is strictly concave.
(c) ˆη(y) is continuous on S.
6.5 Applications 395
Proof.
(a) Suppose that (x
˜
, y
˜
, c
˜
) and (x
˜
, y
˜
, c
˜
) are optimal processes
from the same initial stock y > 0. Then V (y) =
∞
t=0
δ
t
Eu(c
t
) =
∞
t=0
δ
t
Eu(c
t
). Define a new process (
¯
x
˜
,
¯
c
˜
,
¯
y
˜
) as follows:
Fix some θ ∈ (0, 1). Let
¯
x
t
≡ θ x
t
+ (1 −θ)x
t
for t ≥ 0;
¯
y
0
≡ y;
¯
y
t
= f
k
(
¯
x
t−1
) for t ≥ 1;
¯
c
t
=
¯
y
t
−
¯
x
t
for t ≥ 0. (5.15)
Clearly,
¯
x
t
≥ 0,
¯
y
t
≥ 0 for t ≥ 0. Using the strict concavity of f
k
,
¯
c
t
= f
k
(
¯
x
t−1
) −
¯
x
t
= f
k
[θ x
t−1
+ (1 −θ)x
t−1
] − [θ x
t
+ (1 −θ)x
t
]
≥ θ f
k
(x
t−1
) + (1 − θ) f
x
(x
t−1
) − [θ x
t
+ (1 −θ)x
t
]
= θ [ f
k
(x
t−1
) − x
t
] + (1 − θ)[ f
k
(x
t−1
) − x
t
]
= θ c
t
+ (1 −θ)c
t
. (5.16)
Hence (
¯
x
t
,
¯
c
t
,
¯
y
t
) is a well-defined (resp. input, consumption, and stock)
process from
¯
y
0
. It follows from the monotonicity of u that
Eu(
¯
c
t
) Eu(θc
t
+ (1 −θ)c
t
). (5.17)
If for some period τ ≥ 1, x
τ −1
is different from x
τ −1
on a set of strictly
positive probability, using the strict concavity of f
k
, the inequality in the
third line of (5.16) holds with strict inequality, and by [U.2], (5.17) holds
with strict inequality for t = τ . Hence,
∞
t=0
δ
t
Eu(
¯
c
t
) >
∞
t=0
δ
t
Eu(θc
t
+ (1 −θ)c
t
)
≥ θ
∞
t=0
δ
t
Eu(c
t
) + (1 − θ)
∞
t=0
δ
t
Eu(c
t
)
= θ V (y) + (1 − θ )V (y)
= V (y). (5.18)
This is a contradiction. Hence x
t
= x
t
a.s. for all t ≥ 0 and it follows that
y
t
= y
t
and c
t
= c
t
a.s. for all t ≥ 0.
396 Discounted Dynamic Programming Under Uncertainty
(b) We shall provide a sketch, since some of the details can be easily
provided by studying (a) carefully. Fix y > y
≥ 0. Let (x
˜
, c
˜
, y
˜
) and
(x
˜
, c
˜
, y
˜
) be optimal processes from y and y
, respectively. Fix θ ∈ (0, 1)
and define the process (
¯
x
˜
,
¯
c
˜
,
¯
y
˜
) as follows:
¯
x
t
= θ x
t
+ (1 −θ)x
t
for t ≥ 0;
¯
y = θ y + (1 −θ )y
;
¯
y
t
= f
k
(
¯
x
t−1
) for t ≥ 1;
¯
c
t
=
¯
y
t
−
¯
x
t
for t ≥ 0.
Show that (
¯
x
˜
,
¯
c
˜
,
¯
y
˜
) is a well-defined (resp. input, consumption, stock)
process from
¯
y = θ y + (1 − θ)y
, and, recalling the arguments leading
to (5.17),
Eu(
¯
c
t
) ≥ Eu(θc
t
+ (1 −θ)c
t
) for t ≥ 0. (5.19)
Then
V [θ y + (1 − θ )y
] = V [
¯
y] ≥
∞
t=0
δ
t
Eu(
¯
c
t
)
≥
∞
t=0
δ
t
Eu[θc
t
+ (1 −θ)c
t
]
≥ θ
∞
t=0
δ
t
Eu(c
t
) + (1 − θ)
∞
t=0
δ
t
Eu(c
t
)
= θ V (y) + (1 − θ )V (y
).
Hence, V is concave. Now if for some τ ≥ 0, c
τ
is different from c
τ
on
a set of positive probability, then using [U.3] we get
Eu(
¯
c
τ
) ≥ Eu(θc
τ
+ (1 −θ)c
τ
)
>θEu(c
τ
) + (1 − θ)Eu(c
τ
). (5.20)
If alternatively c
t
= c
t
for all t ≥ 0, since y > y
,wehavex
0
> x
0
. This
means, by [T.4] and [U.2], that
Eu(
¯
c
1
) > Eu(θc
1
+ (1 −θ)c
1
). (5.21)
Hence, we complete the proof of (b) by noting that
V (θ y + (1 − θ )y
) >θV (y) +(1 − θ)V (y
). (5.22)
6.5 Applications 397
(c) Going back to the right-hand side of (5.7), consider the optimiza-
tion problem:
max
a∈A
u[(1 −a)y] +δ
N
k=1
V [ f
k
(ay)]q
k
.
The assumed properties of u and f
k
and the derived properties of V
imply that the maximum is attained at a unique
¯
a. Hence the continuity
of ˆη.
A direct consequence of Theorems 5.2 and 5.3 is the following:
Corollary 5.1 Assume [T.1]–[T.4] and [U.1]–[U.3]. Then
(a) the investment policy function i(y):S → S is continuous and
nondecreasing and
(b) the consumption policy function c(y) is continuous.
6.5.2 Interior Optimal Processes
It is useful to identify conditions under which optimal processes are
interior. To this effect we introduce the following assumptions:
[T.5] f
k
(x) is continuously differentiable at x > 0, and lim
x↓0
f
k
(x) =
∞ for k = 1,...,N .
[U.4] u(c) is continuously differentiable at c > 0, and lim
c↓0
u
(c) =
∞.
One can derive some strong implications of the differentiability as-
sumptions.
Theorem 5.4 Under [T.1]–[T.5] and [U.1]–[U.4] if (x
˜
∗
, c
˜
∗
, y
˜
∗
) is an op-
timal process from some initial stock y > 0, then for all t ≥ 0,x
∗
t
(h
t
) > 0,
c
∗
t
(h
t
) > 0, and y
∗
t
(h
t
) > 0 for all histories h
t
.
Proof. First, we show that
i(y) > 0 for y > 0. (5.23)
If (5.23) holds, y
∗
1
= f
k
[i(y
∗
0
)] > 0 for k = 1, 2,...,N . Hence x
∗
1
=
i(y
∗
1
) > 0. Repeating this argument, we see that x
∗
t
(h
t
) > 0, y
∗
t
(h
t
) > 0
for all histories h
t
.
398 Discounted Dynamic Programming Under Uncertainty
Fix
¯
y > 0, and suppose that x
∗
0
= i(
¯
y) = 0. Then c
∗
0
=
¯
y. Let ε be
some positive number, ε<
¯
y. Define a process (x
˜
, c
˜
, y
˜
) as follows:
y
0
=
¯
y > 0, x
0
= ε, c
0
=
¯
y − ε>0,
y
1
= f
k
(ε), x
1
= 0, c
1
= f
k
(ε) = y
1
;
c
t
= x
t
= y
t
= 0 for t ≥ 2.
Clearly, (x
˜
, c
˜
, y
˜
) is a (well-defined) input, consumption, stock process
from
¯
y, and let I (
¯
y) be the expected total discounted utility of (x
˜
, c
˜
,
y
˜
). Let (x
˜
∗
, c
˜
∗
, y
˜
∗
) be the optimal process from
¯
y (with V (
¯
y) being its
expected total discounted utility). Then
0 ≤ V (
¯
y) − I (
¯
y) =
∞
t=0
δ
t
E[u(c
∗
t
) − u(c
t
)]
=
u(
¯
y) +δu(0)] − [u(
¯
y − ε) + δ
N
k=1
[u( f
k
(ε))]q
k
.
Hence,
N
k=1
[u( f
k
(ε))q
k
] − u(0) ≤
1
δ
[u(
¯
y) −u(
¯
y − ε)]. (5.24)
Let
u( f
k
1
(ε)) = min
k
u( f
k
(ε)).
Then,
N
k=1
[
u( f
k
(ε))q
k
]
u( f
k
1
(ε)). (5.25)
From (5.24) and (5.25)
u( f
k
1
(ε)) − u(0)
f
k
1
(ε)
·
f
k
1
(ε)
ε
1
δ
[u(
¯
y) −u(
¯
y − ε)]
ε
.
Letting ε ↓ 0 the left side goes to infinity and the right side goes to
1
δ
u
(
¯
y),
a contradiction. We now show that
c(y) > 0 for y > 0.
6.5 Applications 399
Suppose that c(
¯
y) = 0 for some
¯
y > 0. Then i(
¯
y) =
¯
y, and
V (
¯
y) = u(0) +δ
N
k=1
[
V ( f
k
(
¯
y))q
k
]
. (5.26)
Consider some ε satisfying 0 <ε<
¯
y. Let c
0
= ε and x
0
=
¯
y − ε. With
x
0
as the input in period 0, y
1
= f
k
(x
0
) = f
k
(
¯
y − ε) with probability q
k
>
0. Choose the optimal policy ζ
∗
from f
k
(
¯
y − ε) yielding V [ f
k
(
¯
y − ε)]
as the expected total discounted utility from f
k
(
¯
y − ε). Hence,
u(0) +δ
N
k=1
V [ f
k
(
¯
y)]q
k
≥ u(ε) + δ
N
k=1
[V [ f
k
(
¯
y − ε)]q
k
].
Simple manipulations lead to
N
k=1
V [ f
k
(
¯
y)]−V [ f
k
(
¯
y−ε)]
f
k
(
¯
y)− f
k
(
¯
y − ε)
·
f
k
(
¯
y) − f
k
(
¯
y−ε)
ε
q
k
≥
1
δ
u(ε) − u(0)
ε
.
Now, letting ε ↓ 0, we see that the left side goes to
N
k=1
[V
( f
k
(
¯
y)) ·
f
k
(
¯
y)]q
k
whereas the right side goes to infinity, a contradiction.
Theorem 5.5 Under [T.1]–[T.5] and [U.1]–[U.4], the value function
V (y) is differentiable at y > 0 and
V
(y) = u
(c(y)). (5.27)
Proof. Recall the corresponding result from the deterministic model.
Theorem 5.6 Under [T.1]–[T.5] and [U.1]–[U.4],
u
(c(y)) = δ
N
k=1
u
[c( f
k
(i(y)))] f
k
(i(y))
q
k
. (5.28)
Proof. This proof is elementary but long, and is split into several steps.
To prove the theorem, first fix y
∗
> 0 and write
c
∗
≡ c(y
∗
), x
∗
≡ i(y
∗
) and x
k
1
≡ i( f
k
(x
∗
)) for k = 1,...,N .
400 Discounted Dynamic Programming Under Uncertainty
Let
H(c) ≡ u(c) + δ
N
k=1
u
f
k
(y
∗
− c) − x
k
1
q
k
(5.29)
for all c > 0 such that the second expression in the right-hand side is
well defined. We begin with
Step 1. The following maximization problem (P) is solved by c
∗
:
maximize H(c)
subject to
0 ≤ c ≤ y
∗
and f
k
(y
∗
− c) ≥ x
k
1
, k = 1, 2 ...N .
Proof of Step 1. Note that if c
∗
were not a solution to (P), there is some
ˆ
c such that
0 ≤
ˆ
c ≤ y
∗
, f
k
(y
∗
−
ˆ
c) x
k
1
for k = 1,...,N ;
and H(
ˆ
c) > H(c
∗
).
Define a process (
ˆ
x
˜
,
ˆ
c
˜
,
ˆ
y
˜
) from y
∗
as follows:
ˆ
c
0
=
ˆ
c,
ˆ
x
0
= y
∗
−
ˆ
c;
ˆ
x
1
= x
k
1
,
ˆ
c
1
= f
k
(
ˆ
x
0
) − x
k
1
,
ˆ
y
1
= f
k
(
ˆ
x
0
),
and for t ≥ 2
ˆ
c
t
= c
∗
t
ˆ
x
t
= x
∗
t
ˆ
y
t
= y
∗
t
,
where (x
˜
∗
, c
˜
∗
, y
˜
∗
) is the optimal process from y
∗
. The difference
between the expected total discounted utility of (x
˜
∗
, c
˜
∗
, y
˜
∗
) and
(
ˆ
x
˜
,
ˆ
c
˜
,
ˆ
y
˜
)is(H (c
∗
) − H (
ˆ
c)) < 0, a contradiction to the optimality
of (x
˜
∗
, c
˜
∗
, y
˜
∗
) from y
∗
.
Step 2. There exists ξ>0 such that for all c in U ≡ (c
∗
− ξ, c
∗
+ ξ )
the constraints “0 ≤ c ≤ y
∗
”, and “ f
k
(y
∗
− c) − x
k
1
≥ 0, for all
k = 1,...,N ” are satisfied.
Proof of Step 2. Use Theorem 5.4 to conclude that 0 < c
∗
< y
∗
;so
let ξ
1
> 0 be such that 0 < c
∗
− ξ
1
< c
∗
< c
∗
+ ξ
1
< y
∗
. Then
define U
1
≡ (c
∗
− ξ
1
, c
∗
+ ξ
1
). Clearly for all c ∈ U
1
the first con-
straint in Step 2 holds. Now, x
k
1
≡ i( f
k
(x
∗
)) < f
k
(x
∗
) for each
k = 1, 2,...,N . Hence, there is some constant J
1
> 0 such that
f
k
(x
∗
) − x
k
1
J
1
uniformly for all k = 1,...,N .
6.5 Applications 401
By continuity of f
k
there exists some ξ ∈ (0,ξ
1
) such that uniformly
for all k = 1, 2,...,N
| f
k
(x
∗
+ ε) − f
k
(x
∗
)| <
J
1
2
for all ε ∈ (−ξ,ξ). Define U ≡ (c
∗
− ξ , c
∗
+ ξ ). Then for each
c ∈ U, putting ε = c
∗
− c we get uniformly for all k = 1,...,N
f
k
(y
∗
− c) − x
k
1
≡ f
k
(x
∗
+ ε) − x
k
1
=
f
k
(x
∗
) − x
k
1
+ [ f
k
(x
∗
+ ε) − f
k
(x
∗
)]
≥ J
1
−
J
1
2
=
J
1
2
.
In other words, for each c ∈ U
f
k
(y
∗
− c) − x
∗
1
≥
J
1
2
for all k = 1, 2 ...N . (5.30)
In particular, the second constraint is satisfied for c ∈ U . Since
U ⊂ U
1
, both constraints are satisfied for c ∈ U.
Thus, we see that H (c ) is well defined on the open interval U ,
and c
∗
maximizes H (c)onU. The first-order condition leads to
H
(c
∗
) = 0. Hence,
u
(c
∗
) = δ
N
k=1
u
f
k
(y
∗
− c
∗
) − x
k
1
f
k
(y
∗
− c
∗
)
q
k
.
In other words,
u
(c(y
∗
)) = δ
k
u
(c( f
k
(i(y
∗
)))) f
k
(i(y
∗
))
q
k
.
We refer to (5.28) as the stochastic Ramsey–Euler condition.
Corollary 5.2 Under [T.1]–[T.5] and [U.1]–[U.4]
(a) the investment policy function i(y) is continuous and increasing;
(b) the consumption policy function c(y) is continuous and increa-
sing.
402 Discounted Dynamic Programming Under Uncertainty
Proof.
(a) One needs to prove that, in fact, i (y) is increasing. Note that for
y > 0
V
(y) = u
(c(y)) =
N
k=1
V
( f
k
(i(y))) f
k
(i(y))
q
k
. (5.31)
Let y
> y > 0. Since V is strictly concave, if y
>
¯
y > y
V
(y
) < V
(
¯
y) < V
(y).
Suppose i(y
) = i(y). Then i(y
) = i(
¯
y) = i (y) for
¯
y ∈ [y, y
]. Now,
V
(
¯
y) > V
(y
)
=
N
k=1
V
( f
k
(i(y
))) f
k
(i(y
))
q
k
=
N
k=1
V
( f
k
(i(
¯
y))) f
k
(i(
¯
y))
q
k
= V
(
¯
y),
a contradiction. If y
> y = 0, i(y
) > 0 = i(y).
(b) To prove that c(y) is increasing, take y
> y > 0 and note that
u
(c(y)) = V
(y) > V
(y
) = u
(c(y
)),
and use the strict concavity of u.Ify
> y = 0, c(y
) > 0 and c(y) = 0.
6.5.3 The Random Dynamical System of Optimal Inputs
We shall study the long-run behavior of the optimal process as a random
dynamical system. We spell out the details of the optimal input process
x
˜
∗
= (x
∗
t
) generated by the optimal investment policy function i. Let
y > 0 be any initial stock. Then x
∗
0
= i(y) > 0 is the optimal initial
input. The optimal input process is given by
x
∗
t+1
= i[ f (x
∗
t
)], (5.32)