Bhattacharya R., Majumdar M. Random Dynamical Systems: Theory and Applications

Подождите немного. Документ загружается.

2.13 Complements and Details 213

(λ), namely, 1. Thus we have proved ρ

(1)

< ∞ a.s., no matter what the

initial state (x,θ) of the process {W

: n ≥ 0} is.

To see that the blocks (C9.17) are i.i.d., note that if X

∈ A

and θ

1, then X

j+1

= Z

j+1

, so that W

j+1

= (Z

j+1

,θ

j+1

), which is independent

of {W

, W

,...,W

}. Hence, by the strong Markov property of {W

n ≥ 0} applied with the stopping time ρ

( j)

, the conditional distribution

of {W

( j)

: n ≥ 1}, given the pre-ρ

( j)

sigmaﬁeld F

( j)

, is the distribution

of a {W

: n ≥ 0} process with an initial random vector W

= (Z ,θ):

Z and θ independent having distributions ν and B

(λ), respectively. In

particular, this conditional distribution does not depend on W

( j)

. Hence

( j)

: n ≥ 1} is independent of F

( j)

. It follows that the blocks U

( j ≥ 1) are independent and each has the same distribution as that of

:1 ≤ n ≤ ρ

(1)

}when W

= (Z ,θ) as above. This completes the proof

of part (a).

To prove part (b), we need to show that Eρ

(1)

< ∞ when X

∈ A

and θ

= 1 a.s. Write

(1)

∞



j=1

( j)

{θ

(i)

=0 for 1≤i< j,θ

( j)

=1}

, (C9.25)

the ﬁrst term on the right-hand side being η

(1)

{θ

(1)

=1}

. Taking expec-

tations and noting that (i) θ

( j)

is independent of F

( j−1)

and of η

( j)

, (ii)

E(η

( j)

− η

( j−1)

| F

( j−1)

) ≤ c := sup{E(η

(1)

| X

= x): x ∈ A

}, one has

Eρ

(1)

∞



j=1

{θ

(i)

=0,1≤i≤j −1}

(η

( j)

P(θ

( j)

= 1)

= λ

∞



j=1

{θ

(i)

=0,1≤i≤j −1}



( j−1)

+ E



( j)

− η

( j−1)

| F

( j−1)



≤ λ

∞



j=1

{θ

(i)

=0,1≤i≤j −1}



( j−1)

+ c



= λ

∞



j=1

c(1 − λ)

j−1

+ λ

∞



j=1

{θ

(i)

=0,1≤i≤j −1}

( j−1)

= c + λ

∞



j=2

(1 − λ)E

{θ

(i)

=0,1≤i≤j −2}



( j−2)

+ E(η

( j−1)

− η

( j−2)

| F

( j−2

)



214 Markov Processes

≤ c + cλ

∞



j=2

(1 − λ)

j−1

+ λ(1 −λ)

∞



j=3

{θ

(i)

=0,1≤i≤j −2}

( j−2)

≤ c + c(1 − λ) +c(1 − λ)

+···=c/λ.

Theorem C9.2 (Ergodicity of Harris recurrent processes) Let p(x, dy)

be a transition probability on (S, S), a Borel subset of a Polish space.

Assume that a Markov process {X

: n ≥ 0} with this transition probabil-

ity is A

-recurrent and locally minorized, according to Deﬁnition C9.2,

and that the process is strongly aperiodic, i.e., N = 1. If, in addition,

sup{E

(1)

: x ∈ A

} < ∞, there exists a unique invariant probability π

and, whatever the initial distribution of {X

:n≥ 0},

sup

A∈S

⊗∞

(n)

(A) − Q

(A)|→0asn →∞, (C9.26)

where Q

(n)

is the distribution of the after-n process X

:={X

n+m

m ≥ 0}, and Q

is the distribution of the process with initial distribu-

tion π. In particular, sup{|p

(n)

(x, B) − (π(B)|: B ∈ S}→0 for every

x ∈ S.

Proof. Deﬁne the ﬁnite measure π

on (S, S)by

(B):= E

(1)



n=ρ

(0)

∈B}

(B ∈ S). (C9.27)

We will ﬁrst show that

π(B):= π

(B)/π

(S)(B ∈ S) (C9.28)

is the unique invariant probability for p(x, dy). For this note that, for

every bounded measurable f ,

( f ):=

ρ( j)



m=ρ

( j−1)

f (X

), ( j ≥ 1) (C9.29)

is, by Proposition C9.2, an i.i.d. sequence with ﬁnite expectation

fdπ

By the SLLN,

−1



j=1

( f ) →

f π

a.s. as N →∞, (C9.30)

2.13 Complements and Details 215

whatever be the initial distribution. In particular, taking f ≡ 1 one has,

with probability 1,

(N )

(0)

(N )

− ρ

(0)

→ E



(1)

− ρ

(0)



≡ π

(S), asN →∞.

(C9.31)

Now deﬁne

:= sup{j ≥ 1: ρ

( j)

≤ n} (n = 1, 2,...).

Then, by (C9.31),

)

→ π

(S). (C9.32)

Using the fact that 0 ≤ n − ρ

)

≤ ρ

+1)

− ρ

)

and that (ρ

+1)

−

)

)/M

→ 0 a.s. as n →∞(see Exercise 8.2),

→ π

(S) a.s. as n →∞. (C9.33)

In the same manner, for all bounded measurable f , and for all n >ρ

(1)



m=1

f (X

) =

(1)



m=1

f (X

) +



j=1

( f ) +



)

<m≤n

f (X

)

≈



j=1

( f ) →

(S)

fdπ

, (C9.34)

since (1/n)(n − ρ

)

) = (M

/n)(n − ρ

)

)/M

→ 0, as shown above.

Thus, with probability 1,



m=1

f (X

) →

fdπ ∀ bounded measurable f , (C9.35)

which implies



m=1

f (X

m+1

) →

fdπ a.s. as n →∞. (C9.36)

Taking expectations term by term after conditioning the mth term with

respect to X

, one gets





m=1

(Tf)(X

)



→

fdπ. (C9.37)

216 Markov Processes

But the random sequence within brackets on the left-hand side con-

verges a.s. to

Tf dπ , by (C9.35). Hence

Tf dπ =

fdπ for

all bounded measurable f , proving the invariance of π. Unique-

ness of the invariant probability follows on integrating g

(x):=

(1/n)



m=1

f (y) p

(m)

(x, dy) with respect to an invariant probability

˜π,say,toget

fd˜π ∀n, and noting that g

(x) →

fdπ as n →∞.

We will complete the proof by a coupling argument. One may

construct a (common) probability space (



, F



, P



), say, on which

are deﬁned two independent families {X

,θ

, (θ

, Z

,α

)

n≥1

} and

{

, (

, ˜α

)

n≥1

} as above, with {θ

, (θ

, Z

,α

)

n≥1

} and

{

, (

, ˜α

)

n≥1

} independent of X

and

. Let X

have (an ar-

bitrary initial) distribution µ and let

have the invariant distribution.

Let {ρ

( j)

: j ≥ 0} and {˜ρ

( j)

: j ≥ 0} denote the corresponding indepen-

dent sequences of regeneration times, and let {Y

= ρ

(0)

, Y

= ρ

(k)

−

(k−1)

(k ≥ 1)}, {

= ˜ρ

(0)

= ˜ρ

(k)

− ˜ρ

(k−1)

(k ≥ 1)} be the corre-

sponding sequences of lifetimes of two independent renewal processes.

Under the hypothesis, Y

and

(k ≥ 1) each has a lattice span 1, and

= E

= E(ρ

(2)

− ρ

(1)

) < ∞. Hence, as in the proof of Feller’s re-

newal theorem C9.1 (see (C9.10)), ρ := inf{n ≥ 0: R

= 0} < ∞

a.s., where {R

: n ≥ 0} and {

: n ≥ 0} are the two residual lifetime

processes corresponding to {Y

: k ≥ 0} and {

: k ≥ 0}, respectively.

Note that ρ is the ﬁrst common renewal epoch, i.e., there are m,

m such

that ρ

= ˜ρ

= ρ. Now deﬁne



if ρ>n,

if ρ ≤ n.

(C9.38)

Since ρ is a stopping time for the Markov process {(W

): n ≥ 1},

with W

:= (X

,θ

= (

), one may use the strong Markov

property to see that {X

: n ≥ 0} and {X



: n ≥ 0} have the same distri-

bution. Hence, for all A ∈ S

⊗∞

)



∈ A



− P



∈ A



)

≤

)







∈ A,ρ ≤ n) − P



∈ A,ρ ≤ n



)







∈ A,ρ > n

− P



∈ A,ρ > n



)

≤ P(ρ>n).

(C9.40)

Here we have used the fact (X



)

= X

on {ρ ≤ n}.

If one lets N > 1 in the local minorization condition in Theorem C9.2

(or Theorem 9.2), one must allow the possibility of cycles. For example,

2.13 Complements and Details 217

every (irreducible) positive recurrent Markov chain satisﬁes the hypothe-

sis of Theorem 9.2. with A

a singleton {x

}and N =period of the chain.

In particular, for positive recurrent birth–death chains with p

x,x+1

= β

x,x−1

≡ δ

= 1 − β

(0 <β

< 1), which are periodic of period 2,

clearly, the conclusion of Theorem C9.2 does not hold; in particular,

(n)

(x, y) does not converge as n →∞. It is known, however, that, under

the hypothesis of Theorem 9.2, there exists a unique invariant probability

π, and the state space has a decomposition S = (∪

i=1

) ∪ M, where

, D

,...,D

, M are disjoint sets in S such that (i) p(x, D

i+1

) = 1

∀x ∈ D

,1≤ i ≤ d (with D

d+1

:= D

) and (ii) M is π-null: π (M) = 0.

The hypothesis of Theorem C9.2 holds on each D

as the state space,

with the transition probability q(x, dy) = p

(d)

(x, dy) where N is a

multiple of d. In view of the recurrence property, the convergence of



m=1

(m)

(x, dy)toπ (dy) (in variation distance) follows for every

x ∈ S, using Theorem C9.2.

Theorems 9.2, C9.2 were obtained independently by Athreya and Ney

(1978) and Nummelin (1978a). For a general analysis of the structure of

φ-irreducible and φ-recurrent processes, we refer to the original work by

Jain and Jamison (1967) and Orey (1971). For a comprehensive account

of stability in distribution for φ-irreducible and φ-recurrent Markov pro-

cesses, we refer to Meyn and Tweedie (1993). For the sake of complete-

ness, we state the following useful result (see Meyn and Tweedie, loc.

cit. Chapter 10).

Theorem C9.3 If a Markov process is φ-irreducible (with respect to a

nonzero sigmaﬁnite measure φ) and admits an invariant probability π,

then (i) the process is positive Harris ( π-) recurrent, (ii) π is the unique

invariant probability, and (iii) the convergence (9.30) holds.

The original work of Harris appears in Harris (1956). Our treatment

follows that of Bhattacharya and Waymire (2006b, Chapter 4, Sec-

tion 16).

Section 2.10. In order to prove Proposition 10.1, we will make use of

the following inequality, which is of independent interest. Write F

for

the sigmaﬁeld generated by {X

:0≤ j ≤ k}.

Lemma C10.1 (Kolmogorov’s maximal inequality) Let X

, X

,...,X

be independent random variables, E X

= 0 for 1 ≤ j ≤ n, EX

< ∞

for 0 ≤ j ≤ n. Write S

:=X

+···+X

:=max{|S

|:0≤ k ≤ n}.

218 Markov Processes

Then for every λ>0, one has

P(M

≥ λ) ≤

≥λ}

dP ≤

Proof. Write {M

≥ λ}=∪

k=0

, where A

={|S

|≥λ}, A

{|S

| <λfor 0 ≤ j ≤ k − 1, |S

|≥λ},1≤ k ≤ n. Then

P(M

≥ λ) =



k=0

P(A

) =



k=0





≤



k=0



/λ





k=0





≤



k=0



| F



(C10.1)

since E(S

)=E[S

+2S

− S

)+(S

−S

)

)=E(S

) +

E(S

− S

| F

) + E{(S

− S

)

| F

)}=E(S

| F

) + 2S

0 + E(S

− S

)

≥ E(S

| F

). Now the last expression in (C10.1)

equals



k=0







≥λ}



≤

Proof of Proposition 10.1. Without loss of generality, let σ = 1. Write

= X

+···+X

. For any given ε>0, one has



)

− S

[nα]

)

≥ ε[nα]

1/2



≤ P(|ν

− [nα]|≥ε

[nα])

+P( max

{m:|m−[nα]|<ε

[nα]}

− S

[nα]

|≥ε([nα])

1/2

). (C10.2)

Thus (S

− S

[nα]

)/[nα]

1/2

→ 0 in probability, as n →∞. Since

[nα]

/[nα]

1/2

converges to N (0, 1) in distribution, and ν

/[nd ] → 1in

probability, S

√

= (S

/[n

]

1/2

)([n

]/ν

)

1/2

converges to N(0, 1)

in distribution.

The ﬁrst term on the right-hand side goes to zero, by assumption

(10.9). By Lemma C10.1, the second term on the right-hand side is no

more than 2(ε([nα]

1/2

)

−2

(ε

[nα]) = 2ε. The factor 2 here arises from the

maximum of two maxima, the ﬁrst over {m:[nα] − ε

[nα] < m ≤ [nα]}

(in which case Lemma C10.1 is applied to summands X



:= X

[nα]−j

,1 ≤

j <ε

[nα]), while the second maximum is taken over {m :[nα] < m <

2.13 Complements and Details 219

[nα] + ε

[nα]} (in which case Lemma C10.1 is applied to summands



:= X

[nα]+j

,1≤ j <ε

[nα]).

Section 2.11. Among references to the weak convergence theory of

probability measures, we cite Billingsley (1968), Parthasarathy (1967),

Bhattacharya and Ranga Rao (1976), Dudley (1989), and Pollard (1984).

Our treatment is also inﬂuenced by that of Bhattacharya and Waymire

(2006(a)).

A basic result on weak convergence of probability measures on a

metric space (S, d) is Theorem C11.1. For its statement, recall that the

boundary ∂ A :=

A\A

, where

A is the closure of A and A

, is the interior

of A. Finally, S is the Borel sigmﬁeld of S. A set A ∈ S is said to be

a P-continuity set (for some given probability measure P on (S, S)) if

P(∂ A) = 0. Unless stated otherwise, all probability measures on S are

deﬁned on S.

Theorem C11.1 (Alexandrov’s theorem) Let P, P

(n ≥ 1) be probability

measures on S. The following statements are equivalent:

(i) P

converges weakly to P.

(ii) lim

fdP

f d P for all bounded, uniformly continuous

real f .

(iii) lim sup

(F) ≤ P(F) for all closed F.

(iv) lim inf

(G) ≥ P(G) for all open G.

(v) lim

(A) = P(A) for all P-continuity sets A.

Proof. The plan is to ﬁrst prove: (i) implies (ii), (ii) implies (iii), (iii)

implies (i), and hence that (i), (ii), and (iii) are equivalent. We then

directly prove that (iii) and (iv) are equivalent and that (iii) and (v)

are equivalent. (i) implies (ii): This follows directly from the deﬁni-

tion. (ii) implies (iii): Let F be a closed set and δ>0. For a suf-

ﬁciently small but ﬁxed value of ε, G

={x: d(x, F) <ε} satisﬁes

P(G

) < P(F) + δ, by continuity of the probability measure P from

above, since the sets G

decrease to F =∩

ε↓0

. Construct a uniformly

continuous function h on S such that h(x) = 1onF, h(x) = 0onthe

complement G

of G

, and 0 ≤ h(x) ≤ 1 for all x. For example, deﬁne

on [0, ∞)byθ

(u) = 1 − u/ε,0≤ u ≤ ε, θ

(u) = 0 for u >ε, and

let h(x) = θ

(d(x, F)). In view of (ii) one has lim

hdP

hdP.

220 Markov Processes

In addition,

(F) =

hdP

≤

hdP

and

hdP =

hdP ≤ P(G

) < P(F) + δ. (C11.1)

Thus

lim sup

(F) ≤ lim

hdP

hdP < P(F) + δ.

Since δ is arbitrary, this proves (iii).

(iii) implies (i): Let f ∈ C

(S). It sufﬁces to prove

lim sup

fdP

≤

fdP. (C11.2)

For then one also gets inf

fdP

≥

fdP, and hence (i), by replac-

ing f by − f . But in fact, for (C11.2) it sufﬁces to consider f ∈ C

(S)

such that 0 < f (x) < 1, x ∈ S, since the more general f ∈ C

(S) can

be reduced to this by translating and rescaling f . Fix an integer k and let

be the closed set F

={x: f (x) ≥ i/k}, i = 0, 1,...,k. Then taking

advantage of 0 < f < 1, one has



i=1

i − 1



i − 1

≤ f (x) <



≤

fdP≤



i=1



i − 1

≤ f (x) <





i=1

[P(F

i−1

) − P(F

)] =



i=1

P(F

). (C11.3)

The sum on the extreme left may be reduced similarly to obtain



i=1

P(F

) ≤

fdP<



i=1

P(F

). (C11.4)

In view of (iii) lim sup

) ≤ P(F

) for each i. So, using the upper

bound in (C11.4) with P

in place of P and the lower bound with P,it

2.13 Complements and Details 221

follows that

lim sup

fdP

≤

fdP.

Now let k →∞to obtain the asserted inequality (C11.2) to complete

the proof of (i) from (iii).

(iii) iff (iv): This is simply due to the fact that open and closed sets are

complementary.

(iii) implies (v): Let A be a P-continuity set. Since (iii) implies (iv), one

has

A) ≥ lim sup

(

A) ≥ lim sup

(A)

≥ lim inf

(A) ≥ lim inf

) ≥ P(A

). (C11.5)

Since P(∂ A) = 0, P(

A) = P( A

) so that the inequalities squeeze down

to P( A) and lim

(A) = P(A) follows.

(v) implies (iii): Let F be a closed set. Since ∂{x: d(x, F) ≤ δ}⊆{x:

d(x, F) = δ}, these boundaries are disjoint for distinct δ. Thus at most

countably many of them can have positive P-measure (Exercise), all

others being P-continuity sets. In particular there is a sequence of pos-

itive numbers δ

→ 0 such that the sets F

={x: d(x, F ) ≤ δ

} are

P-continuity sets. From (v) one has lim sup

(F) ≤ lim

) =

P(F

) for each k. Since F is closed, one also has F

↓ F, so that (iii)

follows from continuity of the probability P from above.

In the special ﬁnite dimensional case S =

, the following theorem

provides some alternative useful conditions for weak convergence.

Theorem C11.2 (Finite dimensional weak convergence) Let P

, P

,...,

P be probability measures on

. The following are equivalent state-

ments.

(a) {P

: n = 1, 2,...} converge weakly to P.

(b)

fdP

→

f d P for all (bounded) continuous f vanishing

outside a bounded set.

(c)

fdP

→

f d P for all inﬁnitely differentiable functions f

vanishing outside a bounded set.

(d) Let F

(x):= P

((−∞, x

] ×···×(−∞, x

]), and F(x):=

P((−∞, x

] ×···×(−∞, x

]), x ∈ R

, n = 1, 2,... be such that

(x) → F(x) as n →∞, for every point of continuity x of F.

222 Markov Processes

Proof. We give the proof for the one-dimensional case k = 1. First let us

check that (b) is sufﬁcient. It is obviously necessary by deﬁnition of weak

convergence. Assume (b) and let f be an arbitrary bounded continuous

function, | f (x)|≤c, for all x. For notational convenience, write {x ∈

: |x|≥N}={|x|≥N }, etc. Given ε>0, there exists N such that

P({|x|≥N }) <ε/2c. Deﬁne θ

by θ

(x) = 1 for |x|≤N,θ

(x) =

0 for |x|≥N + 1, and linearly interpolate for N ≤|x|≤N + 1.

Then,

lim

n→∞

({|x|≤N + 1}) ≥ lim

n→∞

(x) dP

(x)

(x) dP(x)

≥ P({|x|≤N }) > 1 −

so that

lim

n→∞

({|x| > N + 1}) ≡ 1 − lim

n→∞

({|x|≤N + 1}) <

(C11.6)

Now let θ



:= θ

N +1

and deﬁne f

:= f θ



. Noting that f = f

{|x|≤N + 1} and that on {|x| > N + 1}, one has | f (x)|≤c, we have

lim

n→∞

)

fdP

−

fdP

)

≤

lim

n→∞

)

−

)

lim

n→∞

(cP

({|x| > N + 1}) + cP({|x| > N + 1}))

< c

+ c

= ε.

Since ε>0 is arbitrary,

fdP

→

fdP. So (a) and (b) are equiv-

alent. Let us now show that (b) and (c) are equivalent. It is enough to

prove (c) implies (b). For each ε>0 deﬁne the function

(x) = d(ε)exp



−

1 − x

/ε



for |x| <ε,

= 0 for |x|≥ε, (C11.7)

where d(ε) is so chosen as to make

(x) dx = 1. One may check that

(x) is inﬁnitely differentiable in x. Now let f be a continuous function

that vanishes outside a ﬁnite interval. Then f is uniformly continuous