Bhattacharya R., Majumdar M. Random Dynamical Systems: Theory and Applications

Подождите немного. Документ загружается.

2.11 Markov Processes on Metric Spaces 193

(a) Suppose there exist x ∈ S, a sequence of integers 1 ≤ n

< n

···, and a probability measure π

on S such that

−1



r=0

(r)

(x, dy) converges weakly to π

as k →∞. (11.6)

Then π

is invariant for p.

(b) If the limit (11.6) holds for all x ∈ S with the same limit π

π(x ∈ S), then π is the unique invariant probability for p.

Proof.

(a) By (11.6), recalling (T

f )(x) =

f (y) p

(r)

(x, dy), one has

−1



r=0

f )(x) →

k→∞

f (y)π

(dy) ∀ f ∈ C

(S). (11.7)

Applying this to Tf, which belongs to C

(S) by virtue of the Feller

property of p(x, dy), one gets



r=1

f )(x) →

k→∞

(Tf)(y)π

(dy). (11.8)

But the left-hand sides of (11.7) and (11.8) have the same limits. Hence

(Tf)(y)π

(dy) =

f (y)π

(dy) ∀ f ∈ C

(S),

or (see (9.1



)),

f (y)(T

∗

)(dy) =

f (y)π

(dy) ∀ f ∈ C

(S),

proving the invariance of π

: T

∗

= π

(b) By (a), π is invariant, and (11.7) holds with π in place of π

.If

ν is another invariant probability, then integrating both sides of (11.7)

w.r.t. ν, one obtains

fdν =

fdπ ∀ f ∈ C

(S), proving ν = π.

Remark 11.1 One can deﬁne a metric on the space P(S) of all probabil-

ity measures on (a metric space) S such that the convergence P

→ P in

that metric is equivalent to (11.5) (see Complements and Details). When

194 Markov Processes

S is a compact metric space, one such metric is given by

d(P, Q)=

∞



k=1

−k

(1 +f

)

−1

)

dP −

)

, P, Q ∈ P(S),

(11.9)

where { f

: k ≥ 1} is dense in the sup-norm topology of C(S) ≡

(S). [Here  f  := max{| f (x)|: x ∈ S}]. Such a dense sequence exists

because C(S) is a (complete) separable metric space in the sup-norm dis-

tance  f − g ( f, g ∈ C(S)). By a Cantor diagonal argument one can

then derive the following result, (whose proof is given in Complements

and Details):

Lemma 11.1 If S is a compact metric space then the space P(S) of

all probability measures on (S, S) is a compact metric space under

the metric (11.9), which metrizes the topology of weak convergence

on P(S).

As a consequence of this lemma we can prove the following.

Theorem 11.2 Suppose on a compact metric state space S, p(x, dy) is

a transition probability having the Feller property. Then there exists at

least one invariant probability for p.

Proof. Fix x ∈ S. In view of the Lemma above, there ex-

ists a subsequence of the sequence



n−1

r=0

(r)

(x, dy), n ≥ 1, say,

(1/n

)



−1

r=0

(r)

(x, dy), k ≥ 1, which converges to some probability

measure π

on (S, S), as k →∞. By Theorem 11.1 (a), π

is invariant

for p.

Corollary 11.1 The transition probability of a Markov chain on a ﬁnite

state space S has at least one invariant probability.

Proof. S is compact metric under the metric d(x, y) = 1ifx = y, and

d(x, x) = 0. In this case every transition probability has the Feller prop-

erty (see Example 11.1).

That the Feller property is generally indispensable in Theorems 11.1

and 11.2 is illustrated by the following example due to Liggett.

2.11 Markov Processes on Metric Spaces 195

Example 11.4 Let S ={m/(m + 1): m = 0, 1,...}∪{1}. Deﬁne p(m/

(m + 1), {(m + 1)/(m + 2)}) = 1 ∀m = 0, 1, 2,...,and p(1, {0}) = 1.

Then there does not exist any invariant probability, although S is com-

pact (Exercise). This is, of course, the case of an extremely degener-

ate transition probability given by a deterministic dynamical system:

f (m/m + 1) = (m + 1)/(m + 2) (m = 0, 1, 2,...), f (1) = 0. One may,

however, easily perturb this example to construct one in which the

transition probability is not degenerate. For example, let p(m/

(m +

1), {(m + 1)/(m + 2)}) = θ, p(m/(m + 1); {(m + 2)/(m + 3)}) = 1 −

θ (m = 0, 1, 2,...), p(1, {0}) = θ, p(1, {1/2}) = 1 − θ, for some θ ∈

(0, 1). One may show that there does not exist an invariant probability in

this case as well (Exercise).

It may be noted that the results on the existence and uniqueness of

an invariant probability in Section 2.9 did not require topological as-

sumptions. However, the assumptions were strong enough to imply the

convergence of the sequence (1/n)



n−1

r=0

(r)

(x, B) to a probability π (B)

uniformly for all B ∈ S. The existence of an invariant probability may be

demonstrated without the assumption of uniform convergence, as shown

below.

Proposition 11.1 Let (S, S) be a measurable state space. If for some

x ∈ S, there exist a probability measure π

and a sequence 1 ≤ n

< ··· such that

−1



r=0

(r)

(x, B) →

k→∞

(B) ∀B ∈ S, (11.10)

then π

is an invariant probability for p. If, in addition, π

does not

depend on x, then it is the unique invariant probability.

Proof. By standard measure theoretic arguments for the approximation

of bounded measurable functions by the so-called simple functions of

the form



i=1

,...,A

disjoint sets in S, and a

, a

,...,a

nonzero reals), (11.10) may be shown to be equivalent to

−1



r=0

f )(x) →

k→∞

f (y)π

(dy) ∀ f ∈ B(S). (11.11)

Now follow the same line of argument as used in the proof of Theorem

11.1, but for all f ∈ B(S).

196 Markov Processes

Remark 11.2 The property (a) in Theorem 11.1 is implied by the

tightness of the sequence of probability measures {



r=1

(r)

(x, dy):

n ≥ 1} (see Complements and Details). Here a sequence {P

: n ≥ 1}

⊂ P(S) is tight if ∀ε>0 there exists a compact set K

such that

) ≥ 1 − ε ∀n. By combining this fact with Theorem C9.3 in Com-

plements and Details, we get the following result:

Proposition 11.2 Suppose a Markov process on a metric space (S, d)

has the Feller property, and it is ν-irreducible (with respect to some

nonzero sigmaﬁnite measure ν). If, in addition, {



r=1

(r)

(x, dy):

n ≥ 1} is tight for some x, then the process is positive Harris recur-

rent and



r=1

(r)

(z, dy) converges in variation distance to a unique

invariant probability π, for all z ∈ S.

2.12 Asymptotic Stationarity

We have focused much attention in the preceding sections on the stability

in distribution of Markov processes {X

: n ≥ 0}, i.e., convergence of the

distribution X

to a steady state, or invariant probability π, whatever be

the initial distribution as n →∞. One may suspect that this means that

the process X

: = (X

, X

n+1

,...) converges in distribution, as n →∞,

to the stationary Markov process (X

∗

, X

∗

,...) whose initial distribution

is π and which has the same transition probability as {X

: n ≥ 0}. In this

section we demonstrate that this is indeed true under broad conditions.

Deﬁnition 12.1 By the distribution of a process {Y

: k = 0, 1, 2,...}

on a measurable state space (S, S) we mean a probability measure Q on

the Kolmogorov (or product) sigmaﬁeld (S

∞

, S

⊗∞

), such that Prob({Y

k ≥ 0}∈B) = Q(B) ∀B ∈ S

⊗∞

. Here S

∞

is the set of all sequences

, x

,...)inS (i.e., x

∈ S ∀j ≥ 0), and S

⊗∞

is the sigmaﬁeld on S

∞

generated by all ﬁnite dimensional measurable sets of the form {x ∈

∞

:(x

, x

,...,x

) ∈ A}∀A ∈ S

⊗{0,1,...,k}

≡ S ⊗ S ⊗···⊗S(k + 1

terms).

Deﬁnition 12.2 If (S, d) is a separable metric space and S its Borel

sigmaﬁeld, a process {X

: n ≥ 0} on (S, S) is said to be asymptotically

stationary if X

:= (X

, X

n+1

, X

n+2

,...) converges in distribution, as

n →∞,to(X

∗

, X

∗

, X

∗

,...), where the latter is a stationary process.

Here distributions of X

and (X

∗

, X

∗

,...) are probability measures Q

2.12 Asymptotic Stationarity 197

Q, say, on the space (S

∞

, S

⊗∞

), where S

∞

has the product topology and

⊗∞

is its Borel sigmaﬁeld. One can show that S

∞

is a separable metric

space with the metric (see Appendix)

∞

(x, y) =

∞



n=0

(d(x

, y

) ∧ 1) ∀x, y ∈ S

∞

. (12.1)

On the product space S

≡ S

{0,1,...,m−1}

we will use the metric

(x, y) =

m−1



n=0

(d(x

, y

) ∧ 1) ∀x = (x

,...,x

m−1

y = (y

,...,y

m−1

) ∈ S

. (12.2)

Lemma 12.1 Let (S, d) be a separable metric space, and S

∞

the space

of all sequence x = (x

, x

,...) of elements of S, endowed with the

metric d

∞

in (12.1). Let C

∞

) be the set of all real-valued bounded

continuous functions on S

∞

depending only on ﬁnitely many coordinates.

If P

(n ≥ 1), P are probability measures on the Borel sigmaﬁeld B(S

∞

)

such that

hdP

→ h d P for all h ∈ C

∞

),asn→∞, (12.3)

then P

converges weakly to P.

Proof. Let g be an arbitrary real-valued bounded uniformly continuous

function on S

∞

.Givenε>0, there exists h ∈ C

∞

) such that

sup

|g(x) − h(x)|≡g − h

∞

<ε/2. (12.4)

To see this note that there exists δ ≡ δ(ε) > 0 such that |g(x) − g(y)| <

ε/2ifd

∞

(x, y) <δ. Find k ≡ k(ε) such that



∞

n=k+1

<δ. Then

∞

(x, y) <δif x

= y

for all n ≤ k.Nowﬁxz ∈ S and deﬁne

h(x): = g(x

, x

,...,x

, z, z, z,...). (12.5)

Then (12.4) holds. In view of this, one has

)

gdP

−

hdP

)

<ε/2 and

)

gdP −

hdP

)

<ε/2,

198 Markov Processes

so that

)

gdP

−

gdP

)

gdP

−

hdP

)

hdP

−

hdP

)

hdP−

gdP

)

<ε+

)

hdP

−

hdP

)

By (12.3) this implies

lim

n→∞

)

gdP

−

gdP

)

≤ ε.

Since ε is arbitrary, it follows that

gdP

→

gdP.

This implies that P

converges weakly to P. (See Theorem C 11.1 (ii) in

Complements and Details.)

We now state and prove one of the main results on the asymptotic

stationarity of Markov processes {X

: n ≥ 1}.

Theorem 12.1 Assume that X

has a Feller continuous transition prob-

ability and converges in distribution to a probability measure π as n →

∞. Then (a) π is an invariant probability and (b) X

= (X

, X

n+1

,...)

converges in distribution to Q

as n →∞, where Q

is the distribu-

tion of the stationary Markov process (X

∗

, X

∗

,...) having the initial

distribution π and the same transition probability p(x, dy) as that of

(n ≥ 0).

Proof. (a) Let ν

denote the distribution of X

. For every bounded

continuous f : S →

R, one has

f (y)π (dy) = lim

n→∞

Ef(X

n+1

) = lim

n→∞

EE[ f (X

n+1

) | X

]

= lim

n→∞



f (y) p(x, dy)



(dx)



f (y) p(x, dy)



π(dx),

2.12 Asymptotic Stationarity 199

since, by Feller continuity, x →

f (y) p(x, dy) is continuous (and

bounded) on S. (b) We will show that, for all m ≥ 0, and for all bounded

continuous f : S

{0,1,...,m}

→ R, one has

lim

n→∞

Ef(X

, X

n+1

,...,X

n+m

)

···

f (x

, x

,...,x

)p(x

m−1

, dx

) ···p(x

, dx

)π(dx

(12.6)

This is clearly enough, by Lemma 12.1. Fix m ≥ 0 and f as above. Now

using the Markov property,

Ef(X

, X

n+1

,...,X

n+m

)

···

f (x

, x

,...,x

)p(x

m−1

, dx

) ···p(x

, dx

)ν

(dx

)

g(x

)ν

(dx

), (12.7)

where

g(x

):=

···

f (x

, x

,...,x

)p(x

m−1

, dx

) ···p(x

, dx

(12.8)

To prove continuity of g, we ﬁrst show that the function

, x

,...,x

m−1

):=

f (x

, x

,...,x

m−1

, x

)p(x

m−1

, dx

)

(12.9)

is continuous on S

{0,1,...,m−1}

and bounded. Boundedness follows from

h



∞

≤f 

∞

. Now ﬁx (x

, x

,...,x

m−1

), and let (y

(n)

,...,y

(n)

m−1

)

converge to (x

, x

,...,x

m−1

)asn →∞. Then, by Lebesgue’s Domi-

nated Convergence Theorem,

lim sup

n→∞

)

(n)

,...,y

(n)

m−1

− h

(

,...,x

m−1

)

≤lim sup

)

(n)

,...,y

(n)

m−1

, x

−f (x

,...,x

m−1

, x

)

p(x

m−1

, dx

)

+ lim sup

)

(n)

,...,y

(n)

m−1

, x

(n)

m−1

, dx

−

(n)

,...,y

(n)

m−1

, x

p(x

m−1

, dx

)

= lim sup

)

(n)

,...,y

(n)

m−1

, x

(n)

m−1

, dx

−

(n)

,...,y

(n)

m−1

, x

p(x

m−1

, dx

)

. (12.10)

200 Markov Processes

The set of functions F :={x

→ f (y

(n)

,...,y

(n)

m−1

, x

): n ≥ 1} is a

uniformity class (see Proposition C11.1 in Complements and Details).

Therefore,

sup

h∈F

)

h(x

(n)

m−1

, dx

−

h(x

)p(x

m−1

, dx

)

→ 0

as n →∞. (12.11)

The same proof applies to show that

m−1

, x

,...,x

m−2

)

, x

,...,x

m−2

, x

m−1

)p(x

m−2

, dx

m−1

) (12.12)

is continuous (and bounded). Proceeding in this manner, we ﬁnally show

that

g(x

) = h

) ≡

, x

)p(x

, dx

) (12.13)

is continuous.

Remark 12.1 As an application of Theorem 12.1, let X

(n ≥ 0) be a

Markov process on S =

with an arbitrary initial distribution, and

:= length of run of values > c, beginning with time n. Under the

hypothesis of Theorem 12.1, L

converges in distribution to the length

of runs under Q

, beginning with time 0, provided π({c}) = 0. To see

this, write

P(L

= k) = P(X

n+j

> c for 0 ≤ j ≤ k − 1, X

n+k

≤ c)

→ Q

({(x

, x

,...): x

> c for 0 ≤ j ≤ k − 1, x

≤ c})(k > 0),

P(L

= 0) = P(X

≤ c) → π ({x: x ≤ c}). (12.14)

Note that the topological boundary of the set A ={(x

, x

,...): x

> c

for 0 ≤ j ≤ k − 1, x

≤ c} is contained in the union of the k + 1 sets

B( j) ={(x

, x

,...): x

= c},0≤ j ≤ k, and Q

(B( j)) = π ({c}) = 0,

by assumption.

A stronger notion of asymptotic stationarity applies when the distribu-

tion of X

converges in variation norm, as in the case of Harris positive

recurrent cases considered in Section 2.9.

2.13 Complements and Details 201

Deﬁnition 12.3 Let X

(n ≥ 0) be a Markov process on a measurable

state space (S, S). If the distribution of X

= (X

, X

n+1

,...) converges

in total variation distance to that of a stationary process (X

∗

, X

∗

,...) with

the same transition probability, then we will say X

(n ≥ 0) is asymptot-

ically stationary in the strong sense.

Theorem 12.2 Let p(x, dy) be a transition probability on (S, S). Sup-

pose the n-step transition probability p

(n)

(x, dy) converges to π(dy)

in total variation distance for every n ∈ S. Then a Markov process

(n ≥ 0) having the transition probability p(x, dy) is asymptotically

stationary in the strong sense, no matter what its initial distribution is.

Proof. Let B ∈ S

⊗∞

be arbitrary. Then,

∈ B) =

Prob(X

∈ B | X

= x)µ(dx),

Prob(X

∈ B | X

= x) =

(B) p

(n)

(x, dy),

where P

, P

denote the distributions of the Markov process under the

initial distribution µ and the initial state y, respectively. Hence

sup

B∈S

⊗∞

)



∈ B



− P

(B)

)

= sup

B∈S

⊗∞

)



(B)



(n)

(x, dy) − π(dy)





µ(dx)

≤

sup



)

f (y)



(n)

(x, dy) − π(dy)



)

| f |≤1, f measurable



µ(dx) → 0asn →∞.

2.13 Complements and Details

Section 2.2. A complete measure theoretic proof of Kolmogorov’s ex-

istence theorem may be found in Billingsley (1986, pp. 506–517). For

Kolmogorov’s original proof, see Kolmogorov (1950). We sketch here

an elegant functional analytic proof due to Nelson (1959). First, let S

be a compact metric space and take

as the parameter set. The set

of continuous functions on the compact metric space S

depending

only on ﬁnitely many coordinates is dense (with respect to the supremum

202 Markov Processes

norm) in the set C(S

) of all continuous functions on S

by the Stone–

Weirstrass theorem (see Royden 1968, p. 174). On C

a continuous

linear functional  is deﬁned, by integration of f ∈ C

with respect to

the ﬁnite-dimensional distribution on the coordinates on which f de-

pends. This functional is well deﬁned by the consistency condition. The

unique extension of this to C(S

), also denoted , then has the repre-

sentation ( f ) =

fdµ, for a uniquely deﬁned probability measure µ

on the Borel sigmaﬁeld of S

,bytheRiesz representation theorem (see

Royden 1968, p. 310). In case S is noncompact, but Polish, S may be

mapped homeomorphically onto a Borel subset of [0, 1]

∞

, and thereby

we may regard S as a Borel subset of [0, 1]

∞

(see Royden 1968, p. 326).

Then the closure

S of S in [0, 1]

∞

is compact. This leads to a unique

probability measure µ on

, whose ﬁnite-dimensional distributions

are as those speciﬁed. It is simple to check that µ(S

) = 1, completing

the construction.

An important extension of Kolmogorov’s Existence Theorem to ar-

bitrary measurable state spaces (S, S) holds for Markov processes in

general and, independent sequences in particular. This extension is due

to Tulcea (see, e.g., Neveu 1965, pp. 161–167). Here the probability is

constructed on (S

, S

⊗Z

), where S

⊗Z

is the usual product sigmaﬁeld.

We sometimes write (S

∞

, S

⊗∞

), instead, for this inﬁnite product space.

Section 2.4. Suppose a state i of a Markov chain has period d, and

i ↔ j . Then there exist positive integers a and b such that p

(a)

0 and p

(b)

> 0. If m is a positive integer such that p

(m)

> 0, then

one has p

(2m)

≥ p

(m)

> 0, so that p

(a+m+b)

≥ p

(a)

(m)

(b)

> 0 and

(a+2m+b)

≥ p

(a)

(2m)

(b)

> 0. Hence a + 2m + b and a + m + b are

both multiples of d, as is their difference m. This implies that the period

of j is greater than or equal to the period of i, namely d. By symmetry of

↔, the period of i is greater than or equal to that of j . Thus all essential

states belonging to the same equivalence class of communicating states

have the same period.

Section 2.5. The following result was stated without proof in Section 2.5.

Proposition C5.1 For an irreducible aperiodic Markov chain on a space

S there exists for each pair (i, j ) a positive integer ν = ν

such that

(ν)

> 0. If S is ﬁnite, one may choose ν = max {ν

: i, j ∈ S} indepen-

dent of (i, j ).