2.2 Construction of Stochastic Processes 125
random variables {X
n
: n = 0, 1,...} such that X
0
has distribution µ
and (X
0
, X
1
,...,X
n
) has distribution µ
0,1,...,n
defined by (2.1), i.e.,
P(X
0
= i
0
, X
1
= i
1
,...,X
n
= i
n
) is given by the right-hand side of
(2.1). This process is called a Markov chain having the transition prob-
ability matrix p and initial distribution µ. In the special case in which µ
is a point mass, µ({i
0
}) = 1 for some i
0
∈ S, one says that the Markov
chain has initial state i
0
. Observe that S is a complete separable metric
space with the metric d(i, i) = 0, d(i, j) = 1(i = j), having the class of
all subsets of S as its Borel sigmafield.
Next, on a complete separable metric state space (S, S) one may
similarly define a Markov process {X
n
: n = 0, 1,...} with a transition
probability (function) p(x, B)(x ∈ S, B ∈ S) and initial distribution µ.
Here
(1) x → p(x, B) is a Borel-measurable function for each B ∈ S;
(2) B → p(x, B) is a probability measure on (S, S) for each x ∈ S.
For example, let S =
R and suppose we are given a transition probabil-
ity density function p(x, y), i.e., (i) p(x, y) ≥ 0, (ii) (x, y) → p(x, y)
continuous on S × S, and (iii)
!
R
p(x, y) dy = 1 for every x. Then,
p(x, B):=
!
B
p(x, y) dy (B ∈ S) satisfies the conditions (1)–(2) above.
Let us define for each n ≥ 0 the distribution µ
0,1,...,n
on S
n+1
(with
Borel sigma-field S
⊗(n+1)
), by
µ
0,1,...,n
(B
0
× B
1
×···×B
n
)
=
"
B
0
"
B
1
···
"
B
n−1
"
B
n
p(x
n−1
, x
n
)p(x
n−2
, x
n−1
) ···
p(x
0
, x
1
) dx
n
dx
n−1
···dx
1
µ (dx
0
)
(B
i
∈S; i = 0, 1,...,n), n ≥ 1, (2.2)
where µ is a given probability on (S, S). By integrating out the last coor-
dinate from µ
0,1,...,n+1
one gets µ
0,1,...,n
, proving consistency. Hence there
exists a stochastic process {X
n
: n = 0, 1, 2,...} on some probability
space (, F, P) such that X
0
has a distribution µ and (X
0
, X
1
,...,X
n
)
has distribution µ
0,1,...,n
(n ≥ 1). This process is called a Markov process
with transition probability density p(x, y) and initial distribution µ.
As in the case of Example 2.1, Markov processes with given transi-
tion probabilities and initial distributions can be constructed on arbitrary
measurable state spaces (S, S). (See Complements and Details.)