Jacques J., Raimondo M. Semi-markov risk models for finance, insurance and reliability
Подождите немного. Документ загружается.
Markov renewal processes
97
lim ( ) 0, , .
ij
t
tijT
φ
→∞
=
∈ (10.15)
Moreover, from
Proposition 10.2 and relation (10.3):
1
lim ( ) ( ) , , ,
jj
ij ij
m
t
kk
kl
tG ijC
π
η
φ
πη
→∞
=+
=
∞∈
∑
(10.16)
where
()
1
,...,
lm
π
π
+
represents the unique stationary probability distribution of
the sub-Markov chain with C as state space.
Since
() ,
ij ij
Gf
∞
= (10.17)
we get
,
() ,
ij i C
Gf
∞
= (10.18)
where f
i,C
is the probability that the system, starting in state i, will be absorbing
by the recurrent class C.
As there is only one essential class, we know that for all states i of I:
f
i,C
=1, (10.19)
proving so the following proposition.
Proposition 10.3 For any periodic uni-reducible MRP, we have:
'
lim ( ) , ,
ij j
t
tjI
φ
→∞
=
Π∈ (10.20)
where
'
1
0, ,
,.
jj
j
m
kk
kl
jT
jC
πη
πη
=+
∈
⎧
⎪
⎪
Π
=∈
⎨
⎪
⎪
⎩
∑
(10.21)
Here too, as the limit in (10.21) is independent of the initial state i, this result
gives an ergodic property.
10.2.2. General Case
For any aperiodical MRP, there exists a unique partition of the state space I:
1
,,
r
ITC Crm
=
<∪∪∪ (10.22)
where T represents the set of transient states and , 1,...,Cr
ν
ν
=
represents the
ν
th essential class necessarily formed of positive recurrent states.
From Chapter 2, we know that the system will finally enter one of the essential
classes and will then stay in it forever. So a slight modification of the last
proposition leads to the next result.
Proposition 10.4 For any aperiodic MRP, we have :
98 Chapter 3
'
lim ( ) , , ,
ij ij
t
tijI
φ
→∞
=
Π∈ (10.23)
with, for any
,1,...,jC r
ν
ν
∈= :
'
'
'
'
,
, , 1,..., ,
0, , ', ' 1,..., ,
,
j
ij
iC j
iC r
iC v r
fiT
ν
ν
ν
ν
ν
ν
νν
⎧
Π∈=
⎪
Π= ∈ ≠ =
⎨
⎪
Π∈
⎩
(10.24)
where
'
(, )
j
jC
ν
ν
Π∈is the only stationary distribution of the sub-SMP with C
ν
as
state space, that is :
'
,
jj
j
kk
kC
ν
π
η
π
η
∈
Π=
∑
(10.25)
where
()
,
k
kC
ν
ν
π
∈ is the unique stationary distribution of the sub-Markov chain
with C
ν
as state space and
()
,
,
iC
f
iT
ν
∈ is the unique solution of the linear system
,.
iijj ij
jT jC
ypy piT
ν
∈∈
−
=∈
∑
∑
(10.26)
Note that, in this proposition, the ergodic property is lost; this is due to the
presence of the quantities
,iC
f
ν
in relation (10.24).
11 DELAYED AND STATIONARY MRP
Let us suppose we begin to observe the evolution of an economic or physical
system S at time T
0
and that the probabilistic evolution of this system is like a
semi-Markov process.
There is absolutely no reason that we should observe a transition of the system at
time 0. In fact, we observe the state Z(0) while waiting for the first observed
transition occurring at random time T
1
. The first lifetime X
1
is a residual time and
may have its own distribution function.
This leads to the concept of delayed MRP.
Definition 11.1 The bidimensional process
(
)
(,), 0
nn
JT n≥ with
0
1
0,
,1,
nn
T
TX Xn
=
=
++ ≥
(11.1)
is called a delayed Markov renewal sequence or delayed Markov renewal
process (in short DMRP) of triplet
(
)
,,pQQ
if
Markov renewal processes
99
()
()
0
110
(i) ( ) , ,
(ii) , ( ), , ,
(iii) , ( , ), 0,..., 1 ( ), , , , 1.
i
ij
nnkk ij
PJ i p i I
PJ jX xJ i Q x ij I
PJ jX x J X k n Q x ij Ix n
== ∈
=≤== ∈
=≤ = −= ∈∈>
(11.2)
This definition is clearly based upon the supposition that the m-dimensional
vector
1
( ,..., )
m
p
p=p represents a probability distribution on I and that matrices
,QQ
are two semi-Markov kernels.
As in the case of renewal theory, there exist simple relations between renewal
functions, marginal distributions, etc… of a DMRP of triplet ( , , )
pQQ
and the
corresponding functions of the classical associated MRP of kernel
Q.
So, with the convention of adding a tilde to the functions related to the DMR, let
()
ij
R
t
be the Markov renewal functions of the DMRP, that is:
(
)
0
() () .
ij j
R
tENtJi
=
=
(11.3)
We know that:
()
0
1
(1)
1
1
() ,
= ( )
= ( ).
ij n n
n
n
ik kj
n
ik kj
n
R
tPJjTtJi
QQ t
QRt
∞
=
∞
−
=
∞
=
=
=≤=
•
•
∑
∑
∑
(11.4)
If we now consider the transition probabilities for the delayed semi-Markov
process
()
(), 0Zt t≥
, associated with the DMRP of triplet
(
)
,,pQQ
,that is:
(
)
() () (0) ,
ij
tPZt jZ i
φ
=
==
(11.5)
using as usual, a simple probabilistic argument, we get:
1
0
() (1 ()) ( ) ( ),
t
m
ij ij j kj ik
k
tHt tydQy
φδ φ
=
=− + −
∑
∫
(11.6)
where of course, the
kj
φ
are the transition probabilities of the SMP of kernel Q
and with
1
() (), , 0.
m
jjk
k
Ht Q tj It
=
=
∈≥
∑
(11.7)
Using
Proposition 10.1, relation (11.4) gives
1
() (1 ()) (1 ) (),
m
ij ij j j kj ik
k
tHt HRQt
φδ
=
=− + −••
∑
(11.8)
and from relation (11.6):
100 Chapter 3
() (1 ()) (1 ) ().
ij ij j j ij
tHtHRt
φδ
=− +−•
(11.9)
Relation (11.6) also shows that the limiting distributions of the transition
probabilities , ,
ij
ij I
φ
∈
exist and are known provided the limiting distributions of
the transition probabilities , ,
ij
ij I
φ
∈
exist and are known too.
Indeed, let us suppose that
lim ( ) , , ,
ij ij
t
tijI
φ
→∞
=
Π∈ (11.10)
then, from relation (11.8), the limits
lim ( ) , ,
ij ij
t
tijI
φ
→∞
=
Π∈
(11.11)
also exist and moreover they are given by
1
,, ,
m
ij ik kj
k
p
ij I
=
Π
=Π∈
∑
(11.12)
where
().
ij ij
pQ
=
+∞
(11.13)
Using now
Proposition 10.2 and relation (11.11), we get the following result.
Proposition 11.1 If the MRP associated with the DMRP of triplet
()
,,pQQ
is
irreducible, then
lim ( ) ; ,
ij j
t
tijI
φ
→∞
=
Π∈
(11.14)
with
,.
jj
j
kk
k
jI
π
η
πη
Π
=∈
∑
(11.15)
It follows that in the ergodic case, both DSMP and associated SMP have the
same asymptotic behaviour.
As in renewal theory, a very special but very interesting case of the notion of
DMRP, is the case of the so-called stationary MRP (in short SMRP).This type of
process appears when one begins to observe an MRP which has been running a
long time so that the first observed r.v. X
1
is in fact the excess
γ
.
More precisely, let us define the vector
S
p
whose jth component (j=1,…,m) is
given by
,Sj j
p
=
Π (11.16)
and let us define the semi-Markov kernel
S
Q as follows:
Markov renewal processes
101
,
0
1
((),
0,
()
0.
0,
x
ij ij
Sij
i
pQydy
x
Qx
x
η
⎧
−
≥
⎪
=
⎨
<
⎪
⎩
∫
(11.17)
We can now give the following definition.
Definition 11.2 The DMRP of triplet
(
)
,,
SS
pQQ
with kernel Q ergodic, is
called a stationary MRP (in short SMRP) of kernel
Q.
In this case, it can be proved that (see Janssen and Manca (2006))
11
0
1
(, ) (1()).
x
jjk jk
j
PJ kX x p F y dy
νν
ν
π
πη
=≤= −
∑
∫
∑
(11.18)
This last result comes from the fact that
11 ,,
(, ) ()
Sl Slk
l
PJ kX x p Q x=≤=
∑
(11.19)
and so
11
0
1
(, ) ( ()),
x
jjkjk
j
PJ kX x p Q y dy
νν
ν
π
πη
=≤= −
∑
∫
∑
(11.20)
which is equivalent to (11.18).
The next propositions give specific properties of SMRP.
Of course, r.v. N
S,j
(t) represents the total number of passages in state j on [0,t] for
the considered SMRP. From this point on, we will systematically use the
subscript “S” for parameters related to the SMRP.
Proposition 11.2 For every SMRP of triplet
(
)
,,
SS
pQQ
with kernel Q ergodic,
all renewal functions are linear. More precisely:
,
(()) , .
j
Sj
j
E
Nt tjI
η
Π
=
∈ (11.21)
Corollary 12.1 For every SMRP of triplet
(
)
,,
SS
pQQ
with kernel Q ergodic, we
have:
,
1
() .
m
Sj
j
t
ENt
η
=
⎛⎞
=
⎜⎟
⎝⎠
∑
(11.22)
This corollary shows that in looking at the total number of transitions, every
SMRP is, on the average, equivalent to a stationary renewal process defined by
the d. f. H given by:
102 Chapter 3
1
.
m
j
j
j
HH
π
=
=
∑
(11.23)
The linearity of Markov renewal functions related to ergodic associated SMRP
will give, as the main consequence, the stationarity of the basic stochastic
processes related to it.
This means that all marginal distributions will no longer depend on t, and in
particular, we have
Proposition 11.3 For any ergodic SMRP defined by kernel Q, the stochastic
process
()
(), (), 0Zt t t
γ
≥ is stationary and
(
)
() .
j
PZt j
=
=Π (11.24)
12 PARTICULAR CASES OF MRP
We will devote this paragraph to particular cases of MRP having the advantage
of leading to some explicit results.
12.1 Renewal Processes And Markov Chains
For the sake of completeness, let us first say that with m=1, that is that the
observed system has only one possible state, the kernel
Q has only one element,
say the d.f.
F, and the process (X
n
,n>0) is then a renewal process.
Secondly, to obtain Markov chains studied in Chapter 2, it suffices to choose for
the matrix
F the following special degenerating case:
1
,,
ij
F
UijI
=
∀∈ (12.1)
and of course an arbitrary Markov matrix
P.
This means that all r.v. X
n
have a.s. the value 1, and so the single random
component is the (J
n
) process, which is, from relation (3.4) a homogeneous MC
of transition matrix
P.
12.2 MRP Of Zero Order (PYKE (1962))
There are two types of such processes
12.2.1 First Type Of Zero Order MRP
This type is defined by the semi-Markov kernel
[
]
,
ii
p
F=Q (12.2)
so that:
,,.
ij i ij i
p
pF Fj I
=
=∈ (12.3)
Markov renewal processes
103
Naturally, we suppose that for every i belonging to I, p
i
is strictly positive.
In this present case, we have that the r.v.
,0
n
Jn≥
are independent and identically
distributed and moreover that the conditional interarrival distributions do not
depend on the state to be reached, so that, by relation (3.11),
,.
ii
H
Fi I
=
∈ (12.4)
Moreover, since:
1
((,),1,)(),
n
nkk nJ
PX x J X k n J F x
−
≤≤−= (12.5)
we get:
1
((),1) ().
m
nk jj
j
PX x X k n pF x
=
≤≤−=
∑
(12.6)
Introducing the d.f. F defined as
1
,
m
j
j
j
F
pF
=
=
∑
(12.7)
the preceding equality shows that, for an MRP of zero order of the first type, the
sequence
(, 1)
n
Xn≥
is a renewal process characterized by the d.f. F.
12.2.2 Second Type Of Zero Order MRP
This type is defined by the semi-Markov kernel
,
ij
p
F
⎡
⎤
=
⎣
⎦
Q (12.8)
so that:
,,,.
ij i ij j
p
pF F ij I
=
=∈ (12.9)
Here too, we suppose that for every i belonging to I, p
i
is strictly positive.
Once again, the r.v. , 0
n
Jn≥ are independent and equidistributed and moreover
the conditional interarrival distributions do not depend on the state to be left, so
that, by relation (3.11),
1
(), .
m
ijj
j
HpFFiI
=
=
=∈
∑
(12.10)
Moreover, since:
((,),1,)(),
n
nkk nJ
PX x J X k n J F x≤≤−= (12.11)
we get
1
((),1) ()().
m
nk jj
j
PX x X k n pF x Fx
=
≤≤−= =
∑
(12.12)
The preceding equality shows that, for an MRP of zero order of the second type,
the sequence
(, 1)
n
Xn≥ is a renewal process characterized by the d.f. F as in the
first type;
The basic reason for these similar results is that these two types of MRP are the
reverses (timewise) of each other.
104 Chapter 3
12.3 Continuous Markov Processes
These processes are defined by the following particular semi-Markov kernel
(
)
() 1 , 0,
i
x
ij
xp e x
λ
−
⎡⎤
=
−≥
⎣⎦
Q (12.13)
where
ij
p
⎡⎤
=
⎣⎦
P is a stochastic matrix and where the parameters
,
i
iI
λ
∈
are
strictly positive.
The standard case corresponds to that in which 0,
ii
p
iI
=
∈ (see Chung (1960)).
From relation (12.13), we get:
() 1 .
i
x
ij
Fx e
λ
−
=− (12.14)
Thus the d.f. of sojourn time in state i has an exponential distribution depending
uniquely upon the occupied state i, such that both the excess and age processes
also have the same distribution.
For m=1, we get the usual Poisson process of parameter .
λ
13 A CASE STUDY IN SOCIAL INSURANCE (JANSSEN
(1966))
13.1 The Semi-Markov Model
We will now return to the problem presented in section 9.7 of Chapter 2 which
was solved using a Markov chain model. Here, we will extend the model to a
semi-Markov one allowing us to take into account the duration of passage from
invalidity degree S
j
to invalidity degree S
k
.
In this case, relation (9.114) of Chapter 2 is replaced by the following one:
'
1
() () .
m
iijj
j
St tS
φ
=
=
∑
(13.1)
The study of the financial equilibrium of the funds thus depends on:
''
lim ( ),
ii
t
SSt
→∞
= (13.2)
or
'
1
lim ( ) .
m
iijj
t
j
StS
φ
→∞
=
=
∑
(13.3)
It suffices here to apply Proposition 10.3 in order to get the result in the general
case.
For the case of uni-reducibility, with P represented by relation (9.117) of Chapter
2, we get:
Markov renewal processes
105
5
'
3
,
ijj
j
SS
=
=Π
∑
(13.4)
with
5
3
,
jj
j
kk
k
π
η
π
η
=
Π=
∑
(13.5)
345
(,,)
π
ππ
being given by relation (9.120) of Chapter 2 and with
5
3
,
j
jk jk
k
p
η
β
=
=
∑
(13.6)
j
k
β
being the mean of the d.f. F
jk
related to the duration of passage from
invalidity degree S
j
to invalidity degree S
k
.
Here too, although we have uni-reducibility, the asymptotic state
i
S is
independent of the initial state i.
To pass from the Markov chain model to the semi-Markov one, the additional
information needed is knowledge of the matrix
ij
β
⎡
⎤
=
⎣
⎦
Β .
13.2 Numerical Example
In this section we return to the examples mentioned at the end of Chapter 2 to
give the results obtained in the Markov case and also in a semi-Markov
environment.
First we have the asymptotic result related to the disability example shown in
section 9 of Chapter 2. The means ,
i
iI
η
∈
were computed from real data from
Campania, an Italian region with more than 4 million inhabitants, and were
applied to the M.C. used for the example.
The semi-Markov asymptotic limit vector is given in Table 13.1.
States
i
η
i
Π
1 2.00822 0.00000
2 3.35343 0.00000
3 3.34247 0.22000
4 3.46575 0.34217
5 3.32603 0.43783
Table 13.1: disability
i
Π
From result (13.4) and with the degree given in section 9.7 of Chapter 2, we get
the following result:
'
0.787
i
S = , (13.7)
106 Chapter 3
more or less equivalent to result (9.122) of Chapter 2.
14 (J-X) PROCESSES
In Section 1, we introduced the concept of (J,X) process with Definition 2.4 for
which the r.v. X
n
, n=1,2,…take their values in the whole real line instead of in
+
for what we called in Definition 2.2 a positive (J,X) process.
In fact, in 1969, Janssen showed that the consideration of (J,X) processes leads to
a very interesting generalisation of the classical concept of random walk with a
lot of applications in stochastic modelling. Let us begin this section by recalling
the basic definition.
Definition 14.1 Let p=(p
1
,…,p
m
) be an m-dimensional vector of initial
probabilities and Q an extended semi-Markov kernel as defined by Definition
2.3. Then every two-dimensional process
(( , ), 0,1,...)
nn
JX n
=
with values in
I ×
and satisfying the conditions
P(X
0
=0)=1, a.s.,
P(J
0
=i)=p
i
, i=1,…,m with
1
1
m
i
i
p
=
=
∑
, (14.1)
for all n>0, j=1,…,m, we have:
1
( , ( , ), 0,..., 1) ( ), . .
n
nnkk Jj
PJ jX x J X k n Q x as
−
=≤ = −= , (14.2)
is called a (J,X) process or an extended semi-Markov chain (in short, ESMC).
From this definition, it follows that we can no longer represent the sample paths
of such a process with step functions, as the r.v. X
n
can be positive or negative
but we can see the S-process defined by:
01
, 0,1,...
nn
SXX Xn=+++ = (14.3)
as the successive positions of a particle moving on a real line and starting from
the origin if the r.v. X
n
n=0,1,… represents the successive steps of this random
movement exactly as the interpretation of a classical random walk corresponding
to the case of m=1.
This leads to the following definition.
Definition 14.2 The S-process defined by relation (14.3) is called a semi-Markov
random walk (in short an SMRW).
It is clear that basic results on positive (J,X) processes given in the preceding
chapter are still valid here provided that these properties do not involve the non-
negativity of the X
n
.
The following proposition summarises the basic properties.