Jacques J., Raimondo M. Semi-markov risk models for finance, insurance and reliability
Подождите немного. Документ загружается.
Markov renewal processes
127
Definition 21.8 The associated counting process ((), 0)Nt t≥ or the CNHSMP
Z= ((), 0)Zt t≥ of kernel
Q is explosive iff
,..
L
as
=
∞
(21.11)
and non-explosive iff
,..
L
as
<
∞
(21.12)
For very general counting processes, De Vylder and Haezendonck (1980) have
given necessary and sufficient conditions for non-explosion. Here, in general, we
always assume non-explosive processes.
For the two-dimensional process
(( , ), 0)
nn
JT n≥
, we have the following result:
(
)
()
(0) (1)
1100
(1) (0) (2)
2200
0
, , 0 (0, )( ( )),
,,0 (,)(0,)(()),
ij ij
t
kj ij ij
k
PJ jT tJ iT Q t Q t
PJ jT tJ iT Q xt Q dx Q t
=≤=== =
=≤=== =
∑
∫
(21.13)
and in general
()
(1) (1)
00
0
()
,,0 (,)()
( ( )), 1).
t
n
nn kj ij
k
n
ij
PJ jT tJ iT Q xt Q dx
Qtn
−
=≤===
=>
∑
∫
(21.14)
Using matrix notation, we may write for two
mm
×
matrices of mass functions
A(t), B(t):
00
1
() () ( ) ( ) ,
n
tt
kj ik
k
td t B zdA z
=
⎡
⎤
=
⎢
⎥
⎣
⎦
∑
∫∫
AB
(21.15)
and so relations (21.14) can be written under the matrix form:
() ( 1) (1)
0
() ()
(1) (0)
() ( ,) ( ), 1,
with
() () , 1,
() (0,) .
t
nn
nn
ij
ij
tztdxn
tQtn
tQt
−
=
>
⎡⎤
=≥
⎣⎦
⎡⎤
=
⎣⎦
∫
QQQ
Q
Q
(21.16)
In the particular class of classical SMP, the relation (21.16) gives the n-fold
convolution of the SM kernel
Q.
Another very important distribution is the marginal distribution of the Z process
as it gives the state occupied by the system S at time t.
Let us introduce the following probabilities:
()
()
(,) () (0) , ( ) (), () ,, , 0.
n
ij
st PZt jZ iNs Ns Ns n ij In
φ
===−< =∈≥
(21.17)
The conditioning means that
n
Ts
=
and that there exists a transition at time s
such that the new state occupied after the transition is i.
Clearly, these probabilities satisfy the following relations:
128 Chapter 3
() () () ( 1)
(,) (1 (,)) (,) (, ),, .
t
nnnn
ij ij i kj ik
s
kI
s
tHst utQsduijI
φδ φ
−
∈
=− + ∈
∑
∫
(21.18)
From relation (21.1), it is clear that we have:
1
(1)
1
((,), 1) (),..
n
n
nkk Jjn
PJ j J T k n p T as
−
−
−
=≤−= (21.19)
It follows that the process
(
)
,0
n
Jn≥ can be viewed as a conditional multiple
Markov chain; this means that, given the sequence
(
)
,0
n
Tn≥ , each transition
from
1nn
J
J
−
→
obeys a non-homogeneous Markov chain of kernel
(1)
1
()
n
n
PT
−
−
(see Definition 21.3).
Definition 21.9 The conditional multiple Markov chain
(
)
,0
n
Jn≥ is called the
imbedded multiple MC.
21.1.2 Special Cases
Let us point out that Definition 21.2 is quite general as indeed it is non-
homogeneous both for the time s and for the number of transitions n, this last one
giving the possibility to model epidemiological phenomena such as AIDS for
example (see in Janssen and Manca (2006)) the example of Polya processes and
semi-Markov extensions).
This extreme generality gives importance to the following particular cases.
(i) Non-Homogeneous Markov Additive Process And Semi-Markov Process
If in the sequel Q=
()
(1)
(,), 1
n
st n
−
≥Q , we have:
(1)
(,) (,), 1, ,
n
s
tstnst
−
=
≥<QQ
(21.20)
that is
Q independent of n, then the kernel Q is called a non-homogeneous semi-
Markov kernel (in short NHSMK) defining a non-homogeneous Markov additive
process (in short NHMAP)
(
)
(,), 0
nn
JT n≥ and a non-homogeneous semi-
Markov process (in short NHSMP) ( ( ), 0).ZZtt
=
≥
This family was introduced in a different way by Hoem (1972).
It is clear that the relation (21.20) means that the sequences
(1) (1)
(,) (,), () () 0
nn
st st s s n
−−
=
=∀≥FF PP
(21.21)
are independent of n or equivalently that
(,) () (,).
s
tsst
=
⋅QPF (21.22)
Let us point out that, in this case, relations (21.18) become:
(,) (1 (,)) ( ,) (, ),, .
t
ij ij i ij ik
s
k
s
tHst utQsduijI
φδ φ
=− + ∈
∑
∫
(21.23)
If moreover, we have
Markov renewal processes
129
() , 0,ss
=
≥PP (21.24)
then the kernel
Q is called a partially non-homogeneous semi-Markov kernel (in
short PNHSMK) defining a partially non-homogeneous Markov additive process
(in short PNHMAP)
(
)
(,), 0
nn
JT n≥ and a partially non-homogeneous semi-
Markov process (in short PNHSMP) ( ( ), 0).ZZtt
=
≥
This family was introduced in a different way by Hoem (1972).
(ii) Non-Homogeneous MC
If the sequences
(1)
(), 0
n
ss
−
∀≥P
are independent of s, then
(
)
,0
n
Jn≥ is a
classical non-homogeneous MC (in short NHMC)
(iii) Homogeneous Markov Additive Process
A PNHSMK Q such that
(,) ( ),, 0, 0,Fst Ft s st t s
=
−≥−≥ (21.25)
is of course a classical homogeneous SM kernel as in section 2.
(iv) Non-Homogeneous Renewal Process
For m=1, The CNHMRP of kernel
Q is given by
(1)
(,) ( (,)),, 0, 0
n
st st st t s
−
=
>−≥QF
(21.26)
and characterizes the sequence
(, 0)
n
Xn≥
with, as in (21.1),
0
(1)
11
0
0
0, . .,
(,1)(,),,
0, , . .
n
nk nn
n
nk
k
Xas
PX xX k n FT T x x
TT Xas
−
+
−−
=
=
≤≤−= +∈
==
∑
(21.27)
In this case, the process
(, 0)
n
Xn≥
is called a completely non-homogeneous
dependent renewal process (in short CNHDRP) of kernel
Q.
If moreover,
(1) (1)
(,) ( ),, 0, 0, 1,
nn
Fst Ftsst ts n
−−
=−>−≥≥
(21.28)
it follows that
0
(1)
0, . .,
(,1)(),,1
n
nk
Xas
PX xX k n Fx x n
−+
=
≤≤−= ∈≥
(21.29)
and so the process
(, 0)
n
Xn≥
is a sequence of t independent r.v. called a
completely non-homogeneous renewal process (in short CNHRP) of kernel F.
130 Chapter 3
Remark 21.1 In the non-homogeneous case, it is much more difficult to obtain
asymptotic results (see for example Benevento (1986), Thorisson (1986),
Papadopoulou-Vassiliou (1994)) for interesting theoretical results). That is not so
dramatic as that we can say non-homogeneous models are used for modelling
transient situations and not asymptotic ones and that is why we personally think
that all attention must be given to the construction of numerical methods for
example to be able to solve the non-homogeneous integral equations system
(21.23); this is done in the next chapter.
However, let us mention that, for the particular case of non-homogeneous
Markov chains, there exist more asymptotic results (see for example Isaacson and
Madsen (1976).
Chapter 4
DISCRETE TIME AND REWARD SMP AND
THEIR NUMERICAL TREATMENT
1 DISCRETE TIME SEMI-MARKOV PROCESSES
1.1 Purpose
This chapter will present both discrete time homogeneous (DTHSMP) and
non-homogeneous (DTNHSMP) semi-Markov processes and the numerical
methods to be used for applying semi-Markov models in real-life problems,
furthermore the Semi-Markov ReWard Processes (SMRWP) will be presented.
Although, in general, time in real-life problems is continuous, the real
observation of the considered system is almost always made in discrete time even
if the used time unit may in some cases be very small.
The choice of this time unit depends on what we observe and what we wish to
study.
For example if we are studying the random evolution of the earthquake activity
in a tectonic fracture zone, then it could be observed with a unitary time scale of
ten years. If we are studying the behaviour of a disablement resulting from a job
related illness, the unitary time could be one year, and so on.
So it results that the phenomenon of time evolution is continuous, nevertheless
usually, the observations are discrete in time.
Consequently, if we construct a model to be fitted with real data, in our opinion,
it would be better to begin with discrete time models.
1.2 DTSMP Definition
Though DTHSMP and DTNHSMP definitions are similar to the continuous ones,
for the sake of completeness, we will recall these definitions using directly the
terminology used for discrete time models.
Let I={1, 2, …, m} be the state space and let
{
}
,,PΩℑ be a probability space. Let
us also define the following random variables:
:
n
J
I
Ω
→
,
:
n
T
Ω
→
. (1.1)
Definition 1.1 The process (, )
nn
J
T is a discrete time homogeneous Markov
renewal process or a discrete time non-homogeneous Markov renewal process if
132 Chapter 4
the kernels Q associated with the process are defined respectively in the
following way:
[]
(
)
[]
11
() , - | , ,
ij n n n n
Qt PJ jT T tJ i ij It
++
== = ≤= ∈∈Q , (1.2)
[]
(
)
[]
11
(,) , | , , , ,
ij n n n n
Qst PJ jT tJ iT s ij Ist
++
===≤==∈∈Q . (1.3)
As in the continuous time case, it results that for the homogeneous case, we
define:
[]
lim ( ) ; , ,
ij ij
t
pQtijIt
→∞
⎡⎤
== ∈∈
⎣⎦
P . (1.4)
For the non-homogeneous case, we obtain:
[]
() lim (,) ;, , ,
ij ij
t
ps Qst ij Ist
→∞
⎡⎤
== ∈∈
⎣⎦
P , (1.5)
P being the transition matrix of the embedded Markov chain of the process.
Furthermore it is necessary to introduce the probability that the process will leave
the state i before or at a time t:
[]
(
)
[
]
1
() - | ,
innn
Ht PT T tJ i
+
== ≤=H (1.6)
[]
(
)
[
]
1
(,) | , .
innn
Hst P T tJ iT s
+
==≤==H (1.7)
From the results of Chapter 3, we know that obviously:
11
() ()and (,) (,)
mm
iij i ij
jj
Ht Qt Hst Qst
==
==
∑∑
. (1.8)
The following probabilities only have sense in the discrete time case and to be
concise, we present first the definition for the homogeneous case and then for the
non-homogeneous one.
Definition 1.2 Matrix B
is defined as follows:
[]
(
)
[
]
11
() , - | ,
ij n n n n
bt PJ jT T tJ i
++
== = ==B (1.9)
[]
(
)
[
]
11
(,) , | , .
ij n n n n
bst PJ jT tJ iT s
++
======B (1.10)
From Definition 1.1 it results that:
(0) 0 if 0,
1 if 1,2,...,
ij
ij
Qt
b(t)
Q(t) Q(t ) t
ij ij
==
⎧
⎪
=
⎨
−− =
⎪
⎩
(1.11)
(, ) 0 if ,
(,)
(,) (, 1) if .
ij
ij
Qss t s
bst
Qst Qst t s
ij ij
==
⎧
⎪
=
⎨
−
−>
⎪
⎩
(1.12)
Definition 1.3
The discrete time conditional distribution functions of the waiting
times given the present and the next states, are given by:
[]
(
)
[
]
11
() - | , ,
ij n n n n
F
tPTTtJiJj
++
== ≤==F (1.13)
[]
(
)
[
]
11
(,) | , , .
ij n n n n
F
st P T t J iJ jT s
++
==≤===F (1.14)
Discrete time SMP and numerical solution 133
Obviously the related probabilities can be obtained by means of the following
formulas:
1
() if 0,
()
( ) if 0,
ij ij ij
ij
Qt / p p
Ft
ij
Ut p
≠
⎧
=
⎨
=
⎩
(1.15)
1
(,) () if () 0,
(,)
( , ) if ( ) 0,
ij ij ij
ij
Qst / ps ps
Fst
ij
Ust ps
≠
⎧
=
⎨
=
⎩
(1.16)
where
11
() ( ,) 1 ,Ut Ust st
=
=∀ .
Now, we can introduce the discrete time semi-Markov process
(
)
(),ZZtt
=
∈
where
{
}
()
() , () max :
Nt n
Z
tJ Nt nTt==≤ representing the state occupied by the
process at time t.
For i,j=1,…,m, the transition probabilities are defined in the following way:
(
)
0
() P
ij t
t Z j | Z i
φ
=
==
(1.17)
for the homogeneous case; for the non-homogeneous case, we have:
(
)
(,) P .
ij t s
st Z j | Z i
φ
=
== (1.18)
They are obtained by solving the following evolution equations:
11
() (1 ()) ( ) ( ),
mt
ij ij i i j
tHtbt
ββ
βϑ
φ
δϑφϑ
==
=− + −
∑∑
(1.19)
11
(,) (1 (,)) (, ) ( ,),
mt
ij ij i i j
s
s
tHstbst
ββ
βϑ
φδ ϑφϑ
==+
=− +
∑∑
(1.20)
where, as usual,
δ
ij
represents the Kronecker symbol.
2 NUMERICAL TREATMENT OF SMP
In this section, we present the numerical solutions of the evolution equation of
continuous time semi-Markov process in homogeneous and non-homogeneous
cases.
The proposed approach uses a general quadrature method and we will prove that
the numerical solution tends to the solution of the evolution equation of the
continuous time HSMP.
It will also be shown that, using a very particular quadrature formula for the
numerical solution of evolution equations of continuous time processes, it is
possible to obtain the evolution equation of discrete time processes.
These results were obtained in Corradi et al (2004) and Janssen-Manca (2001a)
generalizing the classical results on integral equation numerical solutions (see
Baker (1977)).
Let us consider a continuous time SMP of kernel
Q supposed to be differentiable.
First of all we write down the evolution equations of the SMP (see Chapter 3,
relation (10.2) and (21.18)) as follows:
134 Chapter 4
1
0
() (1 ()) ( ) ( )
t
m
ij ij i i j
tHtQtd
ββ
β
φ
δϑφϑϑ
=
=− + −
∑
∫
, (2.1)
1
(,) (1 (,)) (, ) ( ,)
t
m
ij ij i i j
s
s
tHstQstd
ββ
β
φ
δϑφϑϑ
=
=− +
∑
∫
, (2.2)
where
ij
Q
represents the derivative respect to time of
ij
Q .
Each generic quadrature formula can be written as (see Evans (1993)):
,
0
0
() ( ),
kh
k
kl
l
f
tdt w f lh
=
≅
∑
∫
(2.3)
where h is the step length,
,
,, ,
kl
kN kN w
≤
∈ are the weights related to
the quadrature formula (10.1); they are functions of both the end point and the
point in which we compute the function value.
If we set:
() (1 ()) ,
ij ij ij
dt Ht
δ
=
− (2.4)
(,) (1 (,)) ,
ij ij ij
dst Hst
δ
=
− (2.5)
we obtain respectively:
10
() () ( ) (),
mk
ij ij k lj il
l
kh d kh w kh h Q h
τ
τ
φφττ
==
⎛⎞
=+ −
⎜⎟
⎝⎠
∑∑
(2.6)
,,
1
(,) (,) (,) (,)
mk
ij ij u k lj il
lu
uhkh d uhkh w hkhQ uh h
τ
τ
φφττ
==
⎛⎞
=+
⎜⎟
⎝⎠
∑∑
, (2.7)
where h is the step length, w the weights related to the quadrature formulas,
0,,,,ukNukN≤≤≤ ∈
such that and [0, ]
N
hY Y
=
is the integration interval.
Now we proceed showing only the homogeneous case but all the results given for
the homogeneous case hold in both cases. The reader interested in acquiring
more details can refer to Janssen and Manca (2001a) and Corradi et al (2004).
We suppose we have already computed:
111 1
222 2
333 3
(0)()(2) ((1))
(0)()(2) ((1))
(0) ( ) (2 ) (( 1) )
(0)()(2) ((1))
jjj j
jjj j
jjj j
mj mj mj mj
hh kh
hh kh
hh kh
hh kh
φ
φφ φ
φφφ φ
φφφ φ
φφφ φ
−
−
−
−
(2.8)
where
1 kN≤≤ and
(0) (0) (1 (0)) , 1, , , 1, ,
ij ij i ij
H
imjm
φφ δ
==− = =
……
. (2.9)
Then from (2.6) it follows that for a fixed j and
1, ,im
∀
= … :
0
111
() () (0) () ( ) ()
mmk
ij k lj il ij k lj il
ll
kh w kh Q d kh w kh h Q h
τ
τ
φφ φττ
===
⎛⎞
−=+−
⎜⎟
⎝⎠
∑∑∑
. (2.10)
Discrete time SMP and numerical solution 135
From relation (2.8), we get
1011
1
2022
1
0
1
() () (0)
() () (0)
() () (0)
m
jkljl
l
m
jkljl
l
m
mj k lj ml m
l
kh w kh Q c
kh w kh Q c
kh w kh Q c
φφ
φφ
φφ
=
=
=
−=
−=
−=
∑
∑
∑
(2.11)
where, for convenience, the
mic
i
,,1, …
=
represent the second member of
(2.10).
The linear system (2.11) in the unknowns
mikh
ij
,,1),(
~
…=
φ
admits solution if
the coefficient matrix is non-singular.
The following theorem holds:
Theorem 2.1 Assume that
[
]
[
]
:0, , :0,
ij ij
QY Y
φ
→→ (2.12)
and
{
}
1, , ,qNN∈∈…, such that YNh
≤
.
Furthermore let
Nkkhkhh
ijij
k
ij
,,2,1,0),()(
~
)( …=−=
φφξ
(2.13)
where
)(kh
ij
φ
is the solution of (2.1) and )(
~
kh
ij
φ
is the solution of (2.6) and
.)()(
1
∑
=
=
m
i
k
ij
k
hh
ξη
(2.14)
Furthermore, let:
∞<==
≤≤≤
h
w
ww
ku
Nku
N
0
max , (2.15)
0
110
() ( ) () ( ) ( )
mmk
kh
k
ij lj il ku lj il
llu
th kh Q d w khuhQuh
φτττ φ
===
⎡
⎤
=−− −
⎢
⎥
⎣
⎦
∑∑∑
∫
, (2.16)
,)()(
1
∑
=
=
m
i
k
ij
k
hth
σ
(2.17)
),(max)()( hhh
k
Nhq
N
σττ
≤≤
==
(2.18)
1
0
() ().
q
u
u
hh
ξη
−
=
=
∑
(2.19)
Then, if
1
()
ij
Qt c≤
for [0, ],tY∈ we have:
136 Chapter 4
1
1
11
() ()
() exp , , 1, ,
11
N
N
k
NN
mw c kh
hmhwch
hkqqN
mhwc mhwc
τξ
η
+
⎛⎞
≤=+
⎜⎟
−−
⎝⎠
…
(2.20)
given that
1
,, , 1
N
mhw c <
Proof This theorem is a particular case of the one given in Janssen and Manca
(2001a).
Remark 2.1
In general, equation (2.6) cannot be solved exactly. Then if the
values
ˆ
()
ij
kh
φ
give the approximate solution of (2.6) (for 0≥k ), then
10
ˆˆ
() () ( ) ()
mk
k
ij ij k lj il ij
l
kh d kh w kh h Q h
τ
τ
φ
φττς
==
⎛⎞
=+ − +
⎜⎟
⎝⎠
∑∑
, (2.21)
if we suppose that:
ˆ
(0) (0) (0).
ij ij ij
d
φφ
==
Then setting:
ˆ
() () ()
k
ij ij ij
hkhkh
πφ φ
=−
(2.22)
it follows that
{}
0
ˆ
() (0) (0), , 1, , .
ij ij ij
hijm
πφφ
=− ∀∈
…
(2.23)
Using relations (2.6) (2.21) and (2.22), we get for
1≥k ,
10
() ( ) () .
mk
kkk
ij k il lj ij
l
hwQhh
τ
τ
τ
π
τπ ς
−
==
⎛⎞
=+
⎜⎟
⎝⎠
∑∑
(2.24)
Using (2.15) and summing up with respect to the first index, it results that:
1
1101
() () .
mmkm
kk
lj N lj ij
lll
hmwhc h
τ
τ
π
πς
====
≤+
∑∑∑∑
(2.25)
If we set:
1
() (),
m
kk
lj
l
hh
ηπ
=
=
∑
and
1
,
m
kk
ij
l
ρ
ς
=
=
∑
(2.26)
it follows that:
1
0
() ().
k
kk
N
hmwhch
τ
τ
ηρ η
=
≤+
∑
(2.27)
Given the following positions:
1
1
0
max , () ()
k
kk
NNN
kN
hmwhch
τ
τ
τρητ η
≤≤
=
=≤+
∑
, (2.28)
that is:
()
1
11
0
1() ()
k
ku
NNN
u
mw hc h mw hc h
ητ η
−
=
−≤+
∑
, (2.29)
it follows finally that: