Jacques J., Raimondo M. Semi-markov risk models for finance, insurance and reliability
Подождите немного. Документ загружается.
Renewal theory and Markov chains
47
Definition 3.1
(i) A renewal process
(
)
,1
n
Tn≥
is recurrent if
n
X
<
∞
for all n; otherwise it
is called transient.
(ii) A renewal process
()
,1
n
Tn≥ is periodic with period
δ
if the possible
values of the random variables
,1
n
Xn≥
form the denumerable set
{
}
0, , 2 ,
δ
δ
… , and
δ
is the largest such number. Otherwise (that is, if there is no
such strictly positive
δ
), the renewal process is aperiodic.
A direct consequence of this definition is the characterization of the type of a
renewal process, with the help of distribution function F.
Proposition 3.2 A renewal process of distribution function F is
(i) recurrent iff () 1F ∞=,
(ii) transient iff () 1F ∞< ,
(iii) periodic with period (0)
δ
δ
> iff F is constant over intervals
[,( 1)),nn n
δ
δ
+∈Ν, and all its jumps occur at points ,nn
δ
∈
.
If t tends toward
+∞ , relation (3.9) gives:
if ( ) 1,
()
()
if ( ) 1.
1()
F
H
F
F
F
+
∞+∞=
⎧
⎪
+∞ =
+∞
⎨
+
∞<
⎪
−+∞
⎩
(3.15)
Or, equivalently by relation (3.13):
if ( ) 1,
()
1
if ( ) 1.
1()
F
R
F
F
+
∞+∞=
⎧
⎪
+∞ =
⎨
+
∞<
⎪
−+∞
⎩
(3.16)
This proves the next proposition.
Proposition 3.3 A renewal process of distribution F is recurrent or transient,
depending on whether ()H +∞ = +∞ or ()H
+
∞<+∞. In the last case, we have
1
()
1()
R
F
+∞ =
−
+∞
or
()
() .
1()
F
H
F
+∞
+∞ =
−
+∞
(3.17)
The interest of the classification given above will be clearer with the concept of
lifetime of a renewal process.
Definition 3.2 The lifetime of a renewal process
(, 1)
n
Tn≥
is the random
variable L, possibly defective, defined by:
{
}
sup : .
nn
LTT
=
<∞ (3.18)
48 Chapter 2
So, if L =
, this means that there is only a finite number of renewals on [0, ).∞
Also, we define a new random variable N giving the total number of renewals on
[0, )L .
Definition 3.3 The total number of renewals on (0, ),
∞
possibly infinite, is given
by
{
}
sup ( ), 0 .NNtt=≥ (3.19)
In reliability theory, the event
{
}
N
k
=
would mean that the (k+1)th component
introduced in the system would have an infinite lifetime! Also the probability
distribution of N is given by
(
)
01 ()PN F
=
=− +∞, (3.20)
(
)
(
)
1()1(),PN F F
=
=+∞−+∞
(3.21)
and in general, for :k ∈
()( )( )
()1 ().
k
PN k F F== +∞ −+∞ (3.22)
Of course if ( ) 1F +∞ = , we have, a.s.,
N
=
+∞ . (3.23)
In the case of a transient renewal process, we can write, using relation (3.22):
()
[]
()
1
()1 ().
k
k
EN kF F
∞
=
=+∞−+∞
∑
(3.24)
As the function
1
1
x
−
can be written for
1x
<
as a power series:
0
1
1
n
n
x
x
∞
=
=
−
∑
, (3.25)
which is analytical on
(
)
1, 1−+
and thus derivable, we have
1
2
1
1
.
(1 )
n
n
nx
x
∞
−
=
=
−
∑
(3.26)
Writing relation (3.24) under the form
() ( )
[]
1
1
()1 (). ()
k
k
EN F F kF
∞
−
=
=+∞−+∞ +∞
∑
(3.27)
we get, using relation (3.26):
()
()
.
1()
F
EN
F
+∞
=
−
+∞
(3.28)
So, it is possible to compute the mean of the total number of renewals very easily
in the transient case. We can also give the distribution function of L. Indeed, we
have:
() ( )
1
0
,.
nn
n
PL t PT t X
∞
+
=
≤= ≤ =+∞
∑
(3.29)
Renewal theory and Markov chains
49
From the independence of
n
T and
1n
X
+
, we may deduce that:
() ( )
()
1
1() ()1().
n
n
PL t F F t F
∞
=
≤=−+∞+ −+∞
∑
(3.30)
And finally, by equality (3.13):
()
(
)
1()()PL t F Rt
≤
=−+∞ . (3.31)
To compute the mean of the lifetime of the process, we use the following trick
based on the independence of the random variables , 1
n
Xn≥ ; we can
successively write:
(
)
{}
(
)
(
)
{}
(
)
11
1
,
TT
EL ET I EL EI
<∞ <∞
=⋅ + ⋅ (3.32)
()
0
()
()1 ()
()
Ft
FdtELF
F
+∞
⎛⎞
=+∞ − + ⋅+∞
⎜⎟
+∞
⎝⎠
∫
(3.33)
so that
() ( ) ()
0
() () ().EL F Ft dt EL F
+∞
=+∞− +⋅+∞
∫
(3.34)
And finally
() ()
0
1
() ().
1()
E
LFFtdt
F
+∞
=+∞−
−+∞
∫
(3.35)
So, for a transient renewal process, the lifetime is always a.s. finite and has a
finite mean given by relation (3.35).
Example 3.1: The Poisson process
In queueing and risk theory presented in Example 2.1, the classical assumption
for the arrival process is that it constitutes a renewal process where the r.v.
n
X
has as common distribution function:
0, 0,
()
1,0,
x
x
Fx
ex
λ
−
<
⎧
=
⎨
−
≥
⎩
(3.36)
with
λ
being a fixed positive constant.
As ( ) 1F +∞ = , the arrival process is recurrent. For (3.3), it is possible to have the
analytical expression of the successive n-fold convolutions. Indeed, we can
successively write:
()
(2) ( )
0
() 1
t
tx x
F
teedx
λλ
λ
−− −
=−
∫
(3.37)
()
0
t
xt
eedx
λλ
λ
−−
=−
∫
(3.38)
1
tt
ete
λ
λ
λ
−
−
=− − (3.39)
1(1),
t
et
λ
λ
−
=− + (3.40)
and in general:
50 Chapter 2
1
()
0
()
() 1 .
!
n
k
nt
k
t
Ft e
k
λ
λ
−
−
=
=−
∑
(3.41)
Applying result (3.5), we have
()
1
00
() ()
P() 1 1
!!
()
.
!
nn
kk
tt
kk
n
t
tt
Nt n e e
kk
t
e
n
λλ
λ
λ
λ
λ
−
−−
==
−
==− −+
=
∑∑
(3.42)
That is, for all fixed t, the process
(
)
()
N
t is a Poisson process of parameter t
λ
.
The value of the renewal function H follows from relations (3.9) and (3.8):
1
()
()
!
n
t
n
t
Ht ne
n
λ
λ
∞
−
=
=
∑
(3.43)
1
()
(1)!
n
t
n
t
e
n
λ
λ
∞
−
=
=
−
∑
(3.44)
1
1
()
,
(1)!
n
t
n
t
et
n
λ
λ
λ
∞
−
−
=
=
−
∑
(3.45)
or
() .Ht t
λ
=
(3.46)
It follows that for a Poisson renewal process, the renewal function is linear.
As we shall see in section 5, a renewal process may also be characterized by its
renewal function. The Poisson renewal process is the one which has a linear
renewal function.
4 THE RENEWAL EQUATION
Coming back to relation (3.9), we get, using the associative property of the
convolution product:
[]
(2) (3)
(2)
0
() () () ()
()
() (),with () ( ) ( ).
t
Ht Ft F t F t
FF FF t
F
tFHt FHt FtxdHx
=+ + +
=+• + +
=+• •= −
∫
(4.1)
This relation is called the integral equation of renewal theory, or simply the
renewal equation. It can also be written as follows:
0
() () ( ) ( ).
t
Ht Ft Ft xdHx=+ −
∫
(4.2)
As:
() (),
F
Ht H Ft
•
=• (4.3)
Renewal theory and Markov chains
51
we also have:
0
() () ( ) ( ).
t
Ht Ft Ht xdFx=+ −
∫
(4.4)
In the particular case where the density function f of F exists, the last integral
equation becomes
0
() () ( ) ( ) .
t
Ht Ft Ht x f xdx=+ −
∫
(4.5)
In this case, we can apply the dominated convergence to series (3.9), showing the
existence of the density function of H, say h, such that:
[]
1
() (),
n
n
ht f t
∞
=
=
∑
(4.6)
with
[]
1
() (),
f
tft= (4.7)
[]
2
0
() ( ) ( ) ,
t
f
tftxfxdx=−
∫
(4.8)
[] [ ]
1
0
() ( ) ( ) .
t
nn
f
tftxfxdx
−
=−
∫
(4.9)
From relation (4.5) or (4.6) we obtain the integral equation for
h:
() () (),
ht f t f ht
=
+⊗ (4.10)
with
0
() ( )() .
t
f
ht f t xhxdx⊗= −
∫
(4.11)
As:
() (),
f
ht h f t
⊗
=⊗ (4.12)
we also have:
() () ().ht f t h f t
=
+⊗ (4.13)
In fact, the renewal equation (4.2) is one particular case of the type of integral
equations:
() () (),
X
tGtXFt
=
+• (4.14)
where X is the unknown function, F and G being known measurable functions
bounded on finite intervals and
•
the convolution product.
Such an integral equation is said to be of renewal type.
When
FG = , we get the renewal equation. The study of these integral equations
has a long history which includes contributions from Lotka (1940), Feller (1941),
Smith (1954) and Çinlar (1969). Çinlar gave the two following propositions.
52 Chapter 2
Proposition 4.1 (Existence and unicity)
The integral equation of renewal type (4.14) has one and only one solution, given
by
() ()
X
tRGt
=
• , (4.15)
R being defined by relation (3.13).
It is also possible to study the asymptotic behaviour of solutions to renewal-type
equations. The basic result is the so-called “key renewal theorem”, proven by
W.L. Smith (1954), and which is in fact mathematically equivalent to
Blackwell’s theorem (1948), given here as
Corollary 4.3.
A proof of the key renewal theorem using Blackwell’s theorem can be found in
Çinlar (1975b).
Proposition 4.2 (Asymptotic behaviour, Key renewal theorem)
(i) In the transient case, we have:
lim ( ) ( ) ( )
t
Xt R G
→∞
=
∞∞
(4.16)
provided the limit
() lim()
t
GGt
∞
= (4.17)
exists.
(ii) In the case of recurrence, we have:
0
1
lim ( ) ( ) ,
t
X
tGxdx
m
∞
→∞
=
∫
(4.18)
provided that G is directly Riemann integrable on [0, ),
∞
that F is not arithmetic,
and supposing:
()( )
0
1()
n
mEX Fxdx
∞
==−
∫
. (4.19)
Corollary 4.1 In the case of a recurrent renewal process with finite variance
2
σ
,
we have:
22
2
lim ( )
2
t
tm
Rt
m
m
σ
→∞
+
⎛⎞
−=
⎜⎟
⎝⎠
. (4.20)
:
Remark 4.1
1) From result (4.20), we immediately get an analogous result for the renewal
function H. Indeed, we know, from relation (3.14), that
0
() () (), 0.Rt Ht U t t
=
+≥
(4.21)
Applying result (4.20), we get:
22
2
lim ( ) 1,
2
t
tm
Ht
m
m
σ
→∞
+
⎛⎞
−
=−
⎜⎟
⎝⎠
(4.22)
Renewal theory and Markov chains 53
or
22
2
lim ( ) .
2
t
tm
Ht
m
m
σ
→∞
−
⎛⎞
−=
⎜⎟
⎝⎠
(4.23)
2) The two results (4.20) and (4.23) are often written under the following forms:
22
2
() O(1)
2
tm
Rt
m
m
σ
+
=+ + , (4.24)
22
2
() O(1),
2
tm
Ht
m
m
σ
−
=+ + (4.25)
where O(1) represents a function of t approaching zero as t approaches infinity.
Corollary 4.2 In the case of a recurrent renewal process with finite mean m, we
have:
() 1
lim .
t
Rt
tm
→∞
= (4.26)
Corollary 4.3 (Blackwell’s theorem)
In the case of a recurrent process with finite mean m, we have, for every positive
number
τ
:
()
lim ( ) ( ) .
t
Rt Rt
m
τ
τ
→∞
−−= (4.27)
Remarks 4.2
1) Probabilistic interpretation of renewal density.
Let ( )ktdt represent the probability that there is a renewal in the time
interval ( , )tt dt+ . It must satisfy the following relation, obtained by a simple
probabilistic argument using the independence property of the successive
“lifetimes”:
0
() () ( ) ( ) .
t
k t dt f t dt f x k t x dt=+ −
∫
(4.28)
From the unicity part of
Proposition 4.1, we get, for all 0t ≥ :
() ().kt ht
=
(4.29)
So, the probability defined above is given by ( )htdt and more generally by
()dH t with a precision error of O( )dt . This interpretation often simplifies the
search for relations useful in renewal theory.
2) The variance of ()
N
t
From Stein’s lemma, we know that ( )
N
t has, for all t, moments of any order.
Also, let
2
()t
α
be the centred moment of order 2 of ( )
N
t :
()
(
)
2
2
() () .tENt
α
= (4.30)
54 Chapter 2
From results (3.5), we can write successively:
()
()
2
20
1
() ()
k
tkUFFt
ν
α
∞
=
=−•
∑
(4.31)
2() 2(1)
11
() ()
kk
kk
kF t kF t
∞∞
+
==
=−
∑∑
(4.32)
2() 2 ()
11
() ( 1) ()
k
k
kF t F t
ν
ν
ν
∞∞
==
=−−
∑∑
(4.33)
[]
()
22
1
(1) ()
F
t
ν
ν
νν
∞
=
=−−
∑
(4.34)
()
1
(2 1) ( )
F
t
ν
ν
ν
∞
=
=−
∑
(4.35)
()
()
11
2( 1) () ().
F
tFt
ν
ν
νν
ν
∞∞
==
=− +
∑∑
(4.36)
Now if we compute
(2)
()Ht by means of the relation:
() () ( )
2'
1'1
() () () ,Ht Ft F t
νν
νν
∞∞
==
⎛⎞⎛ ⎞
=•
⎜⎟⎜ ⎟
⎝⎠⎝ ⎠
∑∑
(4.37)
we easily find:
() ()
2
1
() ( 1) ().
H
tFt
ν
ν
ν
∞
=
=−
∑
(4.38)
Using (3.9) and substituting this last result in relation (4.36), we finally obtain:
()
2
2
() () 2 ().tHt Ht
α
=+
(4.39)
This last result shows that
()
()
()
2
2
Var () () 2 () () .Nt Ht H t Ht=+ − (4.40)
Remark 4.3
The renewal equation (4.4) can be solved, as it will be shown in the previous
sections, directly in some very special cases or by means of the Laplace or
Laplace-Stieltjes transforms in other cases.
In this way the analytical solution of the integral equation (4.4) can be obtained.
But in the largest part of real life applications the equation that is obtained by the
model can’t be solved by analytical methods. In these cases it is necessary to use
a numerical approach to get the solution to the general renewal equation (4.4) in
a bounded horizon time ( see Janssen and Manca(2006), Chapter 2, section 10)
Renewal theory and Markov chains
55
5 THE USE OF LAPLACE TRANSFORM
5.1 The Laplace Transform
To show the power of the Laplace transform in solving the renewal equation, let
us suppose that d.f. F characterizing the recurrent renewal process considered has
f
as density.
Let us use the following general notation: for any function
α
on [0, ),
α
∞
will
represent its Laplace transform, provided it exists:
()
()
0
() () , £ () .
sx
sexdx x
αα α
∞
−
==
∫
(5.1)
With this convention and using the well-known property of the Laplace
transform
()
£ ( )( ) ( ) ( ),
x
ss
αβ α β
⊗=⋅
(5.2)
we get from the renewal equation:
() () () ()hs f s hs f s=+⋅
. (5.3)
The Laplace transform of the unique solution is thus given by:
()
() .
1()
f
s
hs
f
s
=
−
(5.4)
Using the inverse Laplace transform, we can then have the value of ( )ht .
Remark 5.1 From the algebraic equation (5.3), we can deduce the expression of
the density
f
as a function of the renewal density h . In Laplace transforms, we
get:
()
() .
1()
hs
fs
hs
=
−
(5.5)
The inverse Laplace transform gives us a function of h .
This leads to the important result that every renewal process is characterized by
its renewal density, if it exists, or by its renewal function. Thus there is a one-to-
one correspondence between the d.f.
F
of a renewal function and its renewal
function H .
5.2 The Laplace-Stieltjes (L-S) Transform
The L-S transform is a bit more general than the Laplace transform. Indeed, for
any function
α
on [0, ),
α
∞ will represent its L-S transform, provided it exists,
under the form:
56 Chapter 2
()
()
0
() () £ () .
sx
sedx x
αα α
∞
−
==
∫
(5.6)
For a function
α
such that
lim ( ) 0 ,
sx
x
ex
α
−
→∞
= (5.7)
an integration by parts gives the relation between
α
and
α
:
() (0) ().
s
ss
α
αα
=
−+
(5.8)
As
£ satisfies the following property:
(
)
(
)
(
)
£( )() £ () £ () ,xxx
αβ α β
•= ⋅ (5.9)
we get from the renewal equation (4.4):
() () () ().Hs Fs Hs Fs=+⋅
(5.10)
Or:
()
() ,
1()
Fs
Hs
F
s
=
−
(5.11)
which is equivalent to the expression (5.4) if we do not assume the existence of a
density for
F
. Of course, if such a density exists, we have from (5.6):
() (),
F
sfs=
(5.12)
() (),Hs hs=
(5.13)
and consequently, the relations (5.11) and (5.4) are identical.
6 APPLICATION OF WALD’S IDENTITY
We already know that
(
)
,
n
E
Snm
=
(6.1)
as
0nn
SX X
=
++ (6.2)
and
(
)
() ().
E
Nt Rt
=
(6.3)
Now let us consider the time of the first “replacement” after time
t ; it is given
by
'( )Nt
S . Wald’s lemma (see Chapter 1, relation (6.52)) computes the mean of
this random variable.
Proposition 6.1 (Wald’s lemma)
(
)
'( )
().
Nt
E
SmRt= (6.4)
Remark 6.1 As, by (3.10):
'( ) ( ) 1 ,Nt Nt
=
+ (6.5)
we can use relation (3.14) to write result (6.4) as:
(
)
[
]
() 1
() 1 .
Nt
ES mHt
+
=+ (6.6)