Schulz M. Control Theory in Physics and Other Fields of Science: Concepts, Tools, and Applications

Подождите немного. Документ загружается.

8.2 Gaussian Processes 217

in the Fourier transform. Thus, after rescaling k →

−1/2

the cumulant

expansion reads

ˆp





=exp



∞



n=1

(n)

1−n/2







. (8.16)

Apart from the ﬁrst cumulant, we ﬁnd that the second cumulant remains

invariant while all higher cumulants approach zero as R →∞. Thus, only

the ﬁrst and the second cumulants will remain for suﬃciently large R and the

probability distribution function p

(ξ) approaches a Gaussian function. The

result of our naive argumentation is the central limit theorem. The precise

formulation of this important theorem is:

The sum, normalized by R

−1/2

of R random independent and identi-

cally distributed states of zero mean and ﬁnite variance, is a random

variable with a probability distribution function converging to the

Gaussian distribution with the same variance. The convergence is to

be understood in the sense of a limit in probability, i.e., the probability

that the normalized sum has a value within a given interval converges

to that calculated from the Gaussian distribution.

We will now give a more precisely derivation of the central limit theorem.

Formal proofs of the theorem may be found in probability textbooks such as

Feller [18, 29, 30]. Here we follow a more physically motivated way by Sornette

[31], using the technique of the renormalization group theory.

This powerful method [32] introduced in ﬁeld theory and in critical phase

transitions is a very general mathematical tool, which allows one to decompose

the problem of ﬁnding the collective behavior of a large number of elements

on large spatial scales and for long times into a succession of simpler problems

with a decreasing number of elements, whose eﬀective properties vary with

the scale of observation. In the context of the central limit theorem, these

elements refer to the elementary N-component events ξ

The renormalization group theory works best when the problem is domi-

nated by one characteristic scale which diverges at the so-called critical point.

The distance to this criticality is usually determined by a control parame-

ter which may be identiﬁed in our special case as R

−1

. Close to the critical

point, a universal behavior becomes observable, which is related to typical

phenomena like scale invariance of self-similarity. As we will see below, the

form stability of the Gaussian probability distribution function is such a kind

of self-similarity.

The renormalization consists of an iterative application of decimation and

rescaling steps. The ﬁrst step is to reduce the number of elements to transform

the problem in a simpler one. We use the thesis that under certain conditions

the knowledge of all the cumulants is equivalent to the knowledge of the

probability density. So we can write

p (ξ

)=f



(1)

(2)

,...,c

(m)

,...



, (8.17)

218 8 Filters and Predictors

where f is a unique function of ξ

and the inﬁnite set of all cumulants



(1)

(2)

,...



. Every distribution function can be expressed by the same func-

tion in this way, however with diﬀerences in the inﬁnite set of parameters.

The probability distribution function p

(ξ) may be the convolution of

R =2

identical distribution functions p (ξ

). This speciﬁc choice of R is not

a restriction since we are interested in the limit of large R and the way with

which we reach this limit is irrelevant. We denote the result of the 2

-fold

convolution as

(ξ)=f

(l)



ξ,c

(1)

(2)

,...,c

(m)

,...



. (8.18)

Furthermore, we can also calculate ﬁrst the convolution between two identical

elementary probability distributions

(ξ)=



p (ξ − ξ



) p (ξ



)dξ



, (8.19)

which leads because of the general relation (8.13) to the formal structure

(ξ)=f



ξ,2c

(1)

, 2c

(2)

,...,2c

(m)

,...



(8.20)

with the same function f as used in (8.17). With this knowledge we are able

to generate p

(ξ) also from p

(ξ)bya2

l−1

-fold convolution

(ξ)=f

(l−1)



ξ,2c

(1)

, 2c

(2)

,...,2c

(m)

,...



. (8.21)

Here, we see the eﬀect of the decimation. The new convolution considers

only 2

l−1

events. The decimation itself corresponds to the pairing due to the

convolution (8.19) between two identical elementary probability distributions

The notation of the scale is inherent to the probability distribution func-

tion. The new elementary probability distribution function p

(ξ) obtained

from (8.19) may display diﬀerences to the probability density we started from.

We compensate for this by the scale factor λ

−1

for ξ. This leads to the rescal-

ing step ξ → λ

−1

ξ of the renormalization group which is necessary to keep

the reference scale.

With the rescaling of the components of the vector ξ, the cumulants are

also rescaled and each cumulant of order m has to be multiplied by the factor

−m

. This is a direct consequence of (6.55) because it demonstrates that the

cumulants of order m have the dimension |k|

−m

and |ξ|

, respectively. The

conservation of the probabilities p (ξ)dξ = p (ξ



)dξ



introduces a prefactor

−N

as a consequence of the change of the N-dimensional vector ξ → ξ



.We

thus obtain from (8.21)

(ξ)=λ

−N

(l−1)



(1)

(2)

,...,

(m)

, ...



. (8.22)

The successive repeating of both decimation and the rescaling leads after l

steps to

8.2 Gaussian Processes 219

(ξ)=λ

−lN

(0)



(1)

(2)

,...,

(m)

,...



. (8.23)

As mentioned above, f

(l)

(ξ,...c

(m)

,...) is a function which is obtainable from

a convolution of 2

identical functions f(ξ,...c

(m)

,...). In this sense we obtain

the matching condition f

(0)

≡ f so that we arrive at

(ξ)=λ

−lN



(1)

(2)

,...,

(m)

,...



. (8.24)

Finally we have to ﬁx the scale λ. We see from (8.24) that the particular

choice λ =2

1/m

makes the prefactor of the m

-th cumulant equal to 1 while

all higher cumulants decrease to zero as l = log

R →∞. The lower cumulants

diverge with R

(1−m/m

)

, where m<m

The only reasonable choice is m

= 2 because λ =

√

2 keeps the probability

distribution function in a window with constant width. In this case, only the

ﬁrst cumulant may remain divergent for R →∞. As mentioned above, this

eﬀect can be eliminated by a suitable shift of ξ. Thus we arrive at

lim

R→∞

(ξ)=R

−N/2



√

(1)

√

R, c

(2)

, 0,...,0,...



(8.25)

In particular, if we come back to our original problem, we have thus obtained

the asymptotic result that the probability distribution of the sum over in-

coming stochastic events has only its two ﬁrst cumulant nonzero. Hence, the

corresponding probability density is a Gaussian law.

If we return to the original scales, the ﬁnal Gaussian probability distribu-

tion function p

(ξ) is characterized by the mean ξ = Rc

(1)

and the covariance

matrix ˜σ = Rc

(2)

, where c

(1)

and c

(2)

are the ﬁrst two cumulants of the ele-

mentary probability density. Hence, we obtain

lim

R→∞

(ξ)=

(2π)

N/2

√

det ˜σ

exp



−



ξ −



˜σ

−1



ξ −





(8.26)

or with the rescaled and shifted states

lim

R→∞





(2π)

N/2

√

det c

(2)

exp



−



(2)



−1



. (8.27)

The quantity

ξ is simply the sum, normalized by R

−1/2

of R random inde-

pendent and identically distributed events of zero mean and ﬁnite variance,

ξ =

ξ −

√



j=1



− c

(1)



. (8.28)

In other words, (8.27) is the mathematical formulation of central limit theo-

rem. The Gaussian distribution function itself is a ﬁxed point of the convo-

lution procedure in the space of functions in the sense that it is form stable

under the renormalization group approach. Notice that form stability or alter-

natively self-similarity means that the resulting Gaussian function is identical

220 8 Filters and Predictors

to the initial Gaussian function after an appropriate shift and a rescaling of

the variables.

We remark that the convergence to a Gaussian behavior also holds if the

initially variables have diﬀerent probability distribution functions with ﬁnite

variance of the same order of magnitude. The generalized ﬁxed point is now

the Gaussian law (8.26) with

ξ =



n=1

(1)

and ˜σ =



n=1

(2)

, (8.29)

where c

(1)

and c

(2)

are the mean trend vector and the covariance matrix,

respectively, obtained from the now time-dependent elementary probability

distribution function p

(j)

(ξ

Finally, it should be remarked that the two conditions of the central limit

theorem may be partially relaxed. The ﬁrst condition under which this the-

orem holds is the Markov property. This strict condition can, however, be

weakened, and the central limit theorem still holds for weakly correlated vari-

ables under certain conditions. The second condition that the variance of the

variables be ﬁnite can be somewhat relaxed to include probability functions

with algebraic tails |ξ|

−3

. In this case, the normalizing factor is no longer

−1/2

but can contain logarithmic corrections.

8.2.2 Convergence Problems

As a consequence of the renormalization group analysis, the central limit

theorem is applicable in a strict sense only in the limit of inﬁnite R. But,

in practice, the Gaussian shape is a good approximation of the center of a

probability distribution function if R is suﬃciently large. It is important to

realize that large deviations can occur in the tail of the probability distribution

function p

(ξ), whose weight shrinks as R increases. The center is a region

of width at least of the order of

√

R around the average ξ = Rc

(1)

Let us make more precise what the center of a probability distribution

function means. For the sake of simplicity we investigate events of only one

component; i.e., ξ is now again a scalar quantity. As before, ξ is the sum of

R identicales distributed variables ξ

with mean c

(1)

, variance c

(2)

, and ﬁnite

higher cumulants c

(m)

. Thus, the central limit theorem reads

lim

R→∞

(x)=

√

2π

exp



−



, (8.30)

where we have introduced the reduced variable

x =

√

(2)

ξ − Rc

(1)

√

(2)

. (8.31)

In order to analyze the convergence behavior for the tails [34], we start from

the probability

8.2 Gaussian Processes 221

(R)

(z)=P

(R)

(x>z)=

∞



(x)dx (8.32)

and analyze the diﬀerence ∆P

(R)

(z)=P

(R)

(z) − P

(∞)

(z), where P

(∞)

(z)

is simply the complementary error function due to (8.30). If all cumulants

are ﬁnite, one can develop a systematic expansion in powers of R

−1/2

of the

diﬀerence ∆P

(R)

(z)[33]:

∆P

(R)

(z)=

exp



−z



√

2π



(z)

1/2

(z)

···+

(z)

m/2

···



, (8.33)

where Q

(z) are polynomials in z, the coeﬃcients of which depend on the

ﬁrst m + 2 normalized cumulants of the elementary probability distribution

function, λ

= c

(k)

/[c

(2)

]

k/2

. The explicit form of these polynomials can be

obtained from the textbook of Gnedenko and Kolmogorov [34]. The two ﬁrst

polynomials are

(z)=



1 − z



(8.34)

and

(z)=



−

5λ





5λ

−



. (8.35)

If the elementary probability distribution function has a Gaussian behavior,

all its cumulants c

(m)

of order larger than 2 vanish identically. Therefore, all

(z) are also zero and the probability density p

(x) is a Gaussian.

For an arbitrary asymmetric probability distribution function, the skew-

ness λ

is nonvanishing in general and the leading correction is Q

(z). The

Gaussian law is valid if the relative error



∆P

(R)

(z)



(∞)

(z) is small com-

pared to 1. Since the error increases with z, the Gaussian behavior becomes

observable at ﬁrst close to the central tendency. The necessity condition

|λ

|R

1/2

follows directly from



∆P

(R)

(z)



(∞)

(z)  1forz → 0.

For large z, the approximation of p

(x) by a Gaussian law remains valid

if the relative error remains small compared to 1. Here, we may replace

the complementary error function P

(∞)

(z) by its asymptotic representation

exp



−z



√

2πz). We thus obtain the inequality |zQ

(z)|R

1/2

leading



 R

1/2

. Because of (8.31), this relation is equivalent to the condition



ξ − Rc

(1)



|λ

−1/3

σR

2/3

. (8.36)

It means that the Gaussian law holds in a region of an order of magnitude of



ξ − Rc

(1)



|λ

−1/3

σR

2/3

around the central tendency.

A symmetric probability distribution function has a vanishing skewness so

that the excess kurtosis λ

= c

(4)

/σ

provides the leading correction to the

central limit theorem. The Gaussian law is now valid if λ

 R and

222 8 Filters and Predictors



ξ − Rc

(1)



|λ

−1/4

σR

3/4

, (8.37)

i.e., the central region in which the Gaussian law holds is now of an order of

magnitude R

3/4

Another class of inequalities describing the convergence behavior with re-

spect to the central limit theorem was found by Berry [35]andEss´een [36].

The Berry–Ess´een theorems [37] provide inequalities controlling the absolute

diﬀerence



∆P

(R)

(z)



. Suppose the variance c

(2)

and the average

η =





ξ − c

(1)



p (ξ)dξ (8.38)

are ﬁnite quantities, then the ﬁrst theorem reads



∆P

(R)

(z)



≤

3η



(2)

3/2

√

. (8.39)

The second theorem is the extension to not identically by distributed variables.

Here, we have to replace the constant values of c

(2)

and η by

(2)



j=1

(2)

(8.40)

and

η =



j=1

, (8.41)

where c

(2)

and η

are obtained from the individual elementary probability

distribution functions p

(j)

(ξ

). Then, the following inequality holds



∆P

(R)

(z)



≤



(2)

3/2

√

. (8.42)

Notice that the Berry–Ess´een theorems are less stringent than the results

obtained from the cumulant expansion (8.33). We see that the central limit

theorem gives no information about the behavior of the tails for ﬁnite R.

Only the center is well-approximated by the Gaussian law. The width of the

central region depends on the detailed properties of the elementary probability

distribution functions.

The Gaussian probability distribution function is the ﬁxed point or the

attractor of a well-deﬁned class of functions. This class is also denoted as the

basin of attraction with respect to the corresponding functional space. When

R increases, the functions p

(ξ) become progressively closer to the Gaussian

attractor. As discussed above, this process is not uniform. The convergence

is faster close to the center than in the tails of the probability distribution

function.

8.3 L´evy Processes 223

8.3 L´evy Processes

8.3.1 Form-Stable Limit Distributions

While we had derived the central limit theorem, we saw that the probabil-

ity density function p

(ξ) of the accumulated events could be expressed as a

generalized convolution (8.12) of the elementary probability distribution func-

tions p (ξ). We want to use this equation in order to determine the set of all

form-stable probability distribution functions. A probability density p

(ξ)is

called a form-stable function if it can be represented by a function g,which

is independent from the number R of convolutions,

(ξ)dξ = g(ξ



)dξ



, (8.43)

where the variables are connected by the linear relation ξ



= α

ξ + β

.Be-

cause the vector ξ has the dimension N,theN × N matrix α

describes an

appropriate rotation and dilation of the coordinates while the N -component

vector β

corresponds to a global translation of the coordinate system. Within

the formalism of the renormalization group, a form-stable probability density

law corresponds to a ﬁxed point of the convolution procedure. The Fourier

transform of g is given by

ˆg(k)=



g(ξ



ikξ



dξ





(ξ)e

ik(α

ξ+β

)

dξ

ikβ

ˆp

(α

k) , (8.44)

where we have used deﬁnition (6.50) of the characteristic function. The form

stability requires that this relation must be fulﬁlled for all values of R.In

particular, we obtain

ˆp

(k)=ˆg(α

−1

k)e

−iβ

−1

and ˆp(k)=ˆg(α

−1

k)e

−iβ

−1

. (8.45)

Without any restriction, we can choose α

=1andβ

= 0. The substitution

of (8.45) into the convolution formula (8.13) yields now

ˆg(α

−1

k)e

−iβ

−1

=ˆg

(k) . (8.46)

Let us write

ˆg(k) = exp {Φ(k)} , (8.47)

where Φ(k) is the cumulant generating function. Thus (8.46) can be written

Φ(α

−1

k) − iβ

−1

k = RΦ(k) (8.48)

and after splitting oﬀ the contributions linearly in k

Φ(k)=iuk + ϕ (k) , (8.49)

we arrive at the two relations,

= α



−1

− R

(8.50)

224 8 Filters and Predictors

and

ϕ(α

−1

k)=Rϕ(k) . (8.51)

The ﬁrst equation gives simply the total shift of the center of the probability

distribution function resulting from R convolution steps. As discussed in the

context of the central limit theorem, the drift term can be put to zero by a

suitable linear change of the variables ξ.Thus,β

is no object of the fur-

ther discussion. Second equation (8.51) is the true key for our analysis of the

form stability. In the following investigation we restrict ourselves again to the

one-variable case. The mathematical handling of the multidimensional case is

similar, but the large number of possible degrees of freedom complicates the

discussion.

The relation (8.51) requires that ϕ(k) is a homogeneous function, ϕ(λk)=

ϕ(k) with the homogeneity coeﬃcient γ. Considering that α

must be a

real quantity, we obtain a

= R

−1/γ

. Consequently, the function ϕ has the

general structure

ϕ (k)=c

|k|

+ c

−

k |k|

γ−1

(8.52)

with the three parameters c

, c

−

,andγ =1.

A special solution occurs for γ = 1, because in this case ϕ(k) merges

with the separated linear contributions. Here, we obtain the special structure

ϕ (k)=c

|k| + c

−

k ln |k|. The rescaling k → λk leads then to ϕ(λk)=

λϕ(k)+c

−

k ln λ and the additional term c

−

ln λ maybeabsorbedintheshift

coeﬃcient β

It is convenient to use the more common representation [38, 39]

ˆg(k)=L

a,b

(k) = exp



−a |k|



1+ib tan



πγ



|k|



(8.53)

with γ =1.Forγ = 1, tan (πγ/2) must be replaced by (2/π)ln|k|. A more

detailed analysis [38, 40] shows that ˆg(k) is a characteristic function of a

probability distribution function if and only if a is a positive scale factor, γ is

a positive exponent, and the asymmetry parameter satisﬁes |b|≤1.

Apart from the drift term, (8.53) is the representation of any characteristic

function corresponding to a probability density which is form-invariant under

the convolution procedure. The set of these functions is known as the class

of L´evy functions. Obviously, the Gaussian law is a special subclass. The

L´evy functions are fully characterized by the expression of their characteristic

functions (8.53). Thus, the inverse Fourier transform of (8.53) should lead to

the real L´evy functions L

a,b

(ξ).

Unfortunately, there are no simple analytic expressions of the L´evy func-

tions except for a few special cases, namely the Gaussian law (γ = 2), the

L´evy–Smirnow law (γ =1/2, b =1)

1/2

a,1

(ξ)=

√

π (2ξ)

3/2

exp



−

2ξ



for ξ>0 (8.54)

8.3 L´evy Processes 225

and the Cauchy law (γ =1,b =0)

a,0

(ξ)=

+ ξ

, (8.55)

which is also known as Lorentzian. One of the most important properties of

the L´evy functions is their asymptotic power law behavior. A symmetric L´evy

function (b = 0) centered at zero is completely deﬁned by the Fourier integral

a,0

(ξ)=

∞



exp {−a |k|

}cos(kξ)dk. (8.56)

This integral can be written as a series expansion valid for |ξ|→∞

a,0

(ξ)=−

∞



n=1

(−a)

|ξ|

γn+1

Γ (γn +1)

Γ (n +1)

sin



πγn



. (8.57)

The leading term deﬁnes the asymptotic dependence

a,0

(ξ) ∼

|ξ|

1+γ

. (8.58)

Here, C = aγΓ (γ)sin(πγ/2) /π is a positive constant called the tail and the

exponent γ is between 0 and 2. The condition γ<2 is necessary because a

L´evy function with γ>2 is unstable and converges to the Gaussian law. We

will discuss this behavior below.

L´evy laws can also be asymmetric. Then we have the asymptotic behavior

a,b

(ξ) ∼ C

/ξ

1+γ

for ξ →∞and L

a,b

(ξ) ∼ C

−

/ |ξ|

1+γ

for ξ →−∞and

the asymmetry is quantiﬁed by the asymmetry parameter b via

b =

− C

−

+ C

−

. (8.59)

The completely antisymmetric cases correspond to b = ±1. For b =+1and

γ<1 the variable ξ takes only positive values while for b = −1andγ<1

the variable ξ is deﬁned to be negative. For 1 <γ<2andb =1theL´evy

distribution is a power law ξ

−γ−1

for ξ →∞while the function converges

to zero for ξ →−∞as exp



−|ξ|

γ/(γ−1)



. The inverse situation occurs for

b = −1.

All L´evy functions with the same exponent γ and the same asymmetry

coeﬃcient b are related by the scaling law

a,b

(ξ)=a

−1/γ

1,b



−1/γ



. (8.60)

Therefore we obtain

|ξ|



|ξ|

a,b

(ξ)dξ = a

θ/γ



|ξ



1,b

(ξ



)dξ



(8.61)

if the integrals in (8.61) exist. An important property of all L´evy distribu-

tions is that the variance is inﬁnite. This behavior follows directly from the

226 8 Filters and Predictors

substitution of (8.53) into (6.52). Roughly speaking, the L´evy law does not

decay suﬃciently rapidly at |ξ|→∞as it will be necessary for the integral

(6.49) to converge. However, the absolute value of the spread (6.46) exists and

suggests a characteristic scale of the ﬂuctuations D

(t) ∼ a

1/γ

. When γ ≤ 1

even the mean and the average of the absolute value of the spread diverge.

The characteristic scale of the ﬂuctuations may be obtained from (8.61)via



|ξ|



1/θ

∼ a

1/γ

for a suﬃciently small exponent θ. We remark that also for

γ ≤ 1 the median and the most probable value still exist.

8.3.2 Convergence to Stable L´evy Distributions

The Gaussian probability distribution function is not only a form-stable dis-

tribution, it is also the ﬁxed point of the classical central limit theorem. In

particular, it is the attractor of all the distribution functions having a ﬁnite

variance. On the other hand, the Gaussian law is a special distribution of the

form-stable class of L´evy distributions. It is then natural to ask if all other

L´evy distributions are also attractors in the functional space of probability

distribution functions with respect to the convolution procedure (Fig. 8.1).

stable Levy

unstable

Levy

Gaussian

γ =2

γ >2

<2γ

Fig. 8.1. The schematic convergence behavior of probability distribution functions

in the functional space. The Gaussian law separates stable and unstable L´evy laws

There is a bipartite situation. Upon R convolutions, all probability distri-

bution functions p (ξ) with an asymptotic behavior p (ξ) ∼ C

|ξ|

−1−γ

and

with γ

< 2 are attracted to a stable L´evy distribution. In case of asymp-

totically symmetric functions, C

= C

−

= C and γ

= γ

−

= γ, the ﬁxed

point is the symmetric L´evy law with the exponent γ and the scale parameter

a ∼ RC.