Troyan V., Kiselev Y. Statistical Methods of Geophysical Data Processing

Подождите немного. Документ загружается.

Elements of applied functional analysis 283

is given a priori information ϕ ∈ N(ϕ

, K

); this information can be presented

formally in the form of the direct measurement of the ﬁeld ϕ:

= Iϕ + η, η ∈ N (0, K

We note that in the general case the direct measurement model is written as

= P

ϕ+ε

, where P

is a projector, which cuts out ϕ values at the isolated points

of the parametric space, for example, at the isolated space-time points.

As we demonstrated above, the indirect measurement u = Lϕ + ε generates the

normal distribution in the ϕ space

(ϕ) ∼ exp



−

(||ϕ − ˆϕ||

∗

−1

)



ˆϕ = (L

∗

−1

∗

and the direct (a priori) measurement are described by the normal distribution

density

(ϕ) ∼ exp



−

||ϕ − ϕ

−1



The information, obtained from the indirect and direct measurement, is considered

as independent, therefore the joint estimate ϕ is determined by the product of the

probability densities. Then

˜ϕ = arg inf(||ϕ − ˆϕ||

+ ||ϕ − ϕ

), (9.32)

here the Fisher’s operators F

and F

determine the amount of information, ob-

tained accordingly in the indirect measurement (F

= = L

∗

−1

L) and the amount

of a priori information (“direct” measurement): F

= K

−1

. Euler equation for the

extremum problem (9.32) has the form

+ F

) ˜ϕ = F

ˆϕ + F

˜ϕ = (F

+ F

)

−1

ˆϕ + F

)

= (L

∗

−1

L + K

−1

)

−1

∗

−1

u + K

−1

)

= ϕ

+ K

∗

(LK

∗

+ K

)

−1

(u − Lϕ

˜ϕ − ϕ

= K

∗

(LK

∗

+ K

)

−1

(u − Lϕ

i.e. we obtain the estimate of the standard statistical regularization for the model

L(ϕ − ϕ

) + ε = u − Lϕ

Let us consider the general scheme of the integration of the M measurements

(indirect and direct) with using a priori information

ϕ + ε

= u

ϕ + ε

= u

. . .

ϕ + ε

= u

M+1

ϕ + ε

M+1

= u

M+1

≡ ϕ

284 STATISTICAL METHODS OF GEOPHYSICAL DATA PROCESSING

where for the indirect measurement the L

is an integral operator, for the direct

measurement the L

= P

is an projector, such that P = P

∗

, P = P

. If ε

∈

N(0, K

), i = 1 ÷ M + 1, then the optimal estimate of the ﬁeld ϕ, has the form

˜ϕ = F

−1



M+1

i=1



Here the operator F

−1

, is an inverse one in relation to the Fisher operator

F =

M+1

i=1

∗

−1

) + IK

−1

It determines the error correlator of the solution ˜ϕ:

E(δϕδϕ

∗

) = F

−1

Thus, the integration procedure should include a priori information or direct

measurement as far as only in this case it is possible to get the stable solution. It

should be noted that the direct measurement is able to substitute a priori informa-

tion if they envelop the measurement of the whole ﬁeld ϕ, but not the measurement

at isolated points.

The considered statistical interpretation of the measurement integration permits

to write the problem of the recovery of the non-stationary ﬁeld ϕ

, described of the

system state. In this case the role of a priori information can fulﬁll the equation,

described the dynamic of the ﬁeld ϕ

+ η

= g, η

∈ N(0, K

= σ

δ(t − t

)

(g is a source function) provided that the boundedness of the inverse dynamic

operator D

We note that in the particular case, if the dynamic operator has the form D

(∂/∂t + A(t)), were A(t) is a positive operator, we obtain the dynamic model which

is used in the dynamic Kalman–Bucy ﬁltering (Albert, 1972).

Replacing the integration of the indirect and direct measurement by the operator

L, we write the system which determines the solution ϕ taking into account a priori

information (dynamic model):

Lϕ

+ ε = u,

+ η

= g.

The formal solution can be represented using the Fisher operator and writing the

Euler equation:

∗

−1

L + D

∗

−1

) ˜ϕ

= (L

∗

−1

u + D

∗

−1

(g + ϕ

: D

= 0, ϕ

t=0

= ϕ

Elements of applied functional analysis 285

The correlation operator of the solution errors is the inverse operator in relation

to the Fisher operator F = F

= (L

∗

−1

L + D

∗

−1

) > 0

due to the positiveness of the operator D

∗

−1

We analyze the solution, obtained by using the mathematical statistics princi-

ples.

Let the solution of the linear problem u = Lϕ + ε is obtained by the linear

procedure R:

ˆϕ = Ru = RLϕ + Rε.

Then the error of the obtained solution is described by the solution

δϕ

∆

= ˆϕ − ϕ = (RL − I)ϕ + Rε.

The error δϕ does not depend on the true value ϕ, if RL = I. We obtain this

operator R by using the method of least squares, if the inverse operator exists

∗

−1

. In fact:

R = (L

∗

−1

∗

−1

RL = (L

∗

−1

∗

−1

L) = I.

The quantity of the conditional mean value of the error δϕ is called bias (b):

∆

= E

δϕ = (RL −I)ϕ.

The characteristic of the solution accuracy is the error correlation:

δϕ

∆

= E[δϕδϕ

∗

The solution ˜ϕ such that

˜ϕ = arg inf spK

δϕ

is called an eﬀective solution.

First of all, we note that the solution of ill-posed problems by the method of

least squares (if operator (L

∗

−1

exists) is unbiased, but certainly ineﬀective:

mean error are inﬁnite. In fact,

E[δϕδϕ

∗

] = E[(Rε)(Rε)

∗

] = RE[εε

∗

= RK

∗

= (L

∗

−1

∗

−1

L(L

∗

−1

= (L

∗

−1

L), spK

δϕ

= ∞,

by virtue unboundedness of the Fisher inverse operator in ill-posed problems.

We construct an operator R, which gives an eﬀective solution. Such operator

has the minimum errors on the average by all possible realization of errors ε with

the correlator K

and with the functions with correlator K

. We write the error

correlator for an arbitrary “decision” operator R:

δϕ

= Ebb

∗

+ Enn

∗

286 STATISTICAL METHODS OF GEOPHYSICAL DATA PROCESSING

n = Rε,

δϕ

= (RL − I)K

(RL − I)

∗

+ RK

∗

The condition of the algorithm R optimum, which gives the solution with the

minimum error variance at any space point, will be following:

R = arg inf spK

δϕ

. (9.33)

We introduce the Hilbert space for operators with a bounded trace (the Hilbert–

Schmidt space of operators). In this case the scalar product is determined as

(A, B)

= (A, HB) = sp(A

∗

HB),

where H > 0. Then the problem (9.33) is reduced to the minimization of the

quadratic form

R = arg inf[||(RL − I)

∗

+ ||R

∗

]

= arg inf[(L

∗

− I, K

∗

− I)) + (R

∗

, K

∗

)],

therefore the Euler equation has the form

(LK

∗

+ K

∗

= LK

The optimal decision operator

R is described by the expression

R = K

∗

(LK

∗

+ K

)

−1

(9.34)

and gives the following correlator of the solution errors

δϕ

= K

− K

∗

(LK

∗

+ K

)

−1

In this case the error variance at the point x is minimal in comparison with the

solution errors variance, obtained with the help of the linear operator which is

distinct from

We note that the optimal decision operator for

R (see expression (9.34)) is

the operator of the linear regression (Albert, 1972), which is written in the form

R = K

ϕu

−1

. The accuracy of the ﬁeld ϕ recovery is determined by the correlator

δϕ

= K

− K

ϕu

−1

uϕ

The deviation of the decision rule (∆R) from the optimal one

R leads to increasing

of the correlator of the solution errors on the positive operator

∆K

δϕ

= ∆R(LK

∗

+ K

)∆R.

For the solution of the particular problems it is necessary to introduce the quan-

titative information measure, as far as only in this case it is possible to carry out the

quantitative comparison of the diﬀerent experiments, set up a mathematical prob-

lem for the design of experiment and to give the sense to the statement about the

suﬃciency or insuﬃciency of data for the solution of the inverse problem Lϕ+ε = u.

Elements of applied functional analysis 287

The Shannon functional (9.18), being the measure of the uncertainty, is ex-

pressed by means of the probability density p

= −(p(ϕ), ln p(ϕ)).

Starting from the representation of the joint probability density:

p(ϕ, u) = p(ϕ)p(u|ϕ) = p(u)p(ϕ|u),

it is possible to write the additive property by the following way:

uϕ

= H

+ H

u|ϕ

= H

+ H

ϕ|u

, (9.35)

where H

u|ϕ

and H

ϕ|u

are conditional entropies. As information measure about the

ﬁeld ϕ, contained in data u, is used the diﬀerence a priori (H

apr

) and a posteriori

apost

) entropies (analogously, ∆S is the diﬀerence of the Boltzmann’s entropies

by transfer of the system from one statement to other):

ϕu

∆

= H

apr

− H

apost

= H

− H

ϕ|u

or if it is used the additivity property (9.35),

ϕu

= H

+ H

− H

uϕ

= H

− H

u|ϕ

We show that the information I

ϕu

is always nonnegative, i.e. H

≥ H

ϕ|u

. We

prove an auxiliary inequality

−

p(ϕ) ln

p(ϕ)

q(ϕ)

dϕ ≤ 0, (9.36)

where q(ϕ) :

q(ϕ)dϕ = 1. Using Jensen’s (Rao, 1972) inequality Ef(ϕ) ≤ f (Eϕ),

which is valid for any convex functions f , it is possible to write:

−

p(ϕ) ln

p(ϕ)

q(ϕ)

dϕ = E ln

q(ϕ)

p(ϕ)

≤ ln E

q(ϕ)

p(ϕ)

= ln

p(ϕ)

q(ϕ)

p(ϕ)

dϕ = ln 1 = 0.

Replacing p(ϕ) on p(ϕ|u) and q(ϕ) on p(ϕ) in the inequality (9.36), we obtain

−

p(ϕ|u) ln

p(ϕ|u)

p(ϕ)

dϕ ≤ 0,

i. e.

−

p(ϕ|u) ln p(ϕ|u)dϕ ≤ −

p(ϕ|u) ln p(ϕ)dϕ.

Averaging the inequality with respect to u, we write:

−

Z Z

p(ϕ, u) ln p(ϕ|u)dϕdu ≤ −

p(ϕ) ln p(ϕ)dϕ,

that proves our statement: H

≥ H

ϕ|u

288 STATISTICAL METHODS OF GEOPHYSICAL DATA PROCESSING

We note, that the inequality (9.36) was a basis for the introduction of the Kull-

back discriminant measure (Kullback, 1978) in the space of distributions:

ρ(p

, p

) =

− p

) ln

dϕ =

dϕ −

dϕ ≥ 0,

where ρ(p

, p

) = 0 only by p

= p

We remind that a priori an a posteriori distribution densities do not connect

with the method of the inverse problem solution and in this sense the amount of

information is an objective measure of the decreasing of the system uncertainty as

a result of the experiment.

We shall ﬁnd an explicit form of the information about the ﬁeld ϕ, containing

in data u = Lϕ + ε for ϕ ∈ N (0, K

) and ε ∈ N(0, K

), using the relation (9.21)

(the Shannon’s entropy for normal distributed function):

ϕu

= H

apr

− H

apost

= H

− H

ϕ|u

= −(p

, ln p

) + [p

ϕ|u

, ln p(ϕ|u)] =

ln DetK

−

ln DetK

ϕ|u

where K

is a priori correlator;

ϕu

≡ K

δϕ

= K

− K

∗

(LK

∗

+ K

)

−1

is a posteriori correlator.

We note, the sense of the amount of information is determined by the diﬀerence

of the Hartley’s measures of information, if we interpret the determinant of the

correlator as a number of independent statements in the functional space, which is

proportional to the value of the volume V = [

]

1/2

. Using the equality

DetK

(DetK

δϕ

)

−1

= DetK

−1

δϕ

it is possible to represent the amount of information as

ϕu

ln Det(I + K

F ),

where F is the Fisher information operator. In the proper basis of the operator

F (Kozlov information operator (Ermakov and Zhigljavsky, 1987; Turchin et al.,

1971)) the amount of information can be presented by the following way

ϕu

(1 + λ

) =

sp ln (1 + K

F ).

We clarify the physical meaning of eigenvalues of the Kozlov information oper-

ator K

F ϕ

= λ

. In this connection we consider the Rayleigh’s functional

(u) =

(u, LK

∗

(u, u)

that has the sense of the variance of the useful signal s = Lϕ, where stationary

functional values J

(u), connected with the eigenfunctions of the correlator, LK

∗

coincide with the correlator eigenvalues:

(u) =

(u, K

(u, u)

Elements of applied functional analysis 289

which have the sense of the noise component n variance of the model u = s + n.

The ratio J = J

has the physical meaning of the signal/noise ratio:

J =

⇒

This clear physical meaning is observed in the case, if the stationary values of the

functional J are reached on the functions, which are simultaneously eigenfunctions

both for the operator LK

∗

and for K

(conforming basis).

The functional J can be written in the form

J =

(u, LK

∗

(u, K

or introducing the change of variables u = K

−1/2

˜u. The extremal properties of the

functional can be analyzed on the set of functions ˜u:

J =

(˜u, K

−1/2

∗

−1/2

˜u)

(˜u, ˜u)

The stationary points are determined by equations on eigenvalues:

−1/2

∗

−1/2

˜u

= λ

˜u

which are identical to the equation

F ϕ

= λ

where ϕ

and ˜u

are connected by the relation ˜u

= K

−1/2

Lϕ

Thus the eigenvalues of the operator K

F have the sense of the ratio signal/noise

for diﬀerent components of the solution in the canonical basis, and the Shannon’s

information to make it clear, what components of the solution ϕ =

(ϕ

, ϕ)

are supplied by information:

ϕu

ln (1 + λ

9.4 Elements of the Mathematical Design of Experiment

Introduced in Sec. 9.3 information measures are a natural basis for the problem

setting of the mathematical design of the experiment. The possibility of the math-

ematical design is connected with the choice of conditions (σ) from some set Σ. For

example, the choice of the measurement channel, choice of the time and frequency

intervals for measurements, the choice of spread of receivers in the remote sounding

problems. The general mathematical problem deﬁnition of the design of experiment

(searching of σ) is reduced to:

˜σ = arg inf

σ∈Σ

Φ(σ) = arg sup

σ∈Σ

−1

(σ).

It is advisable to choice the functional Φ as convex, where it should be bounded

below (above).

290 STATISTICAL METHODS OF GEOPHYSICAL DATA PROCESSING

The statement of the problem, connected with the solution of the inverse problem

Lϕ + ε = u, is called a regression model, if the function ϕ is represented a priori in

the ﬁnite-dimensional basis. In this case it is looked for the best unbiased estimate,

which, as shown above, can be found by the method of least squares. The standard

representation of the regression equation

) + ε

(by the lack of a priori information about ϕ) where

) is the regression

function. From the expression

L(x

)ϕ = L

(ψ

, ϕ) =

(ψ

, ϕ)Lψ

follows the correspondence ϕ

= (ψ

, ϕ) and f

) = L

)ψ

({ϕ

} are unknown

parameters; x

are controlled variables, for example, coordinates of the registration

i = 1 ÷ n, index j changes from 1 until r

= N ). The standardized design of

experiment σ(N ) is called a totality of values:

σ =



. . . x

. . . p



where

= 1, p

= r

/N . In this case as a functional argument is used the

Fisher’s information matrix:

F(σ) =

−1

)f(x

) = ||f

), f

) . . . f

) . . . ||,

) = L(x

)ψ

The continuous design of experiment is called the function p(x), such that

p(x) ≥ 0,

p(x)dx = 1.

As optimum criteria it are used the following (Penenko, 1981; Ermakov and Zhigl-

javsky, 1987; Fedorov, 1972):

(1) ˜σ = arg inf DetF

−1

(σ) = arg sup DetF (σ),

where F is the Fisher’s operator, depending on the design as on a parametric

function, and then the optimal design ˆσ is called D-optimal. At that, it is

minimized the volume of the concentration ellipsoid of the estimate obtained

by the least squares method. The problem can be empty even in the case of

the ﬁnite basis for the completely continuous L-operator.

Elements of applied functional analysis 291

(2) ˜σ = arg inf λ

max

(F (σ)) = arg sup λ

min

(F (σ))

(λ

min

, λ

max

are the minimum and maximum eigenvalues of the Fisher’s operator

F (σ)). This criterion leads to the E-optimal design of experiment, in this case

it is minimized the error of the least informative linear combination of the

parameters ϕ:

F ϕ = λ

min

ϕ.

(3) ˜σ = arg inf

σ∈Σ

sup

x∈X

(x), F

−1

(σ)f(x)).

This expression is a criterion, which minimizes the maximum value of the vari-

ance of the regression function estimate. The corresponding design is called

G-optimal.

(4) ˜σ = arg inf

(x)F

−1

(σ)f(x)dx.

This criterion minimizes the average on X of the variance value of the regression

function estimate.

(5) ˜σ = arg inf AF

−1

(σ).

This criterion minimizes the risk value which is given by the matrix A by the

generalized squared loss:

ϕ −

ϕ, A(

ϕ −

ϕ)).

This criterion is called L-optimal.

The design of experiment problems have the evident practical signiﬁcance. At

that it is involved to the optimum criteria: outlay, resources, qualitative parameters,

etc.

In each particular case, the problem of the criterion construction may be very

complicated. The averaging measures can possess an essential uncertainty. As a

rule, it is impossible to construct an unique criterion. The design of experiment

should satisfy to the totality of the criteria, i.e. the problem of the design of

experiment becomes multicriterion. By introducing a vector criterion {Φ

(σ)} the

plans can be partially ordered. One says that σ

dominates over σ

in respect with

the set {Φ

(σ)}, if {Φ

(σ

)} ≤ {Φ

(σ

)} for all values α, in this case at least for

one of all the strong inequality is fulﬁlled. Any plan is called Pareto optimal, if it

belongs to Pareto set (with criterion Φ

), and the elements of this set Σ

⊆ Σ do

not have the dominating conditions σ in the set Σ.

The speciﬁc character of the problem statement of the design of experiment,

which is connected with the inverse problems of the mathematical physics, is the

obligatory inclusion of a priori information, in particular, the use of conforming

basis for the solution regularization.

This page intentionally left blankThis page intentionally left blank