Troyan V., Kiselev Y. Statistical Methods of Geophysical Data Processing
Подождите немного. Документ загружается.
Methods for parameter estimation 173
where
ˆ
θ
T
is an estimate obtained by LSM without taking linear constraints into
account.
To find λ we multiply both sides of the equality (6.23) by A
T
from the right side
and take into account the condition (6.21). As the result we obtain
λ
T
= (
V
V
T
−
ˆ
θ
θ
T
A
T
)(A(ψ
T
ψ)
−1
A
T
)
−1
. (6.24)
Substituting the equality (6.24) into expression (6.23), we come to final estimate
for desired parameters
˜
θ
T
=
ˆ
θ
T
+ (
V
V
T
−
ˆ
θ
T
A
T
)(A(ψ
T
ψ)
−1
A
T
)
−1
A(ψ
T
ψ)
−1
. (6.25)
As it is possible to see from the formula (6.25), the estimate contains two addends:
the first one corresponds to an estimate obtained by the least squares method with-
out an account of the linear constraints, and second one characterizes the contri-
bution of the linear constraints to common estimate. The covariance matrix of the
estimates is expressed by the following formula:
R
˜
θ
= R
ˆ
θ
− σ
2
ε
(ψ
T
ψ)
−1
A
T
(A(ψ
T
ψ)
−1
A
T
)
−1
A(ψ
T
ψ)
−1
, (6.26)
R
ˆ
θ
= ˆσ
2
ε
(ψ
T
ψ)
−1
. (6.27)
On a main diagonal there are dispersions of the estimates of arguments, and the off-
diagonal elements characterize a cross correlation between estimates of arguments.
It is interesting to note, that the elements of the matrix consist in two terms: the
first term corresponds to elements of the covariance matrix without an account of
the linear constraints, and the second term characterizes influencing of the linear
constraints.
To find an estimate for the variance by the formula
ˆσ
2
ε
=
(
u
u − ψ
˜
θ)
T
(u − ψ
˜
θ)
n − S −K
.
Let us consider a case of a known inverse covariance matrix of the experimental
data W . In this case the estimates (6.14), (6.25) are rewritten as
˜
θ
θ
T
=
ˆ
θ
θ
T
+ (V
T
−
ˆ
θθ
T
A
T
)(A(ψ
T
W ψ)
−1
A
T
)
−1
A(ψ
T
W ψ)
−1
, (6.28)
ˆ
θθ = (ψ
T
W ψ)
−1
ψ
T
W u. (6.29)
The covariance matrix of the estimates
˜
θ,
ˆ
θ reads as
R
˜
θ
= R
ˆ
θ
− (ψ
T
W ψ)
−1
A
T
(A(ψ
T
W ψ)
−1
A
T
)
−1
A(ψ
T
W ψ)
−1
, (6.30)
R
ˆ
θ
= (ψ
T
W ψ)
−1
. (6.31)
It is possible to use the obtained estimates (6.28), (6.29) and covariance matrices
(6.30), (6.31) at presence of a priori information about the weight of some experi-
mental data. Such cases frequently meet, when the data under the approximation
are the result of the processing and represent estimates of the desired parameters.
Then the appropriate variances and covariance coefficients of the estimates of pa-
rameters can be used to create the weight matrix W .
174 STATISTICAL METHODS OF GEOPHYSICAL DATA PROCESSING
6.8 Linear Estimation — Case of Nonstationary Model
Let us consider the methods for an estimation of the linear parameters
u = ψθθ + ε
under a condition, that random component is nonstationary one. For the simplicity
we will assume that
σ
2
i
= Σ(f
i
,
ρ
ρ) = ρ
0
+ ρ
1
f
2
i
,
where f
i
= ψ
T
i
θ. In a special case ρ
1
= 0 we come to the stationary model. The
procedure of finding of the estimates of parameters θ and ρ is reduced to following:
(1) Computation of the zero approximation of the parameter vector
ˆ
θ
( 0)
by LSM
with the unit weight coefficients
ˆ
θ
θ
( 0)
= (ψ
T
ψ)
−1
ψ
T
u.
(2) Determination of the estimate of the model function
ˆ
f
i
= ψ
T
i
ˆ
θ
( 0)
.
(3) Computation of the squared difference
ˆε
2
i
= (u
i
−
ˆ
f
i
)
2
.
(4) Finding of estimate
ˆ
ρρ:
ˆρ
ρ = arg min (
ˆ
C
C −
ˆ
B
ρ
ρ)
T
(
ˆ
C
C −
ˆ
Bρ
ρ
ρ),
ˆρ
ρ = (
ˆ
B
T
ˆ
B)
−1
ˆ
B
T
ˆ
C
C
C,
ρ
0
≥ 0, ρ
1
≥ 0,
where
ˆ
B =
1
ˆ
f
2
1
1
ˆ
f
2
2
. . .
1
ˆ
f
2
n
,
ˆ
C
C =
ˆε
2
1
ˆε
2
2
. . .
ˆε
2
n
.
(5) Computation of the variance of the random component
ˆσ
2
i
= ˆρ
0
+ ˆρ
1
ˆ
f
2
i
.
(6) Determination (by weighted LSM) the estimate
˜
θ
θ
θ = (ψ
T
W ψ)
−1
ψ
T
W
u
u,
where W = diag (ˆσ
−2
1
, ˆσ
−2
2
, . . . , ˆσ
−2
n
).
(7) Computation of the deviation ˜ε
2
i
subject to
˜
θ
θ
˜ε
2
i
= (u
i
−
˜
f
i
)
2
,
where
˜
f
i
= ψψ
T
i
˜
θ.
Methods for parameter estimation 175
(8) Comparison of ˜ε
2
i
with the the threshold value β
0
. If ˜ε
2
i
≤ β
0
, then the iterative
procedure is finished. If ˜ε
2
i
> β
0
, then
ˆ
θ
θ
( 0)
=
˜
θ
θ, ˆc
i
= ˜ε
2
i
,
ˆ
f
i
=
˜
f
i
and go to the
point 4.
At the enough universal propositions about the regularity of the function Σ,
which one are usually fulfilled in practice, the parametric models σ
2
i
lead to the es-
timates having the asymptotically normal distribution, which one at a great enough
sample size coincides with the distribution of estimates θθ, obtained by the weighted
LSM provided that the weight coefficients are known exactly.
6.9 Bayes’ Criterion and Method of Statistical Regularization
At the solution of practical problems of processing of the geophysical data the great
importance has an account of the available a priori information about parameters of
the explored object. The Bayes’ criterion opens the ample opportunities of the use
of a priori information in a statistical shape. Let’s assume, that in the model (6.15)
the vector θ is random. The joint density function of the vector of experimental
data u and the parameter vector
θ
θ can be written as
p(
u
u,θθ) = p(
θ
θ)p(u/θ) = p(u)p(
θ
θ/
u
u), (6.32)
where p(θθ) is a priori density function of the vector of desired parameters, p(uu/θ)
is a conditional density function of the vector of experimental data u at the fixed
parameter vector θ, p(u) is a priori density function of the vector of experimental
data, p(θθ/uu) is a posteriori density function for the desired parameter vector θθ
at fixed realization of the experimental data
u
u. Using the expression (6.32) it is
possible to find a posteriori density function of the desired parameter vector
p(θ/u) =
p(
θ
θ)p(u
u
u/θ
θ
θ)
p(uu)
. (6.33)
In accordance with the Bayes’ criterion, as the estimate of the desired parame-
ter vector, the mathematical expectation of the desired parameter vector under a
posteriori distribution is accepted
ˆ
θ
B
=
Z
θ p(
θ
θ/
u
u)dθ. (6.34)
From an analysis of the formula (6.33) we conclude, that a posteriori density
function is a product of a priori density p(θ
θ
θ) and conditional density p(u
u
u/
θ
θ) (the
denominator does not depend on the desired parameter vector and it is introduced
for normalization). Hence, a priori information about desired parameter vector con-
sists of a priori information, which is obtained before experiment, and information
extracted from the experimental data. If one of the multipliers prevails over other,
then it determines a posteriori information in general. For example, if conditional
density has an abrupt maximum in the parametric space ω
0
and it fast tends to
176 STATISTICAL METHODS OF GEOPHYSICAL DATA PROCESSING
zero in other space ω
1
, and if a priori density p(θ
θ
θ) is slow varying function in ω
0
,
then a posteriori density practically does not depend on a shape of p(θ
θ
θ), which may
put a constant in all space ω
0
.
6.10 Method of Maximum a Posteriori Probability
Alongside with an estimate (6.34) which is the conditional mathematical expecta-
tion, wide practical applying has the method of a maximum of a posteriori prob-
ability. In this method as an estimate
ˆ
θ
θ, a position of a maximum of a density
function p(θ
θ
θ/u
u
u) in a parametric space is accepted. As the logarithm is a monotone
function, the location of a maximum can be determined by the function ln p(
θ
θ/u
u
u),
that appears by more convenient at the solution of a wide variety of the problems. If
the maximum is placed inside the accessible region of variation of θ
θ
θ and the density
function has the first continuous derivative, then the necessary condition consist in
the equality to zero of ln p(θ/u)
∂ ln p(θ/u)
∂θ
s
= 0, s = 1, 2, . . . , S. (6.35)
The equation (6.35) is called the equation of the maximum a posteriori proba-
bility. Replacing p(θθ/uu) by its representation (6.33) and taking the logarithm we
obtain
ln p(θ/u) = ln p(
θ
θ) + ln p(
u
u/θ) − ln p(u). (6.36)
For finding an estimate using the maximum a posteriori probability method
(MAP), let’s substitute the expression (6.36) into the system of equations (6.35)
and to obtain the equation of maximum a posteriori probability in a form
∂ ln p(θ)
∂θ
s
+
∂ ln p(u/θθ)
∂θ
s
= 0, s = 1, 2, . . . , S. (6.37)
The first addend from the left hand side of (6.37) characterizes a priori data and
the second one is connected with an experimental data.
By analyzing of (6.37) we can conclude, that in the case of the function ln p(θ)
has a weak variation in the area of allowed values of
θ
θ, then the first term in (6.37)
can be neglected, and MAP transforms to LSM. It means that LSM is a special case
of MAP under the condition of an absence of a priori information, that is equivalent
to a hypothesis p(θ) = const in the area of allowed values of the desired parameter
vector.
At the nontrivial assignment of a priori density p(θ) 6= const MAP improves
LSM estimate, and makes the solution stable. The engaging of a priori information
underlies the statistical regularization method.
Let us consider in more detail a procedure of finding the estimate
ˆ
θ for a special
case of the normal distributed random component ε ∈ N (0, R
ε
) and normal a priori
Methods for parameter estimation 177
distribution
θ
θ ∈ N(h
θ
θi, R
θ
) for the linear model (6.15). The logarithm of a posteriori
density function looks as
ln p(θ/u) = −
1
2
(uu − ψ
θ
θ)
T
R
−1
ε
(u
u
u − ψθ
θ
θ) −
1
2
(θθ − hθθi)
T
R
−1
θ
×(θθ − hθθi) −
n + S
2
ln(2π) −
1
2
(ln |R
θ
| + ln |R
ε
|) − ln p(u).
the necessary condition of the maximum of a posteriori probability leads to the
system of equations
(ψ
T
R
−1
ε
ψ + R
−1
θ
)θ = ψ
T
R
−1
ε
u + R
−1
θ
hθi,
and these equations give the estimate
˜
θθ:
˜
θ
θ = (ψ
T
R
−1
ε
ψ + R
−1
θ
)
−1
(ψ
T
R
−1
ε
u + R
−1
θ
hθi). (6.38)
Using the equality
(ψ
T
R
−1
ε
ψ + R
−1
θ
)
−1
= R
θ
− R
θ
ψ
T
(ψR
θ
ψ
T
+ R
ε
)
−1
,
ψR
θ
, (ψ
T
R
−1
ε
ψ + R
−1
θ
)
−1
ψ
T
R
−1
ε
= R
θ
ψ
T
(ψR
θ
ψ
T
+ R
ε
)
−1
, (6.39)
to represent the estimate (6.38) in a form:
˜
θ = hθi + R
θ
ψ
T
(ψR
θ
ψ
T
+ R
ε
)
−1
(u − ψh
θ
θi). (6.40)
The formula (6.40) enables a simple interpretation: the MAP estimate
ˆ
θ is repre-
sented as the sum of a priori vector h
θ
θi and the correction, which one is the weighted
difference between the experimental data and a priori model.
In the special case of an uncorrelated random component R
ε
= σ
2
ε
I
n
and uncor-
related components of the vector θθ, a priori density function contains: R
θ
= σ
2
θ
I
n
( I
n
and I
S
unitary matrices with dimensions [n × n] and [S × S] respectively,
σ
2
ε
is a variance of the random component, σ
2
θ
is a priori variance of the desired
parameters). The formulas (6.38), (6.40) we rewrite in the next form
˜
θθ = (ψ
T
ψ + αI
S
)
−1
(ψ
T
u + αhθi), (6.41)
˜
θ = hθi + ψ
T
(ψψ
T
+ αI
n
)
−1
(u − ψhθθi), (6.42)
where α = σ
2
ε
/σ
2
θ
is a fixed regularization parameter of zero order. We should note,
that regularization parameter under the statistical interpretation has a clear sense:
it is the energy ratio, noise/signal.
If α tends to zero, the MAP estimate
˜
θ (6.41) transforms to LSM estimate
ˆ
θθ
(6.11).
The covariance matrices of the estimates of the parameter vector, given by the
formulas (6.38), (6.40), (6.41), (6.42), can be written as
R
˜
θ
= (ψ
T
R
−1
ε
ψ + R
−1
θ
)
−1
, (6.43)
R
˜
θ
= R
θ
− R
θ
ψ
T
(ψR
θ
ψ
T
+ R
ε
)
−1
ψR
θ
, (6.44)
178 STATISTICAL METHODS OF GEOPHYSICAL DATA PROCESSING
R
˜
θ
= σ
2
ε
(ψ
T
ψ + αI
S
)
−1
, (6.45)
R
˜
θ
= σ
2
θ
(I
S
− ψ
T
(ψψ
T
+ αI
n
)
−1
ψ). (6.46)
The interpretation quality should estimate on a basis of the analysis of the ma-
trix R
˜
θ
. The main diagonal of such matrix contains the variances of the estimates
of parameters and far-diagonal elements which describe correlations between the
estimates. The matrices (6.43)–(6.46) can be used for the design of the experiment.
We should note, a choice of formulas for the estimation
˜
θ ((6.38) or (6.40)) mainly
depends on the computational capability. If the computation of the inverse covari-
ance matrix of the high dimension R
˜
θ
and R
ε
is difficult, then the form of (6.40) is
preferable.
6.11 The Recursion Algorithm of MAP
At the solution the problem of an approximation of a great volume of the exper-
imental data, the boundedness of a computer memory results in the necessity of
constructing of the interpretative algorithms for the finding of the estimate of the
parameter vector by a method of the maximum of a posterior probability. Let’s
consider a stable iterative algorithm, which is based on the representation of the
estimate in the form (6.40).
As an initial value for the parameter vector estimate we choose a priori vector
θθ
(0)
= h
θ
θi, and an initial covariance matrix is equal to a priori matrix R
(0)
θ
= R
θ
.
The first approximation is computed by the formula
θθ
(1)
= θ
(0)
+ γ
(1)
∆θθ
(1)
,
where ∆θθ
(1)
= R
(0)
θ
ψ
ψ
(1)
, ψ(1) is a column vector, with the components coinciding
with the components of the first row of the matrix ψ:
γ
(1)
= N
1
/δ
2
1
, N
1
= u
1
−
ψ
ψ
(1)T
θθ
(0)
,
δ
2
1
= σ
2
ε1
+ ψ
(1)T
∆
θ
θ
(1)
.
The covariance matrix of the estimate θ
(1)
looks like
R
(1)
θ
= R
(0)
θ
− ∆θ
(1)
∆θ
(1) T
/δ
2
1
.
Let’s we find (i −1)-th approximation
θ
θ
(i−1)
and R
(i−1)
θ
. Then, i-th approximation
we construct as the next:
θ
θ
(i)
=
θ
θ
(i−1)
+ γ
(i)
∆θ
θ
θ
(i)
,
where
γ
(i)
= N
i
/δ
2
i
, N
i
= u
i
−
ψ
ψ
(i)T
θθ
(i−1)
,
δ
2
i
= σ
2
εi
+ ψ
(i)T
∆θ
(i)
,
∆
θ
θ
(i)
= R
(i−1)
θ
ψ
ψ
(i)T
,
R
(i)
θ
= R
(i−1)
θ
− ∆θ
(i)
∆
θ
θ
(i) T
/δ
2
i
.
Methods for parameter estimation 179
6.12 Singular Analysis and Least Squares Method
By the practical application of the least squares method (LSM), using the model
(6.15), we can not obtain satisfactory result in the case of a bad condition of the
matrix ψ
T
ψ, which is subjected to inversion. As a measure of the condition, it is
usually used a condition number
β =
G
g
,
where
G = max
θ
kψ
T
ψ
θ
θk
1
kθk
1
, g = min
θθ
kψ
T
ψθ
θ
θk
1
kθk
1
, kθk
1
=
S
X
s=1
|θ
s
|. (6.47)
The maximum and minimum under conditions (6.47) are determined on all non-zero
parameter vectors. If the matrix ψ
T
ψ is singular, then g = 0.
The condition number β is a measure of the proximity to singularity. The
condition number can be considered as an inverse value of the relative distance
from a given matrix to a set of singular matrices. Let us consider the properties of
the condition number β which follow from its definition.
(1) The condition number β is always greater or equal to 1 (β ≥ 1), because
G ≥ g.
(2) If the matrix ψ
T
ψ is a diagonal one, as in the case of orthogonal polynomials,
then
β =
max |(ψ
T
ψ)
ii
|
min |(ψ
T
ψ)
ii
|
.
(3) If an inverse matrix exists, then
β = kψ
T
ψkk(ψ
T
ψ)
−1
k.
(4) If the matrix ψ
T
ψ is a singular one, then β = ∞.
At the near singularity of the matrix ψ
T
ψ the condition number β has a great value
and independently of the method of solution of the system of normal equations, the
errors in an input information and round-off errors lead to a strong “oscillations” of
the desired parameter vector. As an extreme case, when the columns of the matrix
ψ
T
ψ are linearly dependent, the matrix ψ
T
ψ will be singular one with the condition
number β = ∞. Hence, the high values of the condition number testify to presence
of near linear the dependence of the rows of the matrix.
More stable method of computing of the parameter vector for the linear model in
the case of a bad condition of the matrix is based on the singular value decomposition
(SVD) of the structural matrix ψ. This approach is more affective to account of the
input information, the round-off errors and the linear dependence, than classical
LSM
180 STATISTICAL METHODS OF GEOPHYSICAL DATA PROCESSING
Let us consider the singular value decomposition on an example of the linear
model (6.15).
It is possible to show, that for an arbitrary real matrix ψ is valid the next
representation
ψ = QΣP
T
, (6.48)
where Q
n×n
and P
S×S
are the orthogonal matrices, i.e. QQ
T
= I
n×n
and P P
T
=
I
S×S
respectively, and Σ
n×S
is a diagonal matrix with the elements
Σ
ij
=
σ
i
, i = j, i = 1, . . . , n,
0, i 6= j, j = 1, . . . , S,
, Σ =
σ
1
. . . 0
. . . . . . . . .
0 . . . σ
S
0 . . . 0
. . . . . . . . .
0 . . . 0
. (6.49)
The matrix Q consists of n orthonormal eigenvectors of the matrix ψψ
T
, and
the matrix P consists of orthonormal eigenvectors of the matrix ψ
T
ψ. The diagonal
elements σ
i
of the matrix Σ are non-negative quantities of square root of the eigen-
values (singular numbers) of the matrix ψ
T
ψ. Assume that σ
1
≥ σ
2
≥ ··· ≥ σ
S
≥ 0.
If a matrix rank of the matrix ψ is equal to r, then σ
r+1
= σ
r+2
= ··· = σ
S
= 0.
For a simplicity of the analysis we suppose that the random component of the
model ε is non-correlated one. Then, taking into account (6.48), the LSM criterion
can be written as
ˆ
θ
θ = arg min λ(θ
θ
θ),
where
λ(
θ
θ) = (u
u
u − ψ
θ
θ)
T
(
u
u − ψ
θ
θ) = (u
u
u − QΣP
T
θ
θ)
T
QQ
T
× (
u
u − QΣP
T
θ
θ)(y − Σbb)
2
.
Using the property QQ
T
= I, we obtain the next representation for the quadratic
form λ:
λ(bb) = (y − Σbb)
2
, (6.50)
where
y
y = Q
T
u,
b
b = P
T
θ. (6.51)
Minimizing (6.50) on
b
b, we obtain the estimate
ˆ
bb
ˆ
b = Σ
+
yy (6.52)
or, come back to initial variables, we obtain
ˆ
θ
θ = P Σ
+
Q
T
u
u. (6.53)
The matrix
Σ
+
=
σ
−1
1
. . . 0
. . . . . . . . .
0 . . . σ
−1
S
0 . . . 0
. . . . . . . . .
0 . . . 0
is called pseudoinverse matrix with respect to the matrix Σ, the pseudoinverse
matrix has dimension [S × n] and satisfies the next properties
Methods for parameter estimation 181
(1) ΣΣ
+
Σ = Σ;
(2) Σ
+
ΣΣ
+
= Σ
+
;
(3) ΣΣ
+
is a symmetric matrix;
(4) Σ
+
Σ is a symmetric matrix.
It is possible to demonstrate a unique existence of a such matrix. If the diagonal
elements σ
ii
of the matrix Σ do not equal to zero, then estimate b
i
(i = 1, 2, . . . , S)
will be written as
ˆ
b
i
= y
i
/σ
ii
, i = 1, 2, . . . , S.
However if some σ
ii
are small, such procedure is unwanted. In this case it is neces-
sary to analyze the values σ
ii
before finding of an estimate
ˆ
b. It is possible to show,
that the singular numbers will be distinct from zero if and only if the columns of
the matrix ψ are linearly independent. The analysis of the singular numbers should
be implemented on a basis of the suspected accuracy of the input data and the
calculation accuracy of the computer. On a basis of a priori accuracy to choose the
threshold value α, to which one compare everyone σ
ii
. If σ
ii
≥ α, then
ˆ
b
i
= y
i
/σ
ii
for i = 1, 2, . . . , i
0
. If σ
ii
< α, then
ˆ
b
i
= 0 for i = i
0
+ 1, . . . , S.
As a matter of fact, at σ
ii
< α the component of the vector b
i
can be equal
to an arbitrary value. This arbitrary rule is caused by the non-uniqueness of the
LSM solution. To exclude an influence of unstable components of the vector
ˆ
b
i
at
σ
ii
< α, in the singular analysis their values are putted to zero. Let us consider an
interpretation of the threshold value α by the example of a model with the random
parameter vector θθ. Let’s the mathematical expectation of the parameter vector
is equal to zero hθi = 0 and a priori covariance matrix hθ
θ
θ
T
i = R
θ
θ
is given. For
the simplicity we assume that the components of the parameter vector θθ and the
random component u of the model have a lack of correlation i.e.
R
θ
θ
= σ
2
θ
I, R
ε
= σ
2
ε
I.
Moreover we assume that the random vectors θ
θ
θ ε
ε
ε are uncorrelated among them-
selves.
Using the singular decomposition (6.48) of the structural matrix ψ, and nota-
tions (6.51), to rewrite the model (6.10) in a form
y
y =
X
b +
e
e,
where
e
e = Q
T
εε. We find the solution as a linear form
ˆ
bb = Γ
y
y, (6.54)
where
Γ =
γ
1
0 . . . 0
0 γ
2
. . . 0
. . . . . . . . . . . .
0 0 . . . γ
S
0 . . . 0
0 . . . 0
. . . . . . . . .
0 . . . 0
182 STATISTICAL METHODS OF GEOPHYSICAL DATA PROCESSING
is a diagonal rectangular matrix with the dimension [S × n]. To find an estimate
of the parameter vector
b
b to determine the elements of the matrix Γ is enough.
Let’s to use the mean-square criterion. In this case the error of estimation and the
corresponding covariance matrix have a form
ξ
ξ =
ˆ
b
b
b −
b
b = (ΓΣ − I)
b
b + Γe
e
e,
hξξ
ξ
ξ
T
i = (ΓΣ − I)hbbb
T
i(ΓΣ − I)
T
+ Γh
e
ee
T
iΓ
T
= (ΓΣ − I)R
b
(ΓΣ − I)
T
+ ΓR
e
Γ
T
.
The standard deviation of the desired error is equal to
σ
2
ξ
= Sp(hξξξξ
T
i) =
S
X
j=1
[(γ
j
σ
j
− 1)
2
σ
2
b
j
+ γ
j
σ
2
e
j
],
where Sp(A) is the spur of the matrix A.
In the special case of R
θ
θ
= σ
2
θ
I R
ε
= σ
2
ε
I, we obtain σ
2
b
= σ
2
θ
, σ
2
e
j
= σ
2
ε
, and
introducing the notation λ(γ
j
) = Sp(hξ
ξ
ξξ
ξ
ξ
T
i), we can write
λ(γ
j
) =
S
X
j=1
[(γ
j
σ
j
− 1)
2
σ
2
θ
+ γ
2
j
σ
2
ε
].
We determine the elements γ
j
using the minimum condition of the mean square
error, i.e. by the solution of the system of equations
∂λ(γ
j
)
∂γ
j
= 0, j = 1, . . . , S.
The desired estimate looks like
ˆγ
j
=
σ
j
σ
2
j
+ σ
2
ε
/σ
2
θ
.
If for the certain random parameters σ
2
θ
→ ∞ is valid, then ˆγ
j
= 1/σ
j
and
Γ = Σ
+
, i.e. we obtain the LSM estimate (6.52) with a singular decomposition of
the structural matrix.
In the case of the singular analysis the mean-square value of the desired error
will be written as
λ(i
0
) = hkξ
ξ
ξk
2
i = σ
2
θ
(S − i
0
) + σ
2
ε
i
0
X
j=1
1/σ
2
j
.
The minimization of λ(i
0
) on i
0
leads to a choice of the number i
0
, when an appro-
priate singular number satisfies the relations
σ
i
0
≥
σ
ε
σ
θ
and σ
i
0
+1
<
σ
ε
σ
θ
.
Hence, to find the boundary value i
0
it is necessary to know a priori value of
the noise variance σ
2
ε
and a priori variance of the desired parameters σ
2
θ
.