Klir G.J. Uncertainity and Information. Foundations of Generalized Information Theory
Подождите немного. Документ загружается.
The identification nonspecificity is of course uniquely determined by the given
set P of projections of R. Its maximum is obtained when P =∆,which expresses
our total ignorance about R. Clearly,
The amount of information, I(P), about R contained in the given set P of pro-
jections of R is then calculated by the formula
The dependence of H(R
P
) and I(P) on P is examined in the next example.
EXAMPLE 2.4. Consider a system with three variables, x
1
, x
2
, x
3
. The vari-
ables, whose values are in the set {0, 1}, are constrained by a ternary relation
R Õ {0, 1}
3
that is not known. Labels of all potential states of the system, that
is, elements of {0, 1}
3
, are introduced in Table 2.1a.Assume that the three binary
relations specified Table 2.1b are projections of R. Assume further that we
know either all of these projections or only two of them (see Table 2.1c, the
first column). Our aim is to identify the unknown relation R from information
in each of these four sets of projections, P
1
, P
2
, P
3
, P
4
. For each P
i
(i Œ⺞
4
), we
determine first the cylindric closure and all its complete subsets, , in the
same way as in Example 2.3 (the second column in the Table 2.1c). Then, we
can calculate for each P
i
the identification nonspecificity, H( ), and the infor-
mation content, I( ), of the projections in P
i
(columns 3 and 4 in the Table
2.1c). This example is quite illustrative. It shows that the choice of projections
is important. When we know all the projections (P
1
), the identification is fully
specific and the information content is 8 bits (maximum identification non-
specificity is log
2
2
8
= 8 and the information contained in the three projections
reduces it to 0). When we know only projections R
13
and R
23
(P
4
), the identi-
fication is still fully specific. Therefore I(P
1
) = I(P
4
), which means that adding
projection R
12
to P
4
does not increase the information content. Each of the
remaining pairs of projections, P
2
and P
3
, identifies seven possible overall rela-
tions, so their identification nonspecificity is log
2
7 = 2.81 and their information
content is 8 - 2.81 = 5.19.
EXAMPLE 2.5. The purpose of this example is to illustrate how the various
properties of the Hartley measure, expressed by Eqs. (2.16)–(2.35), can be uti-
lized for analyzing n-dimensional relations (n ≥ 2). A simple system with four
variables, x
1
, x
2
, x
3
, x
4
, is employed here as an example.The variables take their
values in sets X
1
, X
2
, X
3
, X
4
, respectively, where X
1
= X
2
= {0, 1} and X
3
= X
4
=
{0, 1, 2}. All possible overall states of the system are listed in Table 2.2a. This
R
P
i
R
P
i
R
P
i
IP H H
P
()
=
()
-
()
=-=
RR
Ø
16 3 13.
H RP
Ø
()
=
{}
()
==
log ,
log .
2
4
2
16
01
216
42 2. CLASSICAL POSSIBILITY-BASED UNCERTAINTY THEORY
2.2. HARTLEY MEASURE OF UNCERTAINTY FOR FINITE SETS 43
States x
1
x
2
x
3
s
0
00 0
s
1
00 1
s
2
01 0
s
3
01 1
s
4
10 0
s
5
10 1
s
6
11 0
s
7
11 1
(a)
Sets of Known Cylindric Closure (CC) and Its H(R
Pi
) I(P
i
)
Projections: P
i
Complete Subsets: R
Pi
P
1
= {R
12
, R
13
, R
23
}{s
0
, s
1
, s
3
, s
4
} (CC) 0 8
P
2
= {R
12
, R
13
}{s
0
, s
1
, s
2
, s
3
, s
4
} (CC) 2.81 5.19
{s
0
, s
1
, s
2
, s
4
}
{s
0
, s
1
, s
3
, s
4
}
{s
0
, s
2
, s
3
, s
4
}
{s
1
, s
2
, s
3
, s
4
}
{s
0
, s
3
, s
4
}
{s
1
, s
2
, s
4
}
P
3
= {R
12
, R
23
}{s
0
, s
1
, s
3
, s
4
, s
5
} (CC) 2.81 5.19
{s
0
, s
1
, s
3
, s
4
}
{s
0
, s
1
, s
3
, s
5
}
{s
0
, s
3
, s
4
, s
5
}
{s
1
, s
3
, s
4
, s
5
}
{s
0
, s
3
, s
5
}
{s
1
, s
3
, s
5
}
P
4
= {R
13
, R
23
}{s
0
, s
1
, s
3
, s
4
,} (CC) 0 8
(c)
R
12
: x
1
x
2
00
01
10
R
23
: x
2
x
3
00
01
11
R
13
: x
1
x
3
00
01
10
(b)
Table 2.1. System Identification (Example 2.4)
set of overall states is a 4-dimensional relation R on the Cartesian product {0,
1}
2
¥ {0, 1, 2}
2
. An important way of analyzing such a relation is to search for
strong dependencies between various subsets of variables. The capability of
measuring conditional uncertainty and information transmission for any two
disjoint subsets of variables is essential for conducting such a search in a mean-
ingful way.
Table 2.2. Information Analysis of a 4-Dimensional Relation (Example 2.5)
Relation R Conditional uncertainties and information transmissions Normalized counterparts
x
1
x
2
x
3
x
4
00 1 0 H(X
1
|X
2
¥ X
3
¥ X
4
) = 4.39 - 3.91 = 0.48 0.48/log
2
2 = 0.48
00 1 1 H(X
2
¥ X
3
¥ X
4
|X
1
) = 4.39 - 1 = 3.39 3.39/log
2
18 = 0.81
01 0 2 T
H
(X
1
, X
2
¥ X
3
¥ X
4
) = 1 + 3.91 - 4.39 = 0.52 0.52/log
2
2 = 0.52
01 1 1 H(X
2
|X
1
¥ X
3
¥ X
4
) = 4.39 - 3.91 = 0.48 0.48/log
2
2 = 0.48
00 2 0 H(X
1
¥ X
3
¥ X
4
|X
2
) = 4.39 - 1 = 3.39 3.39/log
2
18 = 0.81
00 2 1 T
H
(X
2
, X
1
¥ X
3
¥ X
4
) = 1 + 3.91 - 4.39 = 0.52 0.52/log
2
2 = 0.52
01 2 2 H(X
3
|X
1
¥ X
2
¥ X
4
) = 4.39 - 3.32 = 1.07 1.07/log
2
3 = 0.68
10 0 0 H(X
1
¥ X
2
¥ X
4
|X
3
) = 4.39 - 1.58 = 2.81 2.81/log
2
12 = 0.78
10 0 1 T
H
(X
3
, X
1
¥ X
2
¥ X
4
) = 1.58 + 3.32 - 4.39 = 0.51 0.51/log
2
3 = 0.32
11 0 1 H(X
4
|X
1
¥ X
2
¥ X
3
) = 4.39 - 3.46 = 0.93 0.93/log
2
3 = 0.59
10 1 2 H(X
1
¥ X
2
¥ X
3
|X
4
) = 4.39 - 1.58 = 2.81 2.81/log
2
12 = 0.78
11 1 2 T
H
(X
4
, X
1
¥ X
2
¥ X
3
) = 1.58 + 3.46 - 4.39 = 0.65 0.65/log
2
3 = 0.41
10 0 2 H(X
1
¥ X
2
|X
3
¥ X
4
) = 4.39 - 3.17 = 1.22 1.22/log
2
4 = 0.61
10 1 0 H(X
3
¥ X
4
|X
1
¥ X
2
) = 4.39 - 2 = 2.39 2.39/log
2
9 = 0.75
10 1 1 T
H
(X
1
¥ X
2
, X
3
¥ X
4
) = 3.17 + 2 - 4.39 = 0.78 0.78/log
2
6 = 0.39
10 0 2 H(X
1
¥ X
3
|X
2
¥ X
4
) = 4.39 - 2.85 = 1.54 1.54/log
2
6 = 0.54
10 1 0 H(X
2
¥ X
4
|X
1
¥ X
3
) = 4.39 - 2.85 = 1.54 1.54/log
2
6 = 0.54
10 2 1 T
H
(X
1
¥ X
3
, X
2
¥ X
4
) = 2.85 + 2.85 - 4.39 = 1.31 1.31/log
2
6 = 0.51
11 2 0 H(X
1
¥ X
4
|X
2
¥ X
3
) = 4.39 - 2.85 = 1.54 1.54/log
2
6 = 0.54
11 2 1 H(X
2
¥ X
3
|X
1
¥ X
4
) = 4.39 - 2.85 = 1.54 1.54/log
2
6 = 0.54
11 2 2 T
H
(X
1
¥ X
4
, X
2
¥ X
3
) = 2.85 + 2.85 - 4.39 = 1.31 1.31/log
2
6 = 0.51
(a)(b)(c)
44
2.3. HARTLEY-LIKE MEASURE OF UNCERTAINTY FOR INFINITE SETS 45
Suppose we want to calculate conditional uncertainties and information
transmissions for all partitions of {x
1
, x
2
, x
3
, x
4
} with two blocks, as listed in
Table 2.2b. Due to Eqs. (2.23), (2.24), and (2.31), all these calculations are
based on the following values of the Hartley measure, which are obtained
directly from the given relation R:
These values are shown in the calculations in Table 2.2b. Also shown in Table
2.2c are calculations of their normalized counterparts.
The capability of calculating conditional uncertainties and information
transmissions between groups of variables, illustrated here by a simple
example, is a particularly important tool for analyzing high-dimensional rela-
tions. Normally, we need to do these calculations only for some groups of vari-
ables that are of interest in each individual application.
2.3. HARTLEY-LIKE MEASURE OF UNCERTAINTY FOR INFINITE SETS
The Hartley measure is applicable only to finite sets. Its counterpart for
bounded and convex subsets of the n-dimensional Euclidean space ⺢
n
(n ≥ 1,
finite), which is called a Hartley-like measure, emerged only in the mid-1990s
(see Note 2.3).
2.3.1. Definition
Let X denote a universal set of concern that is assumed to be a bounded and
convex subset of ⺢
n
for some n ≥ 1, and let HL denote the Hartley-like
measure defined on convex subsets of X. Then, HL is a functional of the form
HL C:,Æ
+
⺢
HX HX
HX HX
HX X
HX X
HX X HX X HX X HX X
HX
122
342
12 2
34 2
13 14 23 24
2
21
3158
42
9317
6285
()
=
()
==
()
=
()
==
¥
()
==
¥
()
==
¥
()
=¥
()
=¥
()
=¥
()
==
log ,
log . ,
log ,
log . ,
log . ,
1123 2
124 2
134 234 2
1234 2
11 3 46
10 3 32
15 3 91
21 4 39
¥¥
()
==
¥¥
()
==
¥¥
()
=¥¥
()
==
¥¥¥
()
==
XX
HX X X
HXXX HXXX
HX X X X
log . ,
log . ,
log . ,
log . .
where C denotes the family of all convex subsets of X. Functionals in the class
defined for all convex subsets A of X by the formula
(2.37)
were found to satisfy all axiomatic requirements, stated in Section 2.3.2, that
are essential for measuring uncertainty in this case. Symbols in Eq. (2.37) have
the following meaning:
•
m denotes the Lebesgue measure;
•
T denotes the set of all isometric transformations from one orthogonal
coordinate system to another;
•
denotes the ith projection of A in coordinate system t;
•
b and c denote positive constants (b π 1), whose choice defines a mea-
surement unit.
Equation (2.37) allows us to define a measurement unit for the Hartley-like
measure by choosing any positive values b and c except b = 1. Let the
measurement unit be defined by the requirement that HL(A) = 1 when A
is a closed interval of real numbers of length 1 in some assumed unit of
length, which must be specified in each particular application. That is, we
require that
for the specified unit of length. It is convenient to choose b = 2 and c = 1 to
satisfy this equation. Then,
(2.38)
The chosen measurement unit is intuitively appealing, since HL(A) = 1 when
A is a unit interval, HL(A) = 2 when A is a unit square, HL(A) = 3 when A is
a unit cube, and so on.
2.3.2. Required Properties
Let
X = X
i
denote a universal set of concern that is assumed to be a bounded
and convex subset of ⺢
n
for some finite n ≥ 1. Then the Hartley-like measure
defined on the family of all convex subsets of X, C, by Eq. (2.38) is expected
to satisfy the following axiomatic requirements:
n
i=1
HL A A A A
tT
ii
i
n
i
n
tt
()
=+
()
[]
+
()
-
()
È
Î
Í
˘
˚
˙
Ï
Ì
Ó
¸
˝
˛
Œ
==
’’
min log .
2
11
1 mm m
c
b
log 2 1=
A
i
t
HL A c A A A
tT
bi i
i
n
i
n
tt
()
=+
()
[]
+
()
-
()
È
Î
Í
˘
˚
˙
Ï
Ì
Ó
¸
˝
˛
Œ
==
’’
min log ,1
11
mm m
46 2. CLASSICAL POSSIBILITY-BASED UNCERTAINTY THEORY
Axiom (HL1) Range. For each A Œ C, HL(A) Œ [0, •), where HL(A) = 0 if
and only if A = {x} for some x ŒX.
Axiom (HL2) Monotonicity. For all A, B Œ C, if A Õ B, then HL(A) £ HL(B).
Axiom (HL3) Subadditivity. For each A ΠC, , where A
i
denotes the 1-dimensional projection of A to dimension i in some coordinate
system.
Axiom (HL4) Additivity. For all A ΠC, such that
A = A
i
where A
i
has the
same meaning as in Axiom (HL3),
Axiom (HL5) Coordinate Invariance. Functional HL does not change under
isometric transformations of the coordinate system.
Axiom (HL6) Continuity. HL is a continuous functional.
It is evident that HL defined by Eq. (2.38) is continuous, invariant with
respect to isometric transformations of the coordinate system, and that it sat-
isfies the required range. The monotonicity of the functional follows from the
corresponding monotonicity of the Lebesgue measure, and its subadditivity is
demonstrated as follows: for any A ŒC,
It remains to show that the proposed functional is additive, to be fully justi-
fied as a general measure of nonspecificity of convex subsets of X in ⺢
n
for
any finite n ≥ 1.
To prove that the proposed functional is additive, we must prove that
HL A HL A
i
i
n
()
=
()
=
Â
1
HL A A A A
A
A
tT
ii
i
n
i
n
tT
i
i
n
tT
i
i
n
tt
t
t
()
=+
()
[]
+
()
-
()
È
Î
Í
˘
˚
˙
Ï
Ì
Ó
¸
˝
˛
£+
()
[]
È
Î
Í
˘
˚
˙
Ï
Ì
Ó
¸
˝
˛
=+
()
[]
Ï
Ì
Ó
¸
˝
Œ
==
Œ
=
Œ
=
’’
’
Â
min log
min log
min log
2
11
2
1
2
1
1
1
1
mm m
m
m
˛˛
=
()
=
Â
HL A
i
i
n
1
.
HL A HL A
i
i
n
()
=
()
=
Â
1
.
n
i=1
HL A HL A
i
i
n
()
£
()
=
Â
1
2.3. HARTLEY-LIKE MEASURE OF UNCERTAINTY FOR INFINITE SETS 47
for any A ŒC such that
A = A
i
. It has already been shown that
for any A ŒC. Hence, it remains to prove that
when
A = A
i
. This, in turn, amounts to proving that for any rotation of the
set A,
Since , this inequality can be written as
(2.39)
For n = 1, this inequality is trivially satisfied. For n = 2, 3, it is proved here
by directly examining the effect of rotation of the set
A = A
i
(n = 2, 3)
on projections. The proof for an arbitrary finite n is not presented here,
since it is based on some special results from convexity theory (see
Note 2.3).
Generally, in the n-dimensional space, ⺢
n
, any rotation can be represented
by the orthogonal matrix
where the parameters satisfy the following properties:
11 1
20
0
2
1
2
1
1
1
. cos , , cos , .
. cos cos , , ,
cos cos , , .
aa
aa
aa
ij
i
n
nij
j
n
n
ik jk
k
n
n
ki kj
k
n
n
ji
ij
ij
==
=
=
ÂÂ
Â
Â
="Œ ="Œ
="Œ
="Œ
⺞⺞
⺞
⺞
and
and
I =
È
Î
Í
Í
Í
˘
˚
˙
˙
˙
cos
cos
cos
aa a
aa a
aa a
11 12 1
21 22 2
12
cos
...
cos
cos
...
cos
cos
...
cos
,
n
n
nn nn
MMOM
n
i=1
11
1111
+
()
[]
-
()
≥+
()
[]
-
()
====
’’’’
mm mmAA AA
i
i
n
i
i
n
i
i
n
i
i
n
tt
.
mmAA
i
i
n
()
=
()
=
’
1
11
111
+
()
[]
+
()
-
()
≥+
()
[]
===
’’’
mm m mAA A A
i
i
n
i
i
n
i
i
n
tt
.
n
i=1
HL A HL A
i
i
n
()
≥
()
=
Â
1
HL A HL A
i
i
n
()
£
()
=
Â
1
n
i=1
48 2. CLASSICAL POSSIBILITY-BASED UNCERTAINTY THEORY
For each given rotation defined by matrix I, an arbitrary point
in ⺢
n
is transformed to the point
by the matrix equation
That is,
Let us consider, without any loss of generality, that
A = [0, a
1
]
for some a
i
Œ ⺢, i Œ ⺞
n
. Then, the ith projection of this set subjected to rota-
tion defined by the matrix I is the set
for any i Œ⺞
n
. The Lebesgue measure of the projection is
That is
for any i Œ⺞
n
. Since this maximum must be reached by two vertices of the set
A, the Lebesgue measure of the projection can be rewritten by
maAa
ikik
k
n
()
=
=
Â
cos
1
maAxy
i
A
jj ij
j
n
()
=-
()
{}
Œ
=
Â
max cos
,xy
1
m Axy
i
A
ii
()
=¢-¢
{}
Œ
max .
,xy
Ax A
ii
=¢¢= "Œ
{}
xIxx,
n
i=1
x
xx
nnnnn
1
12 2
¢
=+++
¢= + + +
¢= + + +
x cos x cos
...
x cos
x x cos x cos
...
x cos
x x cos cos
...
cos .
1 11 2 12 n 1n
21 212 22 n 2n
n1
aa a
aa a
aa a
M
¢=xIx.
¢= ¢ ¢ ¢x xx x
n
T
12
,,
...
,
x = xx x
n
T
12
,,
...
,
2.3. HARTLEY-LIKE MEASURE OF UNCERTAINTY FOR INFINITE SETS 49
for any i Œ⺞
n
.
Using the last formula, let us examine some aspects of the proposed func-
tion pertaining to its additivity. First let us present the following two basic
properties:
(a) This is because
(b) This is because of the fact that the Lesbesgue measure
of the set is less than or equal to the Lesbesgue measure of the
Cartesian product of its 1-dimensional projections.
The Two-Dimensional Case Let set A be a rectangle in the standard coor-
dinate system that is shown in Figure 2.3. Since A = [0, a
1
] ¥ [0, a
2
] in this
system, we have
Now we prove that this is the minimum for all rotations. In the 2-dimensional
space, any rotation can be represented by the matrix
Figure 2.3 illustrates a rotated rectangle A and its projections. It is easy to show
that
I =
-
È
Î
Í
˘
˚
˙
cos sin
sin cos
.
qq
qq
111
1
2
1
2
12
+
()
[]
+
()
-
()
=+
()
+
()
==
’’
mm mAA A aa
i
i
i
i
tt
.
m Aa
i
i
n
i
i
n
()
≥
==
’’
11
.
ma
a
a
a
Aa
a
a
a
a
i
i
n
kik
k
n
i
n
kik
i
n
k
n
kik
i
n
k
n
kik
i
n
k
n
k
k
n
()
=
=
≥
=
=
===
==
==
==
=
ÂÂÂ
ÂÂ
ÂÂ
ÂÂ
Â
111
11
2
11
2
11
1
cos
cos
cos
cos
.
m Aa
i
i
n
i
i
n
()
≥
==
ÂÂ
11
.
50 2. CLASSICAL POSSIBILITY-BASED UNCERTAINTY THEORY
Then under the new coordinate system,
Therefore, in the 2-dimensional case the measure is additive.
The Three-Dimensional Case To prove the additivity in the 3-dimensional
space, we only need to prove that for any rotation of set A ΠC,
mm mm mmAA AA AAaaaaaa
12 13 23121322
()()
+
()()
+
()()
≥++.
1
1
1
1
2
1
2
1212
1212 12
1
2
2
2
1
2
2
2
1
2
+
()
[]
+
()
-
()
=+ + + +
++
()
+
()
+
≥+ + + +
++
==
’’
mm m
qqq q
qqq q
qqq q
q
AA A
aaaa
aaaa aa
aaaa
aa
i
i
i
i
tt
cos sin sin cos
cos sin sin cos
cos sin sin cos
cos
22
2
1
2
2
2
12
12
11
sin sin cos
.
qq q
()
+
()
+
≥+
()
+
()
aa aa
aa
mqq
mqq
Aa a
Aa a
11 2
21 2
()
=+
()
=+
cos sin
sin cos .
2.3. HARTLEY-LIKE MEASURE OF UNCERTAINTY FOR INFINITE SETS 51
Y
X
A
2
q
A
A
1
Figure 2.3. Rotation of set A in ⺢
2
.