Klir G.J. Uncertainity and Information. Foundations of Generalized Information Theory

Подождите немного. Документ загружается.

Theorem 6.1. Let r be a possibility proﬁle on X ¥ Y, and let r

and r

be the

associated marginal possibility proﬁles on X and Y, respectively. Then,

(6.13)

where the equality is obtained if and only if r

and r

are noninteractive.

Proof

It is clear that the equality is obtained if and only if

r =

for all

a Œ [0, 1], which is the case when marginal possibility proﬁles are

noninteractive. 䊏

EXAMPLE 6.2. In Figure 6.2a, a joint possibility proﬁle r on X ¥ Y = {x

, x

} ¥ {y

, y

} is given and the associated marginal possibility proﬁles r

and

are derived from it. In Figure 6.2b, a joint possibility proﬁle, r¢, is derived

Urd

rrd

rd rd

()

£¥

=◊

ÚÚ

log

log log

() ().

UU U

rr r

()

202 6. MEASURES OF UNCERTAINTY AND INFORMATION

1.0 0.7 0.2

0.7 0.7 0.0

1.0 0.8 0.9

0.7 0.7 0.7

1.0 0.8 0.9

1.0

0.7

1.0

(a) (b)

1.0

0.7

1.0

Figure 6.2. Joint and marginal possibility proﬁles (Example 6.2).

from r

and r

under the assumption that they are noninteractive. Ordering

the individual proﬁles and applying Eq. (6.2), we obtain

We can see that U(r) < U(r¢) = U(r

) + U(r

), which veriﬁes the subadditivity

and additivity of the U-uncertainty.

6.2.2. Conditional U-Uncertainty

Applying the deﬁnitions of conditional Hartley measures, expressed by Eqs.

(2.19) and (2.20), to sets

, and

, we readily obtain their generalized

counterparts for the U-uncertainty:

(6.14)

(6.15)

It is obvious that these equations can also be rewritten as

(6.16)

(6.17)

Let the generic symbols U(X), U(Y), U(X ¥ Y), U(X | Y), and U(Y | X) now

be used instead of their speciﬁc counterparts U(r

), U(r

), and

U(r

| r

), respectively. This is a common practice in the literature, and is used

in Chapter 2 for the marginal, joint, and conditional Hartley measures and

then again in Chapter 3 for the Shannon entropies. This notation makes it

easier to recognize that marginal, joint, and conditional uncertainties based on

distinct measures of uncertainty are related in analogous ways.

Using the generic symbols, Eqs. (6.16) and (6.17) are rewritten as

(6.18)

(6.19)

Observe that these equations are analogous to Eqs. (2.23) and (2.24) for the

Hartley measure, as well as to Eqs. (3.35) and (3.36) for the Shannon entropy.

UY X UY X UX

()

=¥

()

UX Y UX Y UY

()

=¥

()

UUU

YX X

rr r r

()

UUU

XY Y

rr r r

()

log .

()

log ,

()

=++++ =

()

=+++=

()

=+=

()

01 2 01 3 01 4 05 7 02 8 246

01 2 01 4 01 6 07 9 278

01 2 08 3 137

22222

2222

. log . log . log . log . log . ,

. log . log . log . log . ,

. log . log . ,

. log

222

207 3141+=. log . .

6.2. GENERALIZED HARTLEY MEASURE FOR GRADED POSSIBILITIES 203

Observe also that the conditional U-uncertainties can be calculated without

using conditional possibilities.

U-uncertainty analogies of other equations that hold for the Hartley (and

for the Shannon entropy) can be easily derived as well. For example, when the

marginal possibility proﬁles are noninteractive, we have the equation

(6.20)

by Theorem 6.1, which is analogous to Eq. (2.27) for the Hartley measure.

When we combine this equation with Eqs. (6.18) and (6.19), we readily obtain

the equations

(6.21a)

(6.21b)

which are analogous to Eqs. (2.25) and (2.26) for the Hartley measure. Simi-

larly, the inequality

(6.22)

which is a generic form of Eq. (6.13), is analogous to Eq. (2.30) for

Hartley measure and to Eq. (3.38) for the Shannon entropy. We can also

deﬁne information transmission, T

(X, Y), for the U-uncertainty by the

equation

(6.23)

in analogy with Eq. (2.31) for the Hartley measure and Eq. (3.34) for the

Shannon entropy.

EXAMPLE 6.3. Consider the joint and marginal possibility proﬁles in Figure

6.2a. Applying Eqs. (6.18) and (6.19) and utilizing the joint and marginal U-

uncertainties calculated in Example 6.2, we have

Similarly, for the possibility proﬁles in Figure 6.2b, we obtain

as is expected under the assumption of noninteraction.

UX Y UX

UY X UY

TXY

()

=-==

()

[]

()

=-==

()

[]

()

=--=

278 141 137

278 138 141

278 137 141 0

... ,

,...,

UX Y

UY X

TXY

()

=-=

()

=-=

()

=+-=

246 141 105

246 137 109

137 141 246 032

...,

,.....

TXY UX UY UX Y

()

-¥

()

UX Y UX UY¥

()

UY X UY

()

UX Y UX

()

UX Y UX UY¥

()

204 6. MEASURES OF UNCERTAINTY AND INFORMATION

6.2.3. Axiomatic Requirements for the U-Uncertainty

As any measure of uncertainty, the U-uncertainty must satisfy the axiomatic

requirements listed in Section 6.1. In this case, the requirements must be

expressed in terms of the calculus of graded possibilities. If we consider only

ﬁnite sets, then the requirement of coordinate invariance is not relevant.

Hence, we have the following nine speciﬁc requirements:

Axiom (U1) Subadditivity. For any joint possibility proﬁle r on X ¥ Y, U(r)

£ U(r

) + U(r

Axiom (U2) Additivity. When a joint possibility proﬁle r on X ¥ Y is derived

from noninteractive marginal possibility proﬁles r

and r

, then U(r) = U(r

)

+ U(r

Axiom (U3) Monotonicity. For any pair

r of possibility proﬁles of the

same length, if

r £

r, then U(

r) £ U(

r).

Axiom (U4) Continuity. U is a continuous functional.

Axiom (U5) Expansibility. When components of zero are added to a given

possibility proﬁle, the value of U does not change.

Axiom (U6) Symmetry. If r is a possibility proﬁle and r¢ is a permutation of

r, then U(r) = U(r¢).

Axiom (U7) Range. U(r) Œ [0, M], where the minimum and maximum are

obtained for r

min

=·1,0,...,0Ò and r

max

=·1,1,...,1Ò, respectively, and the

value M depends on the cardinality of the universal set and the chosen unit

of measurement (normalization).

Axiom (U8) Branching/Consistency. For every possibility proﬁle r =·r

, r

...,r

Ò of any length n,

(6.24)

for each k = 3,4,...,n, where, for any given integer i, 1

denotes a sequence

of i ones and 0

denotes a sequence of i zeroes.

Axiom (U9) Normalization. To deﬁne bits as measurement units, it is

required that U(1, 1) = 1.

The branching requirement, which can also be formulated in other forms,

needs some explanation. This requirement basically states that the U-

uncertainty must have the capability of measuring possibilistic nonspeciﬁcity

in two ways. It can be measured either directly for the given possibility proﬁle

UrrrUrrrrrr r

rrU

n k kkk n

kk k

kkknk

12 12 2 1

222

, ,..., , ,..., , , , ,...,

()

()( )

---+

6.2. GENERALIZED HARTLEY MEASURE FOR GRADED POSSIBILITIES 205

or indirectly by adding U-uncertainties associated with a combination of pos-

sibility proﬁles that reﬂect a two-stage measuring process. In the ﬁrst stage of

measurement, the distinction between two neighboring components, r

k-1

and

, is ignored (r

k-1

is replaced with r

) and the U-uncertainty of the resulting,

less reﬁned possibility proﬁle is calculated. In the second stage, the U-uncer-

tainty is calculated in a local frame of reference, which is deﬁned by a possi-

bility proﬁle that distinguishes only between the two neighboring possibility

values that are not distinguished in the ﬁrst stage of measurement. The U-

uncertainty calculated in the local frame must be scaled back to the original

frame by a suitable weighting factor. The sum of the two U-uncertainties

obtained by the two stages of measurement is equal to the total U-uncertainty

of the given possibility proﬁle.

The ﬁrst term on the right-hand side of Eq. (6.24) represents the U-

uncertainty obtained in the ﬁrst stage of measurement. The remaining two

terms represent the second stage, associated with the local U-uncertainty. The

ﬁrst of these two terms expresses the loss of uncertainty caused by ignoring

the component r

k-1

in the given possibility proﬁle, but it introduces some addi-

tional U-uncertainty equivalent to the uncertainty of a crisp possibility proﬁle

with k - 2 components. This additional U-uncertainty is excluded by the last

term in Eq. (6.24). That is, the local uncertainty is expressed by the last two

terms in Eq. (6.24).

The meaning of the branching property of the U-uncertainty is illustrated

by Figure 6.3. Four hypothetical possibility proﬁles involved in Eq. (6.24) are

shown in the ﬁgure, and in each of them the local frame involved in the branch-

ing is indicated.

It turns out that ﬁve of the nine axiomatic requirements are sufﬁcient to

prove the uniqueness of the U-uncertainty: additivity, monotonicity, expansi-

bility, branching, and normalization. The proof, which is quite intricate, is

covered in Appendix A.

6.2.4. U-Uncertainty for Inﬁnite Sets

U-uncertainty for inﬁnite sets is a generalization of the Hartley-like measure

HL (introduced in Section 2.3) for graded possibilities. It is thus reasonable

to denote it by UL. This generalization is obtained directly from Eq. (6.6) by

replacing log

r| with HL(

r). Hence,

(6.25)

or, more speciﬁcally,

(6.26)

where

r is deﬁned in terms of r by Eq. (6.5).

UL r r r d

()

()()

()

()(

’’

min log ,

1 mm ma

aa a

UL HL r dr

()

206 6. MEASURES OF UNCERTAINTY AND INFORMATION

1.0

0.0

Local

frame

1.0

0.0

Local

frame

1.0

0.5

0.0

123

. . .

k–2 k–1 kk+1 k+2

. . .

Local

frame

k 1

1.0

0.5

0.0

Local

frame

123

. . .

k–2 k–1 kk+1 k+2

. . .

123

. . .

k–2 k–1 kk+1 k+2

. . .

123

. . .

k–2 k–1 kk+1 k+2

. . .

k 2

k 1

k 2

–

Figure 6.3. Possibility proﬁles involved in the branching property.

EXAMPLE 6.4. Let the range of a real-valued variable x be [0, 10]. Assume

that we are able to predict the value of x only approximately in terms of the

possibility proﬁle

for each x Œ [1, 10]. The graph of this possibility proﬁle is shown in Figure 6.4.

For each a Œ [0, 1],

r is in this case a closed interval [x(a), x

(a)] of real

numbers. The endpoints of these intervals for all a Œ [0, 1] are functions of a,

which can be obtained by solving the equation

for x. The solution is

Clearly, and . Since we deal with a 1-

dimensional variable, Eq. (6.26) assumes the form

where . Hence,

UL dr

()

=+-

[]

log . .

1 2 1 1 1887aa

aa arx x=

()

=-21

UL r dr

()

[]

log ,

1 ma

x aa

()

=+ -41x aa

()

=- -41

.=± -a

23 3

xx-

()

= a

rx x x

()

{}

max ,02 3 3

208 6. MEASURES OF UNCERTAINTY AND INFORMATION

0 2 4 6 8 10

0.2

0.4

0.6

0.8

r(x)

Figure 6.4. Possibility proﬁle in Example 6.4.

6.3. GENERALIZED HARTLEY MEASURE IN

DEMPSTER–SHAFER THEORY

Once the U-uncertainty was well established as a generalized Hartley measure

for graded possibilities, its further generalization to Dempster–Shafer theory

(DST) became conceptually fairly straightforward. It emerged quite naturally

from two simple facts: (1) the U-uncertainty is a weighted average of the

Hartley measure for all focal subsets; and (2) the weights in this average, which

are expressed by the differences r

- r

i+1

in ordered possibility proﬁles, are

values of the basic probability assignment function.Although the focal subsets

are always nested in the theory of graded possibilities, the concept of the

weighted average of the Hartley measure is applicable to any family of focal

subsets. The generalized Hartley measure, GH, in DST is thus deﬁned by the

functional

(6.27)

where ·F, mÒ is any arbitrary body of evidence in the sense of DST.

Observe that the functional GH is deﬁned in terms of m while the func-

tional U is deﬁned in terms of r. However, the difference r

- r

i+1

, in Eq. (6.1)

is clearly equal to m(A

), which means that the U-uncertainty can be deﬁned

in terms of the basic probability assignment as well.

It is obvious that the GH measure deﬁned by Eq. (6.27) is a continuous

functional, which satisﬁes the expansibility requirement and whose range is

(6.28)

The lower bound is obtained when all focal subsets are singletons, which

means that m is actually a probability distribution on X. This implies that

GH(m) = 0 for all probability measures. That is, probability measures are fully

speciﬁc. We can also see that GH(m) = 1 when m(A) = 1 and |A|=2, which

means that the units in which the functional GH measures nonspeciﬁcity are

bits. For characterizing the functional, this property must be required as a

normalization.

The GH measure is also invariant with respect to permutations of values

of the basic probability assignment function within each group of subsets of

X that have equal cardinalities. This invariance is, in fact, the meaning of the

requirement of symmetry in DST.

The issue of the uniqueness of the GH measure in DST is addressed in

Appendix B.

6.3.1. Joint and Marginal Generalized Hartley Measures

The functional GH deﬁned by Eq. (6.27) is also subadditive and additive, as

is established by the following two theorems.

()

£GH m Xlog .

GH m m A A

()

log ,

6.3. GENERALIZED HARTLEY MEASURE IN DEMPSTER–SHAFER THEORY 209

Theorem 6.2. For any joint basic probability assignment function m on X ¥ Y

and its associated marginal functions m

and m

(6.29)

Proof. Recalling Eqs. (5.54) and (5.55), we have

Similarly,

Hence,

䊏

Theorem 6.3. Let m

and m

be noninteractive basic probability assignment

functions on subsets of X and Y, respectively, and let m be the associated joint

basic probability assignment function. Then,

(6.30)

Proof. Recalling Eq. (5.56), we have

GH m m C C

mAmB AB

CXY

BYAX

()

◊

()

◊

()

◊

()

Õ¥

ÕÕ

ÂÂ

log

GH m GH m GH m

()

GH m GH m m C C C

mC C C

mC C

GH m

XY XY

CXY

()

◊

()

≥

() ()

()

Õ¥

log log

log

GH m m C C

CXY

()

Õ¥

log .

GH m m A A

mC A

mC C

CACAX

CXY

()

=Õ

Õ¥

ÂÂ

log

log .

GH m GH m GH m

()

210 6. MEASURES OF UNCERTAINTY AND INFORMATION

䊏

According to the common notational convention in the literature, it is con-

venient to replace the speciﬁc symbols GH(m), GH(m

), and GH(m

) with

their generic counterparts GH(X ¥ Y), GH(X), and GH(Y). The conditional

GH measures, addressed in Section 6.3.3, are then denoted as GH(X|Y) and

GH(Y|X).

6.3.2. Monotonicity of the Generalized Hartley Measure

Bodies of evidence in DST can be partially ordered on the basis of set inclu-

sion. This ordering is then used for deﬁning the requirement of monotonicity

for the GH measure. For any pair ·F

, m

Ò and ·F

, m

Ò of bodies of evidence on

X, ·F

, m

Ò is said to be smaller than or equal to ·F

, m

Ò, which is written as

if and only if the following requirements are satisﬁed:

(a) For each A ŒF

, there exists some B ŒF

such that A 債 B;

(b) For each B ŒF

there exists some A ŒF

such that A 債 B;

implies A 債 B and

(6.31)

(6.32)

EXAMPLE 6.5. To illustrate the deﬁnition of ordering of bodies of evidence,

especially requirement (c) in the deﬁnition, consider the following two bodies

of evidence on X = {x

, x

111 2 12 13

12 112 113

222 13 23

213

02 02 06

, : ,, ,,

., , ., , .

, : ,,,,

mxxxxx

mx mxx mxx

mxxxxX

mxx

{}{ }{ }{}

{}()

{}{ }{}

{}()

= 550104

223 2

,,., ..mxx mX

{}()

()

mB fAB B X

AA B

()

,.for each

mA fAB A X

BA B

()

,,for each

11 22

,,,mm£

()

◊

()

() ()

◊+

() ()

◊

()

◊+

()

◊

()

ÕÕ

ÕÕ Õ Õ

ÕÕ

ÂÂ

ÂÂ ÂÂ

ÂÂ

mAmB A B

mB m A A m A mB B

mA A mB B

GH m GH m

BYAX

log log

6.3. GENERALIZED HARTLEY MEASURE IN DEMPSTER–SHAFER THEORY 211