Klir G.J. Uncertainity and Information. Foundations of Generalized Information Theory

Подождите немного. Документ загружается.

EXAMPLE 6.18. Consider the credal set

D in Example 4.5, which is deﬁned

by convex combinations of four extreme points:

These points are obtained via the interaction representation of the given lower

probability function m

, as is shown in Figure 4.5. Clearly, S(p

) = 1.485, S(p

)

= 1.361, S(p

) = 1.571, and S(p

) = 1.522. Hence, S(

D) = S(p

) = 1.361. Using

Algorithm 6.1, it is easy to compute S

(

D) = S(0.3, 0.3, 0.4) = S(p

) = 1.571.

Then,

A pictorial overview of justiﬁed measures of uncertainty in GIT is given in

Figure 6.15. Although other disaggregations of S

are possible, at least in prin-

ciple, the two shown in the ﬁgure, TU and

TU, seem to be the best choices on

both intuitive and mathematical grounds. Needless to say that the issue of how

to disaggregate S

has not been sufﬁciently investigated as yet.

0 21 1 361D

()

= .,. .

02 03 05

04 01 05

03 03 04

04 02 04

., ., . ,

., ., . .

252 6. MEASURES OF UNCERTAINTY AND INFORMATION

Classical

uncertainty

theories

Generalized

uncertainty

theories

Aggregation

Disaggregations

= ·

–

= ·

–

S, S

GS N

Classical

possibility

theory: H

Classical

probability

theory: S

Figure 6.15. An overview of justiﬁed measures of uncertainty in GIT.

6.11. UNIFYING FEATURES OF UNCERTAINTY MEASURES

The formalization of uncertainty functions involves a considerable diversity,

as is demonstrated in Chapter 5. However, it also involves some unifying fea-

tures, which are examined in Chapter 4. Functionals that for each type of

uncertainty function measure the amount of uncertainty are equally diverse

while, at the same time, they also exhibit some unifying features. The follow-

ing are some of these unifying features:

1. The aggregate uncertainty S

is deﬁned by one formula, expressed by Eq.

(6.61), regardless of the uncertainty theory involved.

2. The deﬁnition of the generalized Hartley functional GH, when expressed

by Eq. (6.38), does not change when we move from one uncertainty to

another.

3. The measure of conﬂict, S, is deﬁned in the same way, expressed by Eq.

(6.73), regardless of the uncertainty theory involved.

4. It follows from features 1 to 3 that the two versions of the disaggregated

total uncertainty, TU, and

TU, are deﬁned in the same way in all theo-

ries of uncertainty.

5. In all the generalized theories of uncertainty that have been adequately

developed, we observe that the various equations established in classi-

cal uncertainty theories, which express how joint, marginal, and condi-

tional uncertainty measures (and also the various information

transmissions) are interrelated, hold as well.

These unifying features of uncertainty measures allow us to work with any set

of recognized and well-developed theories of uncertainty as a whole. Some

aspects of this issue are discussed further in Chapter 9.

NOTES

6.1. The U-uncertainty was proposed by Higashi and Klir [1983a]. They also formu-

lated the axiomatic requirements for U-uncertainty given in Section 6.2.3 (except

the branching axiom) and proved that the functional deﬁned by Eq. (6.2) or, alter-

natively, Eq. (6.6), satisﬁes them.The possibilistic branching requirement was for-

mulated by Klir and Mariano [1987], and they used it for proving the uniqueness

of the U-uncertainty. It was shown by Ramer and Lander [1987] that the branch-

ing axiom is essential for obtaining a unique generalization of the Hartley

measure for the theory of graded possibilities.

6.2. The generalization of the U-uncertainty to Dempster–Shafer theory was sug-

gested by Dubois and Prade [1985c]. The uniqueness of this generalized Hartley

measure, deﬁned by Eq. (6.27), was proved by Ramer [1986, 1987]. The measure

was also investigated and compared with other measures by Dubois and Prade

[1987d].

NOTES 253

6.3. An alternative measure of nonspeciﬁcity for graded possibilities, which is not a

generalization of the Hartley measure, was proposed by Yager [1982c, 1983]. It is

a functional a deﬁned by the formula

where ·F, mÒ is a given body of evidence in Dempster–Shafer theory. This func-

tional does not satisfy some of the axiomatic requirements for a measure of

uncertainty. In particular, it does not satisfy the requirements of subadditivity and

additivity.

6.4. The ordering of bodies of evidence in DST was introduced by Dubois and Prade

[1986a, 1987d]. A broader discussion of the concept of ordering bodies of evi-

dence in DST is in [Dubois and Prade, 1986a].

6.5. The formulation of the symmetry and branching axioms and the proof of unique-

ness of the generalized Hartley measure in DST, which is presented in Section

6.3 and in Appendix B, are due to Ramer [1986, 1987]. Ramer [1990a] also studied

the relationship among the various axiomatic requirements for the generalized

Hartley measure in possibility theory and DST.

6.6. The generalization of the Hartley measure for convex sets of probability

distributions was proposed by Abellán and Moral [2000]. Relevant proofs

regarding properties of this generalized measure are covered in the paper.

Among them, the proof of monotonicity of the measure is the most complex one.

Contrary to the claims in the paper, the generalized Hartley functional is not sub-

additive for arbitrary credal sets, as is demonstrated in Example 6.7. However,

this is no deﬁciency in the context of the disaggregated measure of total uncer-

tainty TU.

6.7. The following is a chronological list of main publications regarding the various

unsuccessful attempts to generalize the Shannon entropy to DST that are sur-

veyed in Section 6.5:

•

Höhle [1982] proposed the measure of confusion C;

•

Yager [1983] proposed the measure of dissonance E;

•

Lamata and Moral [1988] suggested the sum E + GH;

•

Klir and Ramer [1990] proposed the measure of discord D and further inves-

tigated its properties as well as the properties of D + GH in Ramer and Klir

[1993];

•

Properties of discord in possibility theory were investigated by Geer and Klir

[1991];

•

Further examination of the measure of discord by Klir and Parviz [1992]

revealed that it had a conceptual defect, and to correct it they suggested the

measure of strife ST;

•

Klir and Yuan [1993] showed that the distinction between strife and discord

reﬂects the distinction between disjunctive and conjuctive set–valued state-

ments. The latter distinction is discussed, for example, in [Yager, 1987a];

•

The violation of subadditivity was demonstrated for D + GH by Vejnarová

[1991], and for ST + GH by Vejnarová and Klir [1993];

()

254 6. MEASURES OF UNCERTAINTY AND INFORMATION

•

Functional AS based on the interaction representation was suggested by Yager

[2000] and Marichal and Roubens [2000] and its violation of subadditivity is

shown in [Klir, 2003].

6.8. The idea of the aggregated measure of uncertainty in DST, which is discussed in

Section 6.6, emerged in the early 1990s. It seems that it was conceived indepen-

dently and almost simultaneously by several authors. The following are relevant

publications: [Chokr and Kreinovich, 1991, 1994], [Maeda and Ichihashi, 1993],

[Chau, et al., 1993], [Harmanec and Klir, 1994], and [Abellán and Moral, 1999].

Algorithms for computing the functional AU were investigated by Maeda et al.

[1993], Meyerowitz et al. [1994], and Harmanec et al. [1996]; the algorithm and

its proof of correctness that are presented in Sections 6.6.1 and 6.6.2 and Appen-

dix C are adopted from the last reference. Harmanec [1995, 1996] made progress

toward the proof of uniqueness of AU by proving that AU is the smallest func-

tional among those that satisfy the requirements (AU1)–(AU5), if any of them

exist except AU. Abellán and Moral [2003a] generalized the aggregate measure

of uncertainty to arbitrary credal sets and developed a more efﬁcient algorithm

for computing it within the theory based on reachable interval-valued probabil-

ity distributions.They also illustrated a practical utility of the aggregated measure

[Abellán and Moral, 2003b].

6.9. The disaggregated total uncertainty introduced in Section 6.8 was proposed and

investigated by Smith [2000].A part of his investigations is the proof of the proper

range of the generalized Shannon entropy that is presented in Appendix D.

6.10. The importance of both

S and S

is discussed in Kapur et al. [1995] and in Kapur

[1994, Chapter 23].Also discussed in these references is the role of the difference

- S. More recently, Abellán and Moral [2004, 2005] investigated further prop-

erties of this difference and described an algorithm for calculating the value of

S for any lower probability that is 2-monotone.They also suggested using the dif-

ference as a measure of nonspeciﬁcity.This suggestion led to the alternative view

of disaggregated total uncertainty, which is discussed in Section 6.10.

EXERCISES

6.1. Derive Eq. (6.3) from Eq. (6.2).

6.2. For each of the possibility proﬁles on X in Table 6.5a, calculate:

(a) The U-uncertainty by Eq. (6.2)

(b) The U-uncertainty by Eq. (6.6)

6.3. For each pair of marginal possibility proﬁles on X and Y in Table 6.5b,

calculate

(a) The marginal U-uncertainties;

(b) The joint U-uncertainty under the assumption of noninteraction of

the marginal possibility proﬁles.

6.4. Express each of the marginal possibility proﬁles in Table 6.5b as a body

of evidence in DST and calculate its nonspeciﬁcity. Then, determine by

EXERCISES 255

the calculus of DST for each pair of these marginal bodies of evidence

on X and Y the corresponding joint body of evidence under the assump-

tion of their noninteraction and calculate:

(a) Its nonspeciﬁcity (compare these values with their counterparts in

Exercise 6.3b);

(b) Its aggregate measure of uncertainty AU.

256 6. MEASURES OF UNCERTAINTY AND INFORMATION

Table 6.5. Supporting Information for Various Exercises

r 1.0 0.0 0.3 0.7 0.9 1.0 0.5 0.5

r 0.4 0.5 0.5 1.0 0.8 0.8 0.6 0.5

r 0.6 0.6 0.5 0.4 0.3 1.0 1.0 1.0

r 0.7 0.9 1.0 1.0 1.0 0.9 0.7 0.5

(a)

0.6 1.0 0.8 1.0 0.4 0.9 y

1.0 0.5 0.0 1.0 0.7 1.0 y

1.0 0.3 0.2 0.5 1.0 0.8 y

(b)

0.0 0.5 0.7 0.0 y

1.0 0.5 0.6 1.0

0.8 1.0 0.9 0.7

0.7 0.0 0.0 0.8

0.6 0.7 0.8 1.0 y

0.7 0.4 0.5 0.7

0.4 0.6 0.8 1.0 y

1.0 0.8 0.9 1.0

0.3 0.6 0.9 1.0

0.5 0.7 0.8 0.9

0.5 0.8 0.7 0.7

0.3 0.9 0.6 0.4

(c)

6.5. For each of the joint possibility proﬁles in Table 6.5c, calculate

(a) The joint U-uncertainty and its associated marginal U-uncertainties;

(b) Both conditional U-uncertainties;

6.6. For each of the joint possibility proﬁles in Table 6.5c, calculate:

(a) The aggregate measure of uncertainty AU and its marginal

counterparts;

(b) The total disaggregated uncertainty TU and its marginal

counterparts.

6.7. Repeat Exercise 6.5 for the joint possibility proﬁle derived from

the respective marginal possibility proﬁles under the assumption of

noninteraction.

6.8. For each of the possibility proﬁles in Table 6.5a and some k ≥ 3, apply

the branching axiom in Eq. (6.24) for calculating the U-uncertainty in

two stages.

6.9. Let the range of real-valued variable x be [0, 100]. Assume that we are

able to assess the value of the variable only approximately in terms of

the possibility proﬁle

Plot the possibility proﬁle and calculate:

(a) The nonspeciﬁcity of this assessment;

(b) The amount of information obtained by this assessment.

6.10. Repeat Exercise 6.9 for the following possibility proﬁles and ranges of

the variable

(a) x Œ [-10, 10] and

(b) x Œ [0, 20] and

6.11. For each of the Möbius representations of the joint lower probability

functions on subsets X ¥ Y = {x

, x

} ¥ {y

, y

} that are deﬁned in Table

6.6, where z

=·x

, y

Ò for all i, j = 1, 2, calculate:

(a) The generalized Hartley measure GH(X ¥ Y) deﬁned by Eq. (6.38);

(b) The generalized Shannon measure GS(X ¥ Y) deﬁned by Eq. (6.64).

()

[]

()

[]

222

when

otherwise;

()

-Œ-

[]

{

111

when

otherwise;

()

[]

when

otherwise.

EXERCISES 257

6.12. Repeat Exercise 6.11 for:

(a) The generalized marginal Hartley and Shannon measures;

(b) The generalized conditional Hartley and Shannon measures;

Shannon measures.

6.13. Repeat Exercise 6.11 by calculating the following:

(a) The measure of dissonance deﬁned by Eq. (6.41);

(b) The measure of confusion deﬁned by Eq. (6.42);

(d) The measure of strife deﬁned by Eq. (6.55);

(e) The measure of average Shannon entropy deﬁned by Eq. (6.59).

6.14. Compare S

(m), TU(m), and

TU(m) for the simple body of evidence

illustrated in Figure 6.9.

6.15. For each of the convex sets of probability distributions discussed in

Example 6.6 (see also Table 6.1), compare S

(

D), TU(

D), and

TU(

D).

6.16. Assume that a given body evidence ·F, mÒ consists of pairwise disjoint

focal sets. Show that under this assumption

where GS is the generalized Shannon entropy deﬁned by S

GS m m A m A

()

() ()

log ,

258 6. MEASURES OF UNCERTAINTY AND INFORMATION

Table 6.6. Möbius Representations Employed in Exercises 6.11–6.13

(C)

C:0000 0.0 0.0 0.0 0.0 0.00.00

1000 0.1 0.0 0.0 0.1 0.00.15

0100 0.1 0.1 0.1 0.1 0.00.03

0010 0.2 0.1 0.0 0.0 0.00.00

0001 0.4 0.2 0.2 0.3 0.00.10

1100 0.0 0.0 0.3 0.0 0.30.11

1010 0.0 0.2 0.0 0.0 0.00.00

1001 0.0 0.2-0.2 0.0 0.0 0.30

0110 0.0 0.1-0.1 0.2 0.0 0.00

0101 0.0 0.1 0.0 0.0 0.00.07

0011 0.0 0.3 0.3 0.1 0.50.00

1110 0.1 0.0 0.0 0.1 0.20.00

1101 0.1 0.0 0.2 0.1 0.20.22

1011 0.0 -0.2 0.1 0.1 0.1 0.00

0111 0.1 -0.1 0.1 0.1 0.1 0.00

1111 -0.1 0.0 0.0 -0.2 -0.4 0.02

6.17. Consider the convex sets of probability distributions on X = {x

, x

}

deﬁned by convex hulls of the following extreme points:

(a)

p =·0.4, 0.5, 0.1Ò;

p =·0.6, 0.3, 0.1Ò;

p =·0.6, 0.2, 0.2Ò;

p =·0.4, 0.2, 0.4Ò;

p =·0.1, 0.5, 0.4Ò

(b)

p =·0.4, 0.5, 0.1Ò;

p =·0.4, 0.2, 0.4Ò;

p =·0.1, 0.5, 0.4Ò

(c)

p =·0.4, 0.5, 0.1Ò;

p =·0.6, 0.2, 0.2Ò;

p =·0.4, 0.2, 0.4Ò

(d)

p =·0.4, 0.5, 0.1Ò;

p =·0.4, 0.2, 0.4Ò

(e)

p =·0, 0.2, 0.8Ò;

p =·0.2, 0.2, 0.8Ò

p =·0.5, 0.1, 0.5Ò;

p =·0, 0.5, 0.5Ò

(f)

p =·0, 0.5, 0.5Ò;

p =·0.5, 0, 0.5Ò;

p =·0.5, 0.5, 0.5Ò

(g)

p =·0, 1, 0Ò;

p =·0, 0.5, 0.5Ò;

p =·0.5, 0.5, 0Ò

(h)

p =·0, 1, 0Ò;

p =·1, 0, 0Ò;

p =·0.4, 0.4, 0.4Ò

For each of the convex sets of probability distribution functions, calcu-

late TU and

TU.

6.18. Calculate TU and

TU for the interval-valued probability distribution

I =·[0, 0.2], [0, 0.6], [0.3, 0.5], [0.2, 0.4]Ò on X = {x

, x

EXERCISES 259

FUZZY SET THEORY

260

Vagueness is no more to be done away with in the world of logic than friction in

mechanics.

—Charles Sanders Peirce

7.1. AN OVERVIEW

The notion of the most common type of fuzzy sets, referred to as standard

fuzzy sets, is introduced in Section 1.4. Recall that each standard fuzzy set is

uniquely deﬁned by a membership function of the form

where X is the universal set of concern. For each x Œ X, the value A(x)

expresses the degree (or grade) of membership of the element x of X in stan-

dard fuzzy set A. For the sake of notational simplicity, the symbol of a given

membership function, A, is also employed as a label of the standard fuzzy set

deﬁned by this function. It is obvious that no ambiguity is introduced by this

double use of the same symbol.

Recall also from Chapter 1 that classical sets, when viewed as special fuzzy

sets, are usually referred to as crisp sets. It is common in the literature to

describe a standard fuzzy set A on a ﬁnite universal set X by the special form

Aax ax ax

=+ ++

11 2 2

...

AX:,,Æ

[]

Uncertainty and Information: Foundations of Generalized Information Theory, by George J. Klir

where x

Œ X for all i Œ ⺞

and each a

denotes the degree of membership of

element x

in A. Each slash is employed in this form to link an element of X

with its degree of membership in A, and the plus signs indicate that the listed

pairs collectively form the deﬁnition of the set A. The pairs with zero mem-

bership degrees are usually not listed.

The purpose of this chapter is to introduce the fundamentals of fuzzy set

theory. This background is needed for understanding how the two classical

uncertainty theories (surveyed in Chapters 2 and 3) and their generalizations

based on the various types of monotone measures (discussed in Chapters 4–6)

can be fuzziﬁed. In other words, this background is needed to understand how

these various uncertainty theories, all described in terms of the formalized

language of classical set theory, can be further generalized via the more ex-

pressive formalized languages of fuzzy set theory. This generalization is man-

ifested in the 2-dimensional array in Figure 1.3 by the horizontal expansion of

column 1.

The common feature of all fuzzy sets is that the membership of any rele-

vant object in any fuzzy set is a matter of degree. However, there are distinct

ways of expressing membership degrees, which result in distinct categories of

fuzzy sets. In standard fuzzy sets, membership degrees are expressed by real

numbers in the unit interval. In other, nonstandard fuzzy sets, they may

be expressed by intervals of real numbers, partially ordered qualitative

descriptors of membership degrees, and in numerous other ways. Given any

particular category of fuzzy sets, operations of set intersection, union, and com-

plementation are not unique. Further distinctions within the category can thus

be made by choosing various speciﬁc operations from the class of possible

operations. Each choice induces an algebraic structure of some type on the

given category of fuzzy sets.These algebraic structures are always weaker than

Boolean algebra, that is, they are non-Boolean. The term “fuzzy set theory”

thus stands for a collection of theories, each dealing with fuzzy sets in a par-

ticular category by speciﬁc operations and, consequently, based on a non-

Boolean algebraic structure of some type. A fuzziﬁed uncertainty theory is

then obtained by formalizing a monotone measure of some type in terms of

this algebraic structure, as illustrated in Figure 7.1.

Standard fuzzy sets have been predominant in the literature and, moreover,

virtually all fuzziﬁcations of the various uncertainty theories that are currently

described in the literature are based on standard fuzzy sets. In this chapter it

is thus natural to examine standard fuzzy sets in more detail than other types

of fuzzy sets. However, the amount of research work on the various nonstan-

dard categories of fuzzy sets has visibly increased during the last few years,

primarily in response to emerging applications needs. Since the aim of gener-

alized information theory (GIT) is to expand the development of uncertainty

theories in both the dimensions depicted in Figure 1.3, it is essential to survey

in this chapter those nonstandard types of fuzzy sets that have been proposed

in the literature. However, this survey, which is the subject of Section 7.8, is

only relevant to future research in GIT. It is not needed for understanding the

7.1. AN OVERVIEW 261