Klir G.J. Uncertainity and Information. Foundations of Generalized Information Theory

Подождите немного. Документ загружается.

7.3.1. Complementation Operations / 266

7.3.2. Intersection and Union Operations / 267

7.3.3. Combinations of Basic Operations / 268

7.3.4. Other Operations / 269

7.4. Fuzzy Numbers and Intervals / 270

7.4.1. Standard Fuzzy Arithmetic / 273

7.4.2. Constrained Fuzzy Arithmetic / 274

7.5. Fuzzy Relations / 280

7.5.1. Projections and Cylindric Extensions / 281

7.5.2. Compositions, Joins, and Inverses / 284

7.6. Fuzzy Logic / 286

7.6.1. Fuzzy Propositions / 287

7.6.2. Approximate Reasoning / 293

7.7. Fuzzy Systems / 294

7.7.1. Granulation / 295

7.7.2. Types of Fuzzy Systems / 297

7.7.3. Defuzziﬁcation / 298

7.8. Nonstandard Fuzzy Sets / 299

7.9. Constructing Fuzzy Sets and Operations / 303

Notes / 305

Exercises / 308

8 Fuzziﬁcation of Uncertainty Theories 315

8.1. Aspects of Fuzziﬁcation / 315

8.2. Measures of Fuzziness / 321

8.3. Fuzzy-Set Interpretation of Possibility Theory / 326

8.4. Probabilities of Fuzzy Events / 334

8.5. Fuzziﬁcation of Reachable Interval-Valued Probability

Distributions / 338

8.6. Other Fuzziﬁcation Efforts / 348

Notes / 350

Exercises / 351

9 Methodological Issues 355

9.1. An Overview / 355

9.2. Principle of Minimum Uncertainty / 357

9.2.1. Simpliﬁcation Problems / 358

9.2.2. Conﬂict-Resolution Problems / 364

9.3. Principle of Maximum Uncertainty / 369

9.3.1. Principle of Maximum Entropy / 369

x CONTENTS

9.3.2. Principle of Maximum Nonspeciﬁcity / 373

9.3.3. Principle of Maximum Uncertainty in GIT / 375

9.4. Principle of Requisite Generalization / 383

9.5. Principle of Uncertainty Invariance / 387

9.5.1. Computationally Simple Approximations / 388

9.5.2. Probability–Possibility Transformations / 390

9.5.3. Approximations of Belief Functions by Necessity

Functions / 399

9.5.4. Transformations Between

-Measures and Possibility

Measures / 402

9.5.5. Approximations of Graded Possibilities by Crisp

Possibilities / 403

Notes / 408

Exercises / 411

10 Conclusions 415

10.1. Summary and Assessment of Results in Generalized

Information Theory / 415

10.2. Main Issues of Current Interest / 417

10.3. Long-Term Research Areas / 418

10.4. Signiﬁcance of GIT / 419

Notes / 421

Appendix A Uniqueness of the U-Uncertainty 425

Appendix B Uniqueness of Generalized Hartley Measure

in the Dempster–Shafer Theory 430

Appendix C Correctness of Algorithm 6.1 437

Appendix D Proper Range of Generalized

Shannon Entropy 442

Appendix E Maximum of GS

in Section 6.9 447

Appendix F Glossary of Key Concepts 449

Appendix G Glossary of Symbols 455

Bibliography 458

Subject Index 487

Name Index 494

CONTENTS xi

PREFACE

The concepts of uncertainty and information studied in this book are tightly

interconnected. Uncertainty is viewed as a manifestation of some information

deﬁciency, while information is viewed as the capacity to reduce uncertainty.

Whenever these restricted notions of uncertainty and information may be con-

fused with their other connotations, it is useful to refer to them as informa-

tion-based uncertainty and uncertainty-based information, respectively.

The restricted notion of uncertainty-based information does not cover the

full scope of the concept of information. For example, it does not fully capture

our common-sense conception of information in human communication and

cognition or the algorithmic conception of information. However, it does play

an important role in dealing with the various problems associated with

systems, as I already recognized in the late 1970s. It is this role of uncertainty-

based information that motivated me to study it.

One of the insights emerging from systems science is the recognition that

scientiﬁc knowledge is organized, by and large, in terms of systems of various

types. In general, systems are viewed as relations among states of some vari-

ables. In each system, the relation is utilized, in a given purposeful way, for

determining unknown states of some variables on the basis of known states of

other variables. Systems may be constructed for various purposes, such as pre-

diction, retrodiction, diagnosis, prescription, planning, and control. Unless the

predictions, retrodictions, diagnoses, and so forth made by the system are

unique, which is a rather rare case, we need to deal with predictive uncertainty,

retrodictive uncertainty, diagnostic uncertainty, and the like. This respective

uncertainty must be properly incorporated into the mathematical formaliza-

tion of the system.

In the early 1990s, I introduced a research program under the name “gen-

eralized information theory” (GIT), whose objective is to study information-

based uncertainty and uncertainty-based information in all their

manifestations. This research program, motivated primarily by some funda-

mental issues emerging from the study of complex systems, was intended to

expand classical information theory based on probability. As is well known,

the latter emerged in 1948, when Claude Shannon established his measure of

probabilistic uncertainty and information.

xiii

xiv PREFACE

GIT expands classical information theory in two dimensions. In one dimen-

sion, additive probability measures, which are inherent in classical information

theory, are expanded to various types of nonadditive measures. In the other

dimension, the formalized language of classical set theory, within which prob-

ability measures are formalized, is expanded to more expressive formalized

languages that are based on fuzzy sets of various types. As in classical infor-

mation theory, uncertainty is the primary concept in GIT, and information is

deﬁned in terms of uncertainty reduction.

Each uncertainty theory that is recognizable within the expanded frame-

work is characterized by: (a) a particular formalized language (classical or

fuzzy); and (b) a generalized measure of some particular type (additive or non-

additive). The number of possible uncertainty theories that are subsumed

under the research program of GIT is thus equal to the product of the number

of recognized formalized languages and the number of recognized types of

generalized measures. This number has been growing quite rapidly. The full

development of any of these uncertainty theories requires that issues at each

of the following four levels be adequately addressed: (1) the theory must be

formalized in terms of appropriate axioms; (2) a calculus of the theory must

be developed by which this type of uncertainty can be properly manipulated;

(3) a justiﬁable way of measuring the amount of uncertainty (predictive, diag-

nostic, etc.) in any situation formalizable in the theory must be found; and (4)

various methodological aspects of the theory must be developed.

GIT, as an ongoing research program, offers us a steadily growing inven-

tory of distinct uncertainty theories, some of which are covered in this book.

Two complementary features of these theories are signiﬁcant. One is their

great and steadily growing diversity. The other is their unity, which is mani-

fested by properties that are invariant across the whole spectrum of uncer-

tainty theories or, at least, within some broad classes of these theories. The

growing diversity of uncertainty theories makes it increasingly more realistic

to ﬁnd a theory whose assumptions are in harmony with each given applica-

tion. Their unity allows us to work with all available theories as a whole, and

to move from one theory to another as needed.

The principal aim of this book is to provide the reader with a comprehen-

sive and in-depth overview of the two-dimensional framework by which the

research in GIT has been guided,and to present the main results that have been

obtained by this research. Also covered are the main features of two classical

information theories.One of them,covered in Chapter 3,is based on the concept

of probability.This classical theory is well known and is extensively covered in

the literature. The other one, covered in Chapter 2, is based on the dual

concepts of possibility and necessity. This classical theory is older and more

fundamental, but it is considerably less visible and has often been incorrectly

dismissed in the literature as a special case of the probability-based infor-

mation theory. These two classical information theories, which are for-

mally incomparable, are the roots from which distinct generalizations are

obtained.

PREFACE xv

Principal results regarding generalized uncertainty theories that are based

on classical set theory are covered in Chapters 4–6.While the focus in Chapter

4 is on the common properties of uncertainty representation in all these the-

ories, Chapter 5 is concerned with special properties of individual uncertainty

theories.The issue of how to measure the amount of uncertainty (and the asso-

ciated information) in situations formalized in the various uncertainty theo-

ries is thoroughly investigated in Chapter 6. Chapter 7 presents a concise

introduction to the fundamentals of fuzzy set theory, and the fuzziﬁcation of

uncertainty theories is discussed in Chapter 8, in both general and speciﬁc

terms. Methodological issues associated with GIT are discussed in Chapter 9.

Finally, results and open problems emerging from GIT are summarized and

assessed in Chapter 10.

The book can be used in several ways and, due to the universal applicabil-

ity of GIT, it is relevant to professionals in virtually any area of human affairs.

While it is written primarily as a textbook for a one-semester graduate course,

its utility extends beyond the classroom environment. Due to the compre-

hensive and coherent presentation of the subject and coverage of some pre-

viously unpublished results, the book is also a useful resource for researchers.

Although the treatment of uncertainty and information in the book is math-

ematical, the required mathematical background is rather modest: the reader

is only required to be familiar with the fundamentals of classical set theory,

probability theory and the calculus. Otherwise, the book is completely self-

contained, and it is thus suitable for self-study.

While working on the book, clarity of presentation was always on my mind.

To achieve it, I use examples and visual illustrations copiously. Each chapter

is also accompanied by an adequate number of exercises, which allow readers

to test their understanding of the studied material.The main text is only rarely

interrupted by bibliographical, historical, or any other references. Almost all

references are covered in speciﬁc Notes, organized by individual topics and

located at the end of each chapter.These notes contain ample information for

further study.

For many years, I have been pursuing research on GIT while, at the same

time, teaching an advanced graduate course in this area to systems science stu-

dents at Binghamton University in New York State (SUNY-Binghamton). Due

to rapid developments in GIT, I have had to change the content of the course

each year to cover the emerging new results. This book is based, at least to

some degree, on the class notes that have evolved for this course over the

years. Some parts of the book, especially in Chapters 6 and 9, are based on my

own research.

It is my hope that this book will establish a better understanding of the very

complex concepts of information-based uncertainty and uncertainty-based

information, and that it will stimulate further research and education in the

important and rapidly growing area of generalized information theory.

Binghamton, New York George J. Klir

December 2004

ACKNOWLEDGMENTS

Over more than three decades of my association with Binghamton University,

I have had the good fortune to advise and work with many outstanding doc-

toral students. Some of them contributed in a signiﬁcant way to generalized

information theory, especially to the various issues regarding uncertainty mea-

sures. These students, whose individual contributions to generalized informa-

tion theory are mentioned in the various notes in this book, are (in

alphabetical order): David Harmanec, Masahiko Higashi, Cliff Joslyn,

Matthew Mariano, Yin Pan, Michael Pittarelli, Arthur Ramer, Luis Rocha,

Richard Smith, Mark Wierman, and Bo Yuan. A more recent doctoral student,

Ronald Pryor, read carefully the initial version of the manuscript of this book

and suggested many improvements. In addition, he developed several com-

puter programs that helped me work through some intricate examples in the

book. I gratefully acknowledge all this help.

As far as the manuscript preparation is concerned, I am grateful to two

persons for their invaluable help. First, and foremost, I am grateful to Monika

Fridrich, my Editorial Assistant and a close friend, for her excellent typing of

a very complex, mathematically oriented manuscript, as well as for drawing

many ﬁgures that appear in the book. Second, I am grateful to Stanley Kauff-

man, a graphic artist at Binghamton University, for drawing ﬁgures that

required special skills.

Last, but not least, I am grateful to my wife, Milena, for her contribution to

the appearance of this book: it is one of her photographs that the publisher

chose to facilitate the design for the front cover. In addition, I am also

grateful for her understanding, patience, and encouragement during my

concentrated, disciplined and, at times, frustrating work on this challenging

book.

xvii

INTRODUCTION

The mind, once expanded to the dimensions of larger ideas, never returns to its

original size.

—Oliver Wendel Holmes

1.1. UNCERTAINTY AND ITS SIGNIFICANCE

It is easy to recognize that uncertainty plays an important role in human

affairs. For example, making everyday decisions in ordinary life is insepara-

ble from uncertainty, as expressed with great clarity by George Shackle

[1961]:

In a predestinate world, decision would be illusory; in a world of a perfect fore-

knowledge, empty, in a world without natural order, powerless. Our intuitive atti-

tude to life implies non-illusory, non-empty, non-powerless decision....Since

decision in this sense excludes both perfect foresight and anarchy in nature, it

must be deﬁned as choice in face of bounded uncertainty.

Conscious decision making, in all its varieties, is perhaps the most fundamen-

tal capability of human beings. It is essential for our survival and well-being.

In order to understand this capability, we need to understand the notion of

uncertainty ﬁrst.

In decision making, we are uncertain about the future. We choose a partic-

ular action, from among a set of conceived actions, on the basis of our antici-

Uncertainty and Information: Foundations of Generalized Information Theory, by George J. Klir