Korb K.B., Nicholson A.E. Bayesian Artificial Intelligence

Подождите немного. Документ загружается.

elimination order using a combination of minimum-weight search and stochas-

tic search. Both the junction tree and the elimination order may be viewed.

Netica also reports both the probability of the most recent evidence, and the

probability of all evidence currently entered, and provides the MPE and its

probability (but not for networks containing decision nodes). Netica can gen-

erate random samples by junction tree or logic sampling.

Evidence: Netica supports speciﬁc (which they call “positive”), negative and like-

lihood evidence. Multiple likelihood evidence may be incorporated for the

same nodes. Netica also handles sets of evidence (cases) by case ﬁles and

direct database access.

Decision networks: Netica infers a temporal order for decision network, if it can.

DN evaluation gives the expected utilities for a one-off decision, but only the

decision table for sequential decision making.

DBNs: Netica supports DBN speciﬁcation and roll-out.

Learning: Netica supports parameter learning only. It uses the Spiegelhalter &

Lauritzen parameterization algorithm, allows missing values, and allows the

speciﬁcation of a weighting to the original probabilities, providing a form of

adaptation. Netica can also do EM learning and gradient descent learning, to

handle large amounts of missing data, or latent (unobserved) variables. It also

supports fading, with the user able to specify a factor from 0 (no new learning)

to 1 (removes all previous learning).

Evaluation: Netica supports sensitivity to ﬁndings, as described earlier in

10.3.1.

It also provides a number of measures for statistical validation including a

count form of predictive accuracy, a confusion matrix, the error rate, scoring

rule results, logarithmic loss and quadratic loss, spherical pay off, calibration

results and a “times surprised” table (indicating when the network was conﬁ-

dent of its beliefs but was wrong).

B.5 Bayesian statistical modeling

B.5.1 BUGS

MRC/Imperial College

http://www.mrc-bsu.cam.ac.uk/bugs

BUGS (Bayesian inference Using Gibbs Sampling) provides Bayesian analysis of

complex statistical models using Markov Chain Monte Carlo (MCMC). It’s most ad-

vanced version is WinBUGS, which is available in a free educational version. Other

versions are available for unix systems (Sun, SGI and Linux). The Web pages pro-

vide thorough documentation for the different versions of the program. A BUGS

email discussion list is also maintained.

The Markov Chain Monte Carlo (MCMC) methods available include univariate

Gibbs sampling (BUGS 0.6) and a more sophisticated univariate Metropolis sampler

(WinBUGS). Applications include generalized linear mixed models, latent variable

models, modeling measurement error, handling missing data and Bayesian updating.

BUGS provides a language for specifying a Bayesian network. A compiler then

processes the model and data and sets up the sampling distributions required for

the Gibbs sampling. Finally, appropriate sampling algorithms are implemented to

simulate values of the unknown quantities in the model.

B.5.2 First Bayes

Dept. of Probability and Statistics

University of Shefﬁeld

http://www.shef.ac.uk/˜st1ao/1b.html

First Bayes is program to assist people in learning elementary Bayesian Statis-

tics. It is meant to supplement other educational material and is not a self-contained

tutoring system. It runs under MS-Windows. First Bayes is offered free to any-

one interested in teaching or learning Bayesian Statistics, provided it is not used for

proﬁt.

B.6 Causal discovery programs

B.6.1 Bayesware Discoverer

Bayesware Ltd.

http://www.bayesware.com

Bayesware Discoverer is a new commercial causal discovery tool running on the

MS Windows platforms. The program uses the Cooper and Herskovits formula of

8.2 to construct a Bayesian metric, adjusted to deal with incomplete data by em-

ploying probability intervals. It has a “Wizard-like” interface to assist.

This software is a commercial version of the Bayesian Knowledge Discoverer,

developed at the Knowledge Media Institute of The Open University (UK), based on

the work of Ramoni and Sebastiani (e.g., [230]). A limited educational version is

available free.

B.6.2 CaMML

Chris Wallace, Kevin Korb

School of Computer Science, Monash University, Victoria 3800 Australia

http://www.datamining.monash.edu.au/software/

camml/index.shtml

CaMML (Causal discovery via MML) was described in Chapter 8 in some detail.

It is freely available as an executable download for Linux from the above Web site.

There are two different versions: CaMML-L, which learns linear Gaussian models,

and CaMML, which learns discrete causal models. Both use the Metropolis sampling

search algorithm. These versions have a fairly crude ASCII command-line interface.

There is a project to reimplement CaMML inside the Monash CDMS (Core Data

Mining Software) project, providing CaMML with a GUI interface and data visual-

ization capabilities. When ready, this will be available from the above Web site, as

will any future developments from the CaMML project.

B.6.3 TETRAD

Peter Spirtes, Clark Glymour and Richard Scheines

Dept. of Philosophy, Carnegie Mellon University

http://www.phil.cmu.edu/tetrad/

TETRAD II was the ﬁrst commercially available causal discovery program. Its PC

algorithm is described in Chapter 6. It is no longer available for purchase. TETRAD

III adds Gibbs sampling to the functionality of TETRAD II, but retains the command-

line interface. TETRAD IV has a graphical user interface, running under the Java

Runtime Environment. TETRAD IV currently supports four causal discovery algo-

rithms:

The PC Algorithm

A genetic algorithm search (for linear dag models)

The CCD Algorithm (linear cyclic models)

The FCI Algorithm (dag models with latent variables)

TETRAD IV supports maximum likelihood parameter estimation for both discrete

and linear models. It also has support for building and simulating time series models.

TETRAD III and TETRAD IV are available for free download.

B.6.4 WinMine

David Max Chickering

Microsoft Research, Redmond, Washington 98052

http://research.microsoft.com/˜dmax/WinMine/tooldoc.htm

WinMine is a set of causal discovery programs for Windows 2000/NT/XP [47].

The majority of the programs are command-line executables that can be run in

scripts. It includes GUIs for viewing learned or modeled Bayesian networks and

for displaying classiﬁcation trees (“decision trees”). It is freely downloadable, if it is

not going to be used for commercial purposes.

WinMine can learn Bayesian networks from sample data. It supports prior infor-

mation in the form of partial variable orderings and forbidden arcs. There is also

support for evaluating the learned models.

References

[1] Abdelbar, A. M., and Hedetniemi, S. M. The complexity of approximating

MAP explanation. Artiﬁcial Intelligence 102 (1998), 21–38.

[2] Abramson, B., Brown, J., Edwards, W., Murphy, A., and Winkler, B. Hail-

ﬁnder. http://www.lis.pitt.edu/ dsl/hailﬁnder/.

[3] Abramson, B., and Finizza, A. Using belief networks to forecast oil prices.

International Journal of Forecasting 7, 3 (November 1991), 299–315.

[4] Albrecht, D. W., Zukerman, I., and Nicholson, A. E. Bayesian models for

keyhole plan recognition in an adventure game. User Modeling and User-

Adapted Interaction 8, 1-2 (1998), 5–47.

[5] Andersen, L. R., Krebs, J. H., and Andersen, J. D. Steno: an expert system

for medical diagnosis. Journal of Applied Statistics 18, 1 (1989), 139–153.

[6] Andersen, S., Olesen, K., Jensen, F., and Jensen, F. HUGIN — a shell for

building Bayesian belief universes for expert systems. In Proceedings of

the Eleventh International Joint Conference on Artiﬁcial Intelligence (IJCAI)

(Detroit, MI, 1989).

[7] Anderson, J. The Architecture of Cognition. Harvard University Press, Cam-

bridge, MA, 1983.

[8] Andreassen, S., Benn, J., Hovorks, R., Olesen, K., and Carson, R. A proba-

bilistic approach to glucose prediction and insulin dose adjustment. Computer

Methods and Programs in Biomedicine 41 (1994), 153–165.

[9] Andreassen, S., Jensen, F. V., Andersen, S. K., Falck, B., Kjærulff, U., Wold-

bye, M., Sørensen, A. R., Rosenfalck, A., and Jensen, F. MUNIN — an expert

EMG assistant. In Computer-Aided Electromyography and Expert Systems,

J. Desmedt, Ed. Elsevier, Amsterdam, 1989.

[10] Aristotle. The Nicomachean Ethics.

[11] Asher, H. B. Causal Modeling, second ed. Sage, Beverly Hills, CA, 1983.

[12] Baars, B. A Cognitive Theory of Consciousness. Cambridge University Press,

London, 1988.

[13] Bacon, P. J., Cain, J. D., Kozakiewicz, M., Brzezinski, M., and Liro, A. Pro-

moting more sustainable rural land use and development: a case study in

Eastern Europe using Bayesian network models. Journal of Environmental

Assessment Policy and Management 4, 2 (June 2002), 199–240.

[14] Balakrishnan, N., and Nevzorov, V. A Primer on Statistical Distributions.

Wiley, New York, 2003.

[15] Batt, K. Sea breezes on the NSW coast. Offshore Yachting Oct/Nov (1995).

[16] Bayes, T. An essay towards solving a problem in the doctrine of chances.

Biometrika 45 (1764/1958), 296–315.

[17] Beinlich, I., Suermondt, H., Chavez, R., and Cooper, G. The ALARM moni-

toring system. In Proceedings of the Second European Conference on Artiﬁ-

cial Intelligence in Medicine (1992), pp. 689–693.

[18] Bell, D. Disappointment in decision making under uncertainty. In Deci-

sion making: Descriptive, Normative, and Prescriptive Interactions,H.R.

D.E. Bell and A. Tversky, Eds. Cambridge Press, London, 1988, pp. 358–

384.

[19] Billings, D. Computer poker. Master’s thesis, University of Alberta, 1995.

[20] Billings, D., Pena, L., Schaeffer, J., and Szafron, D. Using probabilistic

knowledge and simulation to play poker. In Proceedings of the Sixteenth

National AAAI Conference (1999).

[21] Blalock, H. Causal Inferences in Nonexperimental Research. University of

North Carolina Press, Chapel Hill, 1964.

[22] Boehm, B. W. A spiral model of software development and enhancement.

IEEE Computer (1988), 61–72.

[23] Boerlage, B. Link strengths in Bayesian Networks. Master’s thesis, Depart-

ment of Computer Science, University of British Columbia, 1995.

[24] Boneh, T. Support for graphical modelling in Bayesian network knowledge

engineering: a visual tool for domain experts. Master’s thesis, Department of

Computer Science, University of Melbourne, 2003.

[25] Borgelt, C., and Kruse, R., Eds. Graphical Models: Methods for Data Anal-

ysis and Mining. Wiley, New York, 2002.

[26] Borsuk, M., Stow, C., and Reckhow, K. Integrative environmental prediction

using Bayesian networks. In Proceedings of the First Biennial Meeting of

the International Environmental Modelling and Software Society (June 2002),

A. E. Rizzoli and A. J. Jakeman, Eds., vol. 1, iEMSs, pp. 102–107.

[27] Bouckeart, R. Probabilistic network construction using the minimum descrip-

tion length principle. Technical Report RUU-CS-94-27, Dept. of Computer

Science, Utrecht University, 1994.

[28] Boudon, R. A new look at correlation analysis. In Methodology in Social

Research, H. Blalock and A. Blalock, Eds. McGraw-Hill, New York, 1968,

pp. 199–235.

[29] Boulton, D. Bayesian poker. Honours thesis, School of Computer Science

and Software Engineering, Monash Univ, 2003.

[30] Boutilier, C., Friedman, N., Goldszmidt, M., and Koller, D. Context-speciﬁc

independence in Bayesian Networks. In UAI96 – Proceedings of the Twelfth

Conference on Uncertainty in Artiﬁcial Intelligence (1996), Horvitz and

Jensen, Eds., pp. 115–123.

[31] Boyen, X., and Koller, D. Tractable inference for complex stochastic pro-

cesses. In UAI98 – Proceedings of the Fourteenth Conference on Uncertainty

in Artiﬁcial Intelligence (1998), pp. 33–42.

[32] Brightwell, G., and Winkler, P. Counting linear extensions is #p-complete.

Technical Report DIMACS 90-49, Dept. of Computer Science, Rutgers, 1990.

[33] Brooks, F. The Mythical Man-Month: Essays on Software Engineering, sec-

ond ed. Addison-Wesley, Reading, MA, 1995.

[34] Buchanan, B., and Shortliffe, E., Eds. Rule-Based Expert Systems:

The MYCIN Experiments of the Stanford Heuristic Programming Project.

Addison-Wesley, Reading, MA, 1984.

[35] Buntine, W. Theory reﬁnement on Bayesian networks. In UAI91 – Pro-

ceedings of the Seventh Conference on Uncertainty in Artiﬁcial Intelligence

(1991), S. D’Ambrosio and Bonissone, Eds., pp. 52–69.

[36] Buntine, W. A guide to the literature on learning probabilistic networks from

data. IEEE Trans on Knowledge and Data Engineering 8, 2 (1996), 195–210.

[37] Burnside, B. E., Rubin, D. L., and Shachter, R. A Bayesian network for

mammography. Technical report, Stanford Medical Informatics Dept, 2000.

[38] Carlton, J. Bayesian poker. Honours thesis, School of Computer Science and

Software Engineering, Monash University, 2000.

[39] Carnap, R. The Logical Foundations of Probability, second ed. University of

Chicago, 1962.

[40] Chaitin, G. J. On the length of programs for computing ﬁnite binary se-

quences. Journal of the ACM 13 (1966), 547–569.

[41] Chan, H., and Darwiche, A. When do numbers really matter? Journal of

Artiﬁcial Intelligence Research 17 (2003), 265–287.

[42] Cheeseman, P. An inquiry into computer understanding. Computational In-

telligence 4 (1988), 58–66.

[43] Cheng, J., and Druzdzel, M. AIS-BN: An adaptive importance sampling algo-

rithm for evidential reasoning in large Bayesian networks. Journal of Artiﬁcial

Intelligence Research 13 (2000), 155–188.

[44] Cherniak, C. Minimal Rationality. MIT, Cambridge, MA, 1986.

[45] Chickering, D. M. A tranformational characterization of equivalent Bayesian

network structures. In UAI95 – Proceedings of the Eleventh Conference on

Uncertainty in Artiﬁcial Intelligence (San Francisco, 1995), P. Besnard and

S. Hanks, Eds., pp. 87–98.

[46] Chickering, D. M. Learning equivalence classes of Bayesian network struc-

tures. In UAI96 – Proceedings of the Twelfth Conference on Uncertainty in

Artiﬁcial Intelligence (San Mateo, CA, 1996), pp. 150–157.

[47] Chickering, D. M. The WinMine toolkit. Technical report MSR-TR-2002-

103, Microsoft, Redmond, WA, 2002.

[48] Chihara, C., and Kennedy, R. The Dutch book argument: Its logical ﬂaws, its

subjective sources. Philosophical Studies 36 (1979), 19–33.

[49] Conati, C., Gertner, A., and VanLehn, K. Using Bayesian Networks to man-

age uncertainty in student modeling. User Modeling and User-Adapted Inter-

action 12, 4 (2002), 371–417.

[50] Conati, C., Gertner, A., VanLehn, K., and Druzdzel, M. On-line student mod-

eling for coached problem solving using Bayesian Networks. In UM97 –

Proceedings of the Sixth International Conference on User Modeling (1997),

pp. 231–242.

[51] Cooper, G. NESTOR: A Computer Based Medical Diagnostic Aid that Inte-

grates Causal and Probabilistic Knowledge. PhD thesis, Medical Computer

Science Group, Stanford University, Stanford, CA, 1984.

[52] Cooper, G. F. The computational complexity of probabilistic inference using

Bayesian belief networks. Artiﬁcial Intelligence 42 (1990), 393–405.

[53] Cooper, G. F., and Herskovits, E. A Bayesian method for constructing Bayes-

ian belief networks from databases. In UAI91 – Proceedings of the Seventh

Conference on Uncertainty in Artiﬁcial Intelligence (1991), S. D’Ambrosio

and Bonissone, Eds., pp. 86–94.

[54] Cooper, G. F., and Herskovits, E. A Bayesian method for the induction of

probabilistic networks from data. Machine Learning 9 (1992), 309–347.

[55] Coupe, V. Sensitivity Analysis of Decision-Theoretic Networks. PhD thesis,

Institute of Information and Computing Sciences, Utrecht University, 2000.

[56] Coupe, V., Peek, N., Ottenkamp, J., and Habbema, J. Using sensitivity anal-

ysis for efﬁcient quantiﬁcation of a belief network. Artiﬁcial Intelligence in

Medicine 17 (1999), 223–247.

[57] Coup´e, V., van der Gaag, L., and Habbema, J. Sensitivity analysis: an aid for

belief-network quantiﬁcation. Knowledge Engineering Review 15.

[58] Cousins, S., Chen, W., and Frisse, M. CABeN: A collection of algorithms for

belief networks. Technical Report WUCS-91-25, Department of Computer

Science, Washington University, 1991.

[59] Covelo, V., von D. Winterfeldt, and Slovic, P. Risk communication: A review

of the literature. Risk Abstracts 3, 4 (1986), 171–182.

[60] Cover, T., and Thomas, J. Elements of Information Theory. Wiley, New York,

1991.

[61] Cowell, R. G., Dawid, A. P., Lauritzen, S. L., and Spiegelhalter, D. Proba-

bilistic Networks and Expert Systems. Statistics for Engineering and Informa-

tion Science. Springer, New York, 1999.

[62] Cozman, F. G. Credal networks. Artiﬁcial Intelligence Journal 120 (2000),

199–233.

[63] Dagum, P., Galper, A., and Horvitz, E. Dynamic network models for fore-

casting. In UAI92 – Proceedings of the Eighth Conference on Uncertainty in

Artiﬁcial Intelligence (1992), pp. 41–48.

[64] Dagum, P., and Luby, M. Approximating probabilistic inference in belief

networks is NP-hard. Artiﬁcial Intelligence (1993), 141–153.

[65] Dai, H., Korb, K. B., Wallace, C. S., and Wu, X. A study of casual dis-

covery with weak links and small samples. In IJCAI97 – Proceedings of

the Fifteenth International Joint Conference on Artiﬁcial Intelligence (1997),

Morgan Kaufmann, pp. 1304–1309.

[66] Davies, J. Mobile monitoring of lower-limb amputees and fall diagnosis in el-

derly patients. Thesis report. Department of Electrical and Computer Systems

Engineering, Monash University, 1995.

[67] de Finetti, B. Foresight: Its logical laws, its subjective sources. In Studies

in Subjective Probability, Kyburg and Smokler, Eds. Wiley, New York, 1964.

(Translation of ”La pr´evision” Annales de l’Institut Henri Poincar´e, 1937).

[68] de Laplace, P. S. A Philosophical Essay on Probabilities. Dover, 1820/1951.

[69] Dean, T., and Kanazawa, K. A model for reasoning about persistence and

causation. Computational Intelligence 5 (1989), 142–150.

[70] Dean, T., and Wellman, M. P. Planning and Control. Morgan Kaufmann

Publishers, San Mateo, CA, 1991.

[71] DeGroot, M. Optimal Statistical Decisions. McGraw-Hill, New York, 1970.

[72] DeGroot, M., and Schervish, M. Probability and Statistics, third ed. Addison-

Wesley, Reading, MA, 2002.

[73] Demographia. Demographic statistics. http://www.demographia.com.

[74] Dempster, A., Laird, N., and Rubin, D. Maximum likelihood from incomplete

data via the EM algorithm. Journal of the Royals Statistical Society, B 39

(1977), 1–38.

[75] Dietterich, T. Approximate statistical tests for comparing supervised classiﬁ-

cation learning algorithms. Neural Computation 7 (1998), 1895–1924.

[76] Draper, D. Localized Partial Evaluation of Belief Networks. PhD thesis,

Department of Computer Science, University of Washington, 1995.

[77] Druzdzel, M. Qualitative verbal explanations in Bayesian belief networks.

Artiﬁcial Intelligence and Simulation of Behaviour Quarterly 94 (1996), 43–

54.

[78] Duda, R. O., Hart, P. E., and Nilsson, N. J. Subjective Bayesian methods for

rule-based inference systems. In AFIPS (1976), vol. 45, pp. 1075–1082.

[79] Edwards, A. L. An Introduction to Linear Regression and Correlation, sec-

ond ed. W.H. Freeman, New York, 1984.

[80] Ellison, A. An introduction to Bayesian inference for ecological research and

environmental decision-making. Ecological Applications 6 (1996), 1036–

1046.

[81] Feigenbaum, E. The art of artiﬁcial intelligence: Themes and case studies of

knowledge engineering. In Fifth International Conference on Artiﬁcial Intel-

ligence – IJCAI-77 (San Mateo, CA, 1977), Morgan Kaufmann, pp. 1014–

1029.

[82] Findler, N. Studies in machine cognition using the game of poker. Communi-

cations of the ACM 20 (1977), 230–245.

[83] Fischoff, B., and Bar-Hillel, M. Focusing techniques: A shortcut to improving

probability judgments? Organizational Behavior and Human Performance 34

(1984), 339–359.

[84] Flynn, J. The effectiveness of a computer games program in teaching decimal

notation. Honours thesis, University of Melbourne, 2002.

[85] Forbes, J., Huang, T., Kanazawa, K., and Russell, S. The BATmobile: To-

wards a Bayesian automated taxi. In Proceedings of the Fourteenth In-

ternational Joint Conference on Artiﬁcial Intelligence (IJCAI’95) (1995),

pp. 1878–1885.

[86] Friedman, N., and Goldszmidt, M. Learning Bayesian networks with lo-

cal structure. In Learning in Graphical Models, M. Jordan, Ed. MIT, 1998,

pp. 421–459.

[87] Fung, R., and Chang, K.-C. Weighting and integrating evidence for stochastic

simulation in Bayesian networks. In Proceedings of the Fifth Conference on

Uncertainty in Artiﬁcial Intelligence (UAI-89) (1989), pp. 475–482.

[88] Fung, R., and del Favero, B. Backward simulation in Bayesian networks.

In UAI94 – Proceedings of the Tenth Conference on Uncertainty in Artiﬁ-

cial Intelligence (San Francisco, 1994), L. de Mantras and D. Poole, Eds.,

pp. 227–234.

[89] Garey, M., and Johnson, D. Computers and Intractability. W.H. Freeman,

New York, 1979.

[90] Geiger, D., and Heckerman, D. Advance in probabilistic reasoning. In Uncer-

tainty in Artiﬁcial Intelligence Vol 6 (1991), S. D’Ambrosio and Bonissone,

Eds., pp. 118–126.

[91] Geman, S., and Geman, D. Stochastic relaxation, Gibbs distributions and the

Bayesian restoration of images. IEEE Transactions on Pattern Analysis and

Machine Intelligence 6, 721–742.

[92] Gigerenzer, G., and Hoffrage, U. How to improve Bayesian reasoning without

instruction: Frequency formats. Psychological Review 102 (1995), 684–704.

[93] Ginac, F. Customer Oriented Software Quality Assurance. Prentice Hall, New

York, 1997.

[94] Glymour, C. Theory and Evidence. Princeton University, Princeton, NJ, 1980.

[95] Good, I. Rational decisions. Journal of the Royal Statistical Society, B 14

(1952), 107–114.

[96] Goodman, N. Fact, Fiction and Forecast, Third ed. Bobbs-Merrill, Indianapo-

lis, 1973.

[97] Haddawy, P., Jacobson, J., and Kahn, C. E. J. BANTER: a Bayesian network

tutoring shell. Artiﬁcial Intelligence in Medicine 10 (1997), 177–200.

[98] Hagen, R., and Sonenberg, E. Automated classiﬁcation of student miscon-

ceptions in physics. In Proceedings of the 1993 Australian Joint Conference

on Artiﬁcial Intelligence, AI’93 (1993), C. Rowles, H. Liu, and N. Foo, Eds.,

World Scientiﬁc, pp. 153–159.

[99] H´ajek, A. Scotching Dutch books. Talk at Monash University, July 2002.

[100] Harnad, S. Minds, machines and Searle. Journal of Theoretical and Experi-

mental Artiﬁcial Intelligence 1 (1989), 5–25.

[101] Heckerman, D. A tutorial on learning with Bayesian networks. In Learning

in Graphical Models. MIT, Cambridge, MA, pp. 301–354.

[102] Heckerman, D. Probabilistic Similarity Networks. MIT, Cambridge, MA,

1991.

[103] Heckerman, D., and Geiger, D. Learning Bayesian networks: A uniﬁcation

for discrete and Gaussian domains. In UAI95 – Proceedings of the Eleventh

Conference on Uncertainty in Artiﬁcial Intelligence (San Francisco, 1995),

P. Besnard and S. Hanks, Eds., pp. 274–284.

[104] Heckerman, D. E. Probabilistic interpretations for MYCIN’s certainty factors.

In Uncertainty in Artiﬁcial Intelligence Vol 1,J.LemmerandL.Kanal,Eds.

Elsevier, Amsterdam, 1986, pp. 167–196.