Nof S.Y. Springer Handbook of Automation

Подождите немного. Документ загружается.

Artiﬁcial Intelligence and Automation 14.1 Methods and Application Examples 265

debugging) or compiling their code (which provides

much higher speed).

See [14.74] for a discussion of other advantages

of Lisp. One notable disadvantage of Lisp is that, if

one has a computer program written in a conventional

language such as C, C++ or Java, it is difﬁcult for

such a program to call a Lisp program as a subroutine:

one must run the Lisp program as a separate process

in order to provide the Lisp execution environment.

(On the other hand, Lisp programs can quite easily in-

voke subroutines written in conventional programming

languages.)

Applications. Lisp was quite popular during the expert-

systems boom of the mid-1980s, and several Lisp

machine computer architectures were developed and

marketed in which the entire operatingsystem was writ-

ten in Lisp. Ultimately thesemachines didnot meet with

long-term commercial success, as they were eventually

surpassed by less-expensive, less-specialized hardware

such as Sun workstations and Intel x86 machines.

On the other hand, development of software sys-

tems in Lisp has continued, and there are many

current examples of Lisp applications. A few of them

include the visual lisp extension language for the Au-

toCAD computer-aided design system (autodesk.com),

the Elisp extension language for the Emacs editor

(http://en.wikipedia.org/wiki/Emacs_Lisp)theScript-Fu

plugins for the GNU Image Manipulation Program

(GIMP), the Remote Agent software deployed on

NASA’s Deep Space 1 spacecraft [14.75], the air-

line fare shopping engine used by Orbitz [14.9], the

SHOP2 planning system [14.38], and the Yahoo Store

e-commerce software. (As of 2003, about 20000 Ya-

hoo stores used this software. The author does not have

access to more recent statistics.)

Prolog

Prolog [14.76] is based on the notion that a general

theorem-prover can be used as a programming environ-

ment in which the program consists of a set of logical

statements. As an example, here is a Prolog program

for concatenating lists, analogous to the Lisp program

given earlier

concatenate([],Y,Y).

concatenate([First|Rest],Y,[First|Z]) :-

concatenate(Rest,Y,Z).

To concatenate two lists [a,b] and [c], one asks the

theorem prover if there exists a list Z that is their

concatenation; and the theorem prover returns Z if it

exists

?- concatenate([a,b],[c],Z)

Z=[a,b,c].

Alternatively, if one asks whether there are lists X and

Y whose concatenation is a given list Z, then there are

several possible values for X and Y, and the theorem

prover will return all of them

?- concatenate(X,Y,[a,b])

X = []; Y = [a,b]

X = [a]; Y = [b]

X = [a,b]; Y = []

One of Prolog’s biggest drawbacks is that several as-

pects of its programming style – for example, the

lack of an assignment statement, and the automated

backtracking – can require workarounds that feel un-

intuitive to most programmers. However, Prolog can

be good for problems in which logic is intimately

involved, or whose solutions have a succinct logical

characterization.

Applications. Prolog became popular during the the

expert-systems boom of the 1980s, and was used as the

basis for the Japanese Fifth Generation project [14.77],

but never achieved wide commercial acceptance. On

the other hand, an extension of Prolog called constraint

logic programming is important in several industrial ap-

plications (see Constraint Satisfaction and Constraint

Optimization).

C, C++, and Java

C and C++ provide much less in the way of high-level

programming constructs than Lisp, hence developing

code in these languages can require much more effort.

On the other hand, they are widely available and pro-

vide fast execution, hence they are useful for programs

that are simple and need to be both portable and fast; for

example, neural networks need very fast execution in

order to achieve a reasonable learning rate, and a back-

propagation procedure can be writtenin justa few pages

of C or C++ code.

Java is a lower-level language than Lisp, but is

higher-level than C or C++. It uses several ideas from

Lisp, most notably garbage collection. As of 2003 it

ran much more slowly than Lisp, but its speed has

improved in the interim and it has the advantages of

being highly portable and more widely known than

Lisp.

Part B 14.1

266 Part B Automation Theory and Scientiﬁc Foundations

14.2 Emerging Trends and Open Challenges

AI has gone through several periods of optimism and

pessimism. The most recent period of pessimism was

the AI winter mentioned in Sect.14.1.7. AI has emerged

from this period in recent years, primarily because of

the following trends. First is the exponential growth im-

provement in computing power: computations that used

to take days or weeks can now be done in minutes or

seconds. Consequently, computers have become much

better able to support the intensive computations that

AI often requires. Second, the pervasive role of comput-

ers in everyday life is helping to erase the apprehension

that has often been associated with AI in popular

culture. Third, there have been huge advances in AI

research itself. AI concepts such as search, planning,

natural-language processing, andmachine learninghave

developed mature theoretical underpinnings and exten-

sive practical histories.

AI technology is widely expected to become in-

creasingly pervasive in applications such as data min-

ing, information retrieval (especially from the web),and

prediction of human events (including anything from

sports forecasting to economics to international con-

ﬂicts).

During the next decade, it appears quite likely that

AI will be able to make contributions to the behav-

ioral and social sciences analogous to the contributions

that computer science has made to the biological sci-

ences during the past decade. To make this happen,

one of the biggest challenges is the huge diversity

among the various research ﬁelds that will be involved.

These include behavioral and social sciences such as

economics, political science, psychology, anthropol-

ogy, and sociology, and technical disciplines such as

AI, robotics, computational linguistics, game theory,

and operations research. Researchers from these ﬁelds

will need to forge a common understanding of princi-

ples, techniques, and objectives. Research laboratories

are being set up to foster this goal (one example is

the University of Maryland’s Laboratory for Computa-

tional Cultural Dynamics (http://www.umiacs.umd.edu/

research/LCCD/), which is co-directed by the author

of this chapter), and several international conferences

and workshops on the topic have been recently estab-

lished [14.78,79].

One of the biggest challenges that currently faces

AI research is its fragmentation into a bewilderingly di-

verse collection of subdisciplines. Unfortunately, these

subdisciplines are becoming rather insular, with their

own research conferences and their own (sometimes id-

iosyncratic) notions of what constitutes a worthwhile

research question or a signiﬁcant result. The achieve-

ment of human-level AI will require integrating the

best efforts among many different subﬁelds of AI,and

this in turn will require better communication amongst

the researchers from these subﬁelds. I believe that the

ﬁeld is capable of overcoming this challenge, and that

human-level AI will be possible by the middle of this

century.

References

14.1 A. Newell, H.A. Simon: Computer science as empir-

ical inquiry: Symbols and search, Assoc. Comput.

Mach. Commun. 19(3), 113–126 (1976)

14.2 S. Hedberg: Proc. Int. Conf. Artif. Intell. (IJCAI) 03

conference highlights, AI Mag. 24(4), 9–12 (2003)

14.3 C. Kuykendall: Analyzing solitaire, Science

283(5403), 791 (1999)

14.4 R. Bjarnason, P. Tadepalli, A. Fern: Searching soli-

taire in real time, Int. Comput. Games Assoc. J.

30(3), 131–142 (2007)

14.5 E. Horowitz, S. Sahni: Fundamentals of Computer

Algorithms (Computer Science, Potomac 1978)

14.6 N. Nilsson: Principles of Artiﬁcial Intelligence (Mor-

gan Kaufmann, San Francisco 1980)

14.7 D. Navinchandra: The recovery problem in product

design, J. Eng. Des. 5(1), 67–87 (1994)

14.8 S. Russell, P. Norvig: Artiﬁcial Intelligence, A

Modern Approach (Prentice Hall, Englewood Cliffs

1995)

14.9 S. Robinson: Computer scientists ﬁnd unexpected

depths in airfare search problem, SIAM News 35(1),

1–6 (2002)

14.10 C. Le Pape: Implementation of resource constraints

in ilog schedule: a library for the development of

constraint-based scheduling systems, Intell. Syst.

Eng. 3, 55–66 (1994)

14.11 P. Shaw: Using constraint programming and local

search methods to solve vehicle routing problems,

Proc. 4th Int. Conf. Princ. Pract. Constraint Pro-

gram. (1998) pp. 417–431

14.12 P. Albertand, L. Henocque, M. Kleiner: Conﬁgura-

tion based workﬂow composition, IEEE Int. Conf.

Web Serv., Vol. 1 (2005) pp. 285–292

14.13 J. Pearl: Heuristics (Addison-Wesley, Reading 1984)

14.14 R. Dechter: Constraint Processing (Morgan Kauf-

mann, San Francisco 2003)

14.15 J. Shoenﬁeld: Mathematical Logic (Addison-

Wesley, Reading 1967)

Part B 14

Artiﬁcial Intelligence and Automation References 267

14.16 J. Pearl, S. Russell: Bayesian networks. In: Hand-

book of Brain Theory and Neural Networks,ed.by

M.A. Arbib (MIT Press, Cambridge 2003) pp. 157–160

14.17 J. Pearl: Probabilistic Reasoning in Intelligent Sys-

tems: Networks of Plausible Inference (Morgan

Kaufmann, San Fransisco 1988)

14.18 M. Sahami, S. Dumais, D. Heckerman, E. Horvitz:

A bayesian approach to ﬁltering junk e-Mail. In:

Learning for Text Categorization: Papers from the

1998 Workshop AAAI Technical Report WS-98-05

(Madison 1998)

14.19 J.A. Zdziarski: Ending Spam: Bayesian Content

Filtering and the Art of Statistical Language Clas-

siﬁcation (No Starch, San Francisco 2005)

14.20 D. Quinlan: BayesInSpamAssassin, http://wiki.

apache.org/spamassassin/BayesInSpamAssassin

(2005)

14.21 K.M. Hanson: Introduction to bayesian image

analysis. In: Medical Imaging: Image Processing,

Vol. 1898, ed. by M.H. Loew, (Proc. SPIE, 1993)

pp. 716–732

14.22 L. Denoyer, P. Gallinari: Bayesian network model

for semistructured document classiﬁcation, Inf.

Process. Manage. 40,807–827(2004)

14.23 S. Fox, K. Karnawat, M. Mydland, S. Dumais,

T. White: Evaluating implicit measures to improve

web search, ACM Trans. Inf. Syst. 23(2), 147–168

(2005)

14.24 L. Zadeh, G.J. Klir, Bo Yuan: Fuzzy Sets, Fuzzy Logic,

and Fuzzy Systems: Selected Papers by Lotﬁ Zadeh

(World Scientiﬁc, River Edge 1996)

14.25 G.J.Klir,U.S.Clair,B.Yuan:Fuzzy Set Theory: Foun-

dations and Applications (Prentice Hall, Englewood

Cliffs 1997)

14.26 A. Tate: Planning. In: MIT Encyclopedia of the Cog-

nitive Sciences, (1999) pp. 652–653

14.27 T. Estlin, R. Castano, B. Anderson, D. Gaines,

F. Fisher, M. Judd: Learning and planning for Mars

rover science, Proc. Int. Joint Conf. Artif. Intell.

(IJCAI) (2003)

14.28 S.K. Gupta, D.A. Bourne, K. Kim, S.S. Krishanan:

Automated process planning for sheet metal bend-

ing operations, J. Manuf. Syst. 17(5), 338–360 (1998)

14.29 A. Gerevini, D. Long: Plan constraints and pref-

erences in pddl3: The language of the ﬁfth

international planning competition. Technical Re-

port (University of Brescia, 2005), available at

http://cs-www.cs.yale.edu/homes/dvm/papers/

pddl-ipc5.pdf

14.30 A.L. Blum, M.L. Furst: Fast planning through plan-

ning graph analysis, Proc. Int. Joint Conf. Artif.

Intell. (IJCAI) (1995) pp. 1636–1642

14.31 A. Tate, B. Drabble, R. Kirby: O-Plan2: An Architec-

ture for Command, Planning and Control (Morgan

Kaufmann, San Francisco 1994)

14.32 D.E. Wilkins: Practical Planning: Extending the

Classical AI Planning Paradigm (Morgan Kauf-

mann, San Mateo 1988)

14.33

D. Nau, T.-C. Au, O. Ilghami, U. Kuter, J.W. Murdock,

D. Wu, F. Yaman: SHOP2: An HTN planning system,

J. Artif. Intell. Res. 20, 379–404 (2003)

14.34 F. Bacchus, F. Kabanza: Using temporal logics to

express search control knowledge for planning,

Artif. Intell. 116(1/2), 123–191 (2000)

14.35 J. Kvarnström, P. Doherty: TALplanner: A temporal

logic based forward chaining planner, Ann. Math.

Artif. Intell. 30, 119–169 (2001)

14.36 M. Fox, D. E. Smith: Special track on the 4th

international planning competition. J. Artif. In-

tell. Res. (2006), available at http://www.jair.org/

specialtrack.html

14.37 B. Bonet, H. Geffner: Labeled RTDP: Improving the

convergence of real-time dynamic programming,

Proc. 13th Int. Conf. Autom. Plan. Sched. (ICAPS)

(AAAI, 2003), pp 12–21

14.38 D. Nau, T.-C. Au, O. Ilghami, U. Kuter, H. Mu

noz-

Avila, J.W. Murdock, D. Wu, F. Yaman: Applications

of SHOP and SHOP2, IEEE Intell. Syst. 20(2), 34–41

(2005)

14.39 M. Boddy, J. Gohde, J.T. Haigh, S. Harp: Course of

action generation for cyber security using classical

planning, Proc. 15th Int. Conf. Autom. Plan. Sched.

(ICAPS) (2005)

14.40 M. Ghallab, D. Nau, P. Traverso: Automated Plan-

ning: Theory and Practice (Morgan Kaufmann, San

Francisco 2004)

14.41 S.M. Lavalle: Planning Algorithms (Cambridge Uni-

versity Press, Cambridge 2006)

14.42 J. Schaeffer, N. Burch, Y. Björnsson, A. Kishimoto,

M. Müller, R. Lake, P. Lu, S. Sutphen: Checkers is

solved, Science 317(5844), 1518–1522 (2007)

14.43 D.E. Knuth, R.W. Moore: An analysis of alpha-beta

pruning, Artif. Intell. 6, 293–326 (1975)

14.44 G. Tesauro: Programming Backgammon Using

Self-Teaching Neural Nets (Elsevier, Essex 2002)

14.45 S.J.J. Smith, D.S. Nau, T. Throop: Computer bridge:

A big win for AI planning, AI Mag. 19(2), 93–105

(1998)

14.46 M. Harris: Laak-Eslami team defeats Polaris in

man-machine poker championship, Poker News

(2007), Available at www.pokernews.

com/news/2007/7/laak-eslami-team-defeats-

polaris-man-machine-poker-championship.htm

14.47 A. Parker, D. Nau, V.S. Subrahmanian: Over-

conﬁdence or paranoia? Search in imperfect-

information games, Proc. Natl. Conf. Artif. Intell.

(AAAI) (2006)

14.48 D. Billings: Thoughts on RoShamBo, Int. Comput.

Games Assoc. J. 23(1), 3–8 (2000)

14.49 B. Johanson: Robust strategies and counter-

strategies: Building a champion level computer

poker player. Master Thesis (University of Alberta,

2007)

14.50 S. Hart, R.J. Aumann (Eds.): Handbook of Game

Theory with Economic Applications 2 (North Hol-

land, Amsterdam 1994)

Part B 14

268 Part B Automation Theory and Scientiﬁc Foundations

14.51 F.-H. Hsu: Chess hardware in deep blue, Comput.

Sci. Eng. 8(1), 50–60 (2006)

14.52 J. Schaeffer. One Jump Ahead: Challenging Human

Supremacy in Checkers (Springer, Berlin, Heidel-

berg 1997)

14.53 J. Schaeffer: A gamut of games, AI Mag. 22(3), 29–46

(2001)

14.54 T. Sandholm: Expressive commerce and its appli-

cation to sourcing, Proc. Innov. Appl. Artif. Intell.

Conf. (IAAI) (AAAI Press, Menlo Park 2006)

14.55 T.-C. Au, D. Nau: Accident or intention: That is

the question (in the iterated prisoner’s dilemma),

Int. Joint Conf. Auton. Agents and Multiagent Syst.

(AAMAS) (2006)

14.56 C. Chelba, F. Jelinek: Structured language modeling

for speech recognition, Conf. Appl. Nat. Lang. Inf.

Syst. (NLDB) (1999)

14.57 B.H. Juang, L.R. Rabiner: Hidden Markov models

for speech recognition, Technometrics 33(3), 251–

272 (1991)

14.58 S. Lee, J. Tsujii, H. Rim: Lexicalized hidden Markov

models for part-of-speech tagging, Proc. 18th Int.

Conf. Comput. Linguist. (2000)

14.59 N. Chomsky: Syntactic Structures (Mouton, The

Hague 1957)

14.60 E. Charniak: A maximum-entropy-inspired parser,

Technical Report CS-99-12 (Brown University

1999)

14.61 F. Gaspari: Online MT Services and Real Users Needs:

An Empirical Usability Evaluation (Springer, Berlin,

Heidelberg 2004), pp 74–85

14.62 P. Jackson, I. Moulinier: Natural Language Pro-

cessing for Online Applications: Text Retrieval,

Extraction, and Categorization (John Benjamins,

Amsterdam 2002)

14.63 M. Sahami, T.D. Heilman: A web-based kernel

function for measuring the similarity of short text

snippets, WWW ’06: Proc. 15th Int. Conf. World Wide

Web (New York 2006) pp. 377–386

14.64 J. Burstein, M. Chodorow, C. Leacock: Automated

essay evaluation: the criterion online writing ser-

vice, AI Mag. 25(3), 27–36 (2004)

14.65 ZhiBiaoWu,LokeSooHsu,ChewLimTan:Asur-

vey of statistical approaches to natural language

processing, Technical Report TRA4/92 (National

University of Singapore 1992)

14.66 C. A. Thompson: A brief introduction to natural

language processing for nonlinguists. In: Learn-

ing Language in Logic, Lecture Notes in Computer

Science (LNCS) Ser., Vol. 1925, ed. by J. Cussens,

S. Dzeroski (Springer, New York 2000) pp. 36–48

14.67 H. Havenstein: Spring comes to AI winter, Comput-

erworld (2005), available at

http://www.computerworld.com/action/article.do?

command=viewArticleBasic&articleId=99691

14.68 B.G. Buchanan, E.H. Shortliffe (Eds.): Rule-Based

Expert Systems: The MYCIN Experiments of the

Stanford Heuristic Programming Project (Addison-

Wesley, Reading 1984)

14.69 R.G. Smith, J.D. Baker: The dipmeter advisor system

– a case study in commercial expert system devel-

opment, Proc. Int. Joint Conf. Artif. Intell. (IJCAI)

(1983) pp. 122–129

14.70 M. Steﬁk: Inferring DNA structures from segmenta-

tion data, Artif. Intell. 11(1/2), 85–114 (1978)

14.71 J.P. McDermott: R1 (“XCON”) at age 12: Lessons from

an elementary school achiever, Artif. Intell. 59(1/2),

241–247 (1993)

14.72 G. Steele: Common Lisp: The Language, 2nd edn.

(Digital, Woburn 1990)

14.73 P. Graham: ANSI Common Lisp (Prentice Hall, En-

glewood Cliffs 1995)

14.74 P. Graham: Hackers and Painters: Big Ideas from

the Computer Age (O’Reilly Media, Sebastopol 2004)

pp. 165–180, also available at

http://www.paulgraham.com/avg.html

14.75 N. Muscettola, P. Pandurang Nayak, B. Pell,

B.C. Williams: Remote agent: To boldly go where

no AI system has gone before, Artif. Intell. 103(1/2),

5–47 (1998)

14.76 W. Clocksin, C. Mellish: Programming in Prolog

(Springer, Berlin, Heidelberg 1981)

14.77 E. Feigenbaum, P. McCorduck: The Fifth Genera-

tion: Artiﬁcial Intelligence and Japan’s Computer

Challenge to the World (Addison-Wesley Longman,

Boston 1983)

14.78 D. Nau, J. Wilkenfeld (Ed): Proc. 1st Int. Conf. Com-

put. Cult. Dyn. (ICCCD-2007) (AAAI Press, Menlo Park

2007)

14.79 H. Liu, J. Salerno, M. Young (Ed): Proc. 1st Int.

WorkshopSoc.Comput.Behav.Model.Predict.

(Springer, Berlin, Heidelberg 2008)

Part B 14

269

Virtual Reality

15. Virtual Reality and Automation

P. Pat Banerjee

Virtual reality of human activities (e.g., in de-

sign, manufacturing, medical care, exploration

or military operations) often concentrates on an

automated interface between virtual reality (VR)

technology and the theory and practice of these

activities. In this chapter we focus mainly on the

role of VR technology in developing this inter-

face. Although the scope and range of applications

is large, two illustrative areas (production/service

applications and medical applications) are ex-

plained in some detail to offer some insight

into the magnitude of the beneﬁts and existing

challenges.

15.1 Overview of Virtual Reality

and Automation Technologies ............... 269

15.2 Production/

Service Applications .............................. 271

15.2.1 Design ........................................ 271

15.2.2 Material Handling

and Manufacturing Systems........... 271

15.3 Medical Applications ............................. 273

15.3.1 Neurosurgical Virtual Automation... 274

15.3.2 Ophthalmic Virtual Automation...... 275

15.3.3 Dental Virtual Automation............. 275

15.4 Conclusions and Emerging Trends .......... 276

References .................................................. 277

15.1 Overview of Virtual Reality and Automation Technologies

Virtual manufacturing and automation ﬁrst came into

prominence in the early 1990s, in part as a result of

the US Department of Defense Virtual Manufactur-

ing Initiative. The topic broadly refers to the modeling

of manufacturing systems and components with effec-

tive use of audio-visual and/or other sensory features

to simulate or design alternatives for an actual manu-

facturing environment mainly through effective use of

high-performance computers.

To understand the area, ﬁrst we introduce a broader

area of virtual reality (VR) and visual simulation sys-

tems. VR brings in a few exciting new developments.

Firstly it provides a major redeﬁnition of perspective

projection, by introducing the concept of user-centered

perspective. Traditional perspective projection is ﬁxed,

whereas in virtual reality one has the option to vary

the perspective in real time, thus closely mimicking our

natural experience of a three-dimensional (3-D)world.

In the past few years, VR and visual simulation sys-

tems have made a huge impact in terms of industrial

adoption; for example, Cyberedge Information Services

conducted a survey [15.1] that provided the following

details:

•

Industry growth remains strong: 9.8% in 2003

•

Industry value: US$42.6 billion worldwide

•

Five-year forecast: industryvalue reaches US$78 bil-

lion in 2008

•

Average system cost: US$356310

•

Number of systems sold in 2003: 338000

•

Companies involved in visual simulation world-

wide: 17334

•

Most common visual display system: monoscopic

desktop monitor

•

Least common visual display system: autostereo-

scopic display

•

Most common operating system: Microsoft Win-

dows XP.

The survey [15.1] further comments on the top 17appli-

cations of visual simulation systems from 1999–2003 as

follows:

Part B 15

270 Part B Automation Theory and Scientiﬁc Foundations

Table 15.1 Automation advantages of virtual reality (VR); * Unique to VR. General advantages of simulation, including

less costs, less risks, less delay, and ability to study as yet unbuilt structures/objects are also valid

Whatisautomated? Application example Advantage*

Application • Testing a simulated mold • Better visualization of tested product

domain of a new engine

• Developing battleﬁeld tactics • Greater ability to collaborate in planning

and design of complex virtual situations

Human supervisory • Training in dental procedures • Detailed hands-on experience in a large simulated

interface library of dental problems

• Teleoperating robots

Human feedback • Improving the safety •Greater precision and quality of feedback through

interface of a process haptics

• Learning the properties • Better understanding of simulated behaviors

of surfaces

Human application • Telerepair or teleassembly • Greater ability to obtain knowledge about and

interaction of remote objects manipulate in remote global and space locations

• Deeper sense of involvement and engagement

by the interacting humans

1. Software development/testing

2. Computer aided design (CAD)/computer-aided

manufacture (CAM) visualization or presentation

3. Postgraduate education (college)

4. Virtual prototype

5. Museum/exhibition

6. Design evaluation, general

7. Medical diagnostics

8. Undergraduate education (college)

a) b) c)

Fig. 15.1 (a) Sensics piSight head-mounted display for virtual prototyping, training, data mining, and other applications

(http://www.sensics.com/, December 2007).

(b) Pinch is an interacting system for interacting in a virtual environment

using sensors in each ﬁngertip (http://www.fakespace.com/, December 2007).

stereoscopic head-mounted display (http://www.vrealities.com/, December 2007)

9. Aerospace

10. Automobile, truck, heavy equipment

11. Games development

12. Collaborative work

13. Architecture

14. Medical training

15. Dangerous environment operations

16. Military operation training

17. Trade show exhibit.

Part B 15.1

Virtual Reality and Automation 15.2 Production/Service Applications 271

In each of these application areas, VR is advan-

tageous in automating and improving three critical

areas, as shown in Table 15.1. A smaller subsection

of VR and visual simulation is virtual manufactur-

ing and automation, which often concentrates on an

interface between VR technology and manufacturing

and automation theory and practice. In this chapter we

concentrate mainly on the role of VR technology in

developing this interface. Some areas that can bene-

ﬁt from development of virtual manufacturing include

product design [15.2, 3], hazardous operations model-

ing [15.4, 5], production process modeling [15.6, 7],

training [15.8,9], education [15.10,11], information vi-

sualization [15.12,13], telecommunications [15.14,15],

and teletravel. Lately a number of surgical simulation

applications have emerged.

VR is closely associated with an environment

commonly known as a virtual environment (VE).

VE systems differ from other previously developed

computer-centered systems in the extent to which real-

time interaction is facilitated, the perceived visual space

is three- rather than two-dimensional, the human–

machine interface is multimodal, and the operator is

immersed in the computer-generated environment. The

interactive, virtual image displays are enhanced by spe-

cial processing and by nonvisual display modalities,

such as auditory and haptic, to convince users that they

are immersed in a synthetic space.

The means of simulating a VE today is through im-

mersion in computer graphics coupled with acoustic

interface and domain-independent interacting devices,

such as wands, and domain-speciﬁc devices such as

the steering and brakes for cars, or earthmovers or

instrument clusters for airplanes (Fig. 15.1a–c). Im-

mersion gives the feeling of depth, which is essential

for a three-dimensional effect. Head-mounted displays,

stereoscopic projectors, and retinal display are some of

the technologies used for such environments.

15.2 Production/Service Applications

15.2.1 Design

The immersive display technology can be used for

creating virtual prototypes of products and processes.

Integrated production system engineering environments

can provide functions to specify, design, engineer, sim-

ulate, analyze, and evaluate a production system. Some

examplesof thefunctions whichmight be includedin an

integrated production system engineering environment

are:

1. Identiﬁcation of product speciﬁcations and produc-

tion system requirements

2. Producibility analysis for individual products

3. Modeling and speciﬁcation of manufacturing pro-

cesses

4. Measurement and analysis of process capabilities

5. Modiﬁcation of product designs to address manu-

facturability issues

6. Plant layout and facilities planning

7. Simulation and analysis of system performance

8. Consideration of various economic/cost tradeoffs of

different manufacturing processes, systems, tools,

and materials

9. Analysis supporting selection of systems/vendors

10. Procurement of manufacturing equipment and sup-

port systems

11. Speciﬁcation of interfaces and the integration of in-

formation systems

12. Task and workplace design

13. Management, scheduling, and tracking of projects.

The interoperability of the commercial engineering

tools that are available today is a challenge. Exam-

ples of production systems which may eventually be

engineered using this type of integrated environment in-

clude: transfer lines, group technology cells, automated

or manually operated workstations, customized multi-

purpose equipment, and entire plants.

15.2.2 Material Handling

and Manufacturing Systems

An important outcome of virtual manufacturing is vir-

tual factories of the future. When a single factory may

cost over a billion US dollars (as is the case, for in-

stance, in the semiconductor industry), it is evident that

manufacturing decision-makers need tools that support

good decision making about their design, deployment,

and operation. However, in the case of manufacturing

models, there is usually no testbed but the factory itself;

development of models of manufacturing operations is

very likely to disrupt factory operations while the mod-

els are being developed and tested.

Part B 15.2

272 Part B Automation Theory and Scientiﬁc Foundations

Virtual factories are based on sophisticated com-

puter simulations fora distributed, integrated, computer-

based composite model of a total manufacturing en-

vironment, incorporating all the tasks and resources

necessary to accomplish the operation of designing,

producing, and delivering a product. With virtual facto-

ries capable of accurately simulating factory operations

over time scales of months, managers would be able

to explore many potential production conﬁgurations

and schedules or different control and organizational

schemes at signiﬁcant savings of cost and time in order

to determine how best to improve performance. A vir-

tual factory model involves a comprehensive model or

structure for integrating a set of heterogeneous and hi-

erarchical submodels at various levels of abstraction.

An ultimate goal would be the creation of a demon-

stration platform that would compare the results of

real factory operations with the results of simulated

factory operations. This demonstration platform would

use a computer-based model of an existing factory and

would compare its performance with that of a similarly

equipped factory running the same product line, but us-

ing, for example, a new layout of equipment, a better

scheduling system, a paperless product and process de-

scription, or fewer or more human operators. The entire

factory would have to be represented in sufﬁcient de-

tail so that any model user, from factory manager to

equipment operator, would be able to extract useful re-

sults. To accomplish this, two broad areas need to be

addressed:

1. Hardware and software technology to handle so-

phisticated graphics and data-oriented models in

a useful and timely manner

2. Representation of manufacturing expertise in mod-

els in such a way that the results of model operation

satisfy manufacturing experts’ needs for accurate

responses.

Some of the research considerations related to vir-

tual modeling technology are as follows:

•

User interfaces. Since factory personnel in the 21st

century will be interacting with many applications,

it will notsufﬁcefor each application tohave its own

set of interfaces, no matter how good any individual

one is. Much thought has to be given both to the

nature of the speciﬁc interfaces and to the integra-

tion of interfaces in a system designed so as not to

confuse the user. Interface tools that allow the user

to ﬁlter and abstract large volumes of data will be

particularly important.

•

Model consistency. Models will be used to perform

a variety of geographically dispersed functions over

a short time scale during which the model informa-

tion must be globally accurate. In addition,pieces of

the model may themselves be widely distributed. As

a result, a method must be devised to ensure model

consistency and concurrency, perhaps for extended

time periods. The accuracy of models used concur-

rently for different purposes is a key determinant of

the beneﬁts of using such models.

•

Testing and validation of model concepts. Because

a major use of models is to make predictions about

matters that are not intuitively obvious to decision-

makers, testing and validation of models and their

use are very difﬁcult. For factory operations and

design alike, there are many potential right an-

swers to important questions, and none of these

is provably correct (e.g., what is the right sched-

ule?). As a result, models have to be validated

by being tested against understandable conditions,

and in many cases, common sense must be used

to judge if a model is correct. Because models

must be tested under stochastic factory conditions,

which are hard to duplicate or emulate, outside

the factory environment, an important area for re-

search involves developing tools for use in both

testing and validating model operation and behav-

ior. Tools for automating sensitivity analysis in

the testing of simulation models would help to

overcome model validation problems inherent in

a stochastic environment. For more details, please

refer to [15.16].

Figure 15.2 shows a schematic user interface for remote

facility management using virtual manufacturing con-

cepts. The foreground represents a factory ﬂoor layout

that identiﬁes important regions of the plant. The walls

separating different regions have been removed to ob-

tain a complete bird’s eye view of the plant. Transmitted

streaming motion from video capture of important re-

gions can be provided as a background. For this purpose

one needs to identify and specify the objects for motion

streaming. A combination of video and VR is a pow-

erful tool in this context. The advantage of streaming

motion coordinates to animate the factory ﬂoor would

be to give the ﬂoor manager an overview of the ﬂoor.

A suitable set of symbols can be designed to designate

the status modes of various regions, namely running

state, breakdown state, shutdown state, waiting state

(e.g., for parts, worker), scheduled maintenance state,

etc. An avataric representation of the ﬂoor manager can

Part B 15.2

Virtual Reality and Automation 15.3 Medical Applications 273

Scrolling text window ...

Floor manager guiding

operations remotely

Warehouse monitoring

supply chain optimization

Main background video,

audio and 3-D scene with

animated overlays

Product testing area,

communication with

product designers

Assembly training

of a new hire

Management of

other plants

Fig. 15.2 Conceptual demonstration of remote facility management user interface showing many telecollaborative activ-

ities

be used to guide operations. A window for the product

testing area and communication with product designers

can be used to efﬁciently streamline the product de-

sign process. A video camera device can be integrated

to stream motion coordinates after the objects in the

product design area and their layouts have been spec-

iﬁed. Another window showing the assembly training

of a new employee can illustrate the use of this con-

cept for another important area of the plant. A similar

feature can be designed for warehouse monitoring and

supply chain management. A window for management

of other plants can take one to different plants, located

anywhere in the world. Finally a scrolling text window

can pop up any important or emergency messages such

as suddenly scheduled meetings, changes of deadline,

sudden shifts in strategy, etc.

15.3 Medical Applications

Medical simulations using virtualautomation have been

a recent area of growth. Some of our activities in

this area are highlighted in this Section. Using hap-

tics and VR in high-ﬁdelity open-surgical simulation,

certain principles of design and operational prototyp-

ing can be highlighted. A philosophy of designing

Part B 15.3

274 Part B Automation Theory and Scientiﬁc Foundations

device prototypes by carefully analyzing contact based

on open surgical simulation requirements and map-

ping them with open-source software and off-the-shelf

hardware components is presented as an example. Ap-

plications in neurosurgery (ventriculostomy), ophthal-

mology (capsulorrhexis), and dentistry (periodontics)

are illustrated. First, a prototype device known as

ImmersiveTouch is described, followed by three appli-

cations.

ImmersiveTouch is a patent-pending [15.17]next-

generation augmented VR technology invented by us,

and is the ﬁrst system that integrates a haptic de-

vice with a head and hand tracking system, and

a high-resolution high-pixel-density stereoscopic dis-

play (Fig. 15.3). Its ergonomic design provides a com-

fortable working volume in the space of a standard

desktop. The haptic device is collocated with the 3-D

Dimensions: 42"W×79"H×29"D (deployed): 45"H×33"D (packed)

Fig. 15.3 Immersive touch and its neurosurgical applica-

tion

graphics, giving the user a more realistic and nat-

ural means to manipulate and modify 3-D data in

real time. The high-performance, multisensorial com-

puter interface allows relatively easy development of

VR simulation and training applications that appeal

to many stimuli: audio, visual, tactile, and kines-

thetic.

ImmersiveTouch represents an integrated hardware

and software solution. The hardware integrates 3-D

stereo visualization, force feedback, head and hand

tracking, and 3-D audio. The software provides a uni-

ﬁed applications programminginterface (API) to handle

volume processing, graphics rendering, haptics render-

ing, 3-D audio feedback, and interactive menus and

buttons. ImmersiveTouch is an evolutionary virtual-

reality system resulting from the integration of a series

of hardware solutions to 3-D display issues and

the development of a unique VR software platform

(API) drawing heavily on open-source software and

databases.

15.3.1 Neurosurgical Virtual Automation

Neurosurgical procedures, particularly cranial applica-

tions, lend themselves to VR simulation. The working

space around the cranium is limited. Anatomical re-

lationships within the skull are generally ﬁxed and

respiratory or somatic movements do not signiﬁcantly

impair imaging or rendering. The same issues that make

cranial procedures so suitable for intraoperative navi-

gation also apply to virtual operative simulation. The

complexity of so many cerebral structures also allows

little room for error, making the need for skill-set acqui-

sition prior to the procedure that much more signiﬁcant.

The emergence of realistic neurosurgical simulators has

been predicted since the mid-1990s when the explo-

sion in computer processing speed seemed to presage

future developments [15.18]. VR techniques have been

attempted to simulate several types of spinal procedures

such as lumbar puncture, in addition to craniotomy-type

procedures.

Ventriculostomy is a high-frequency surgical in-

tervention commonly employed for treatment of head

injury and stroke. This extensively studied procedure is

a good example problem for augmented VR simulators.

We investigated the ability of neurosurgery residents at

different levels of training to cannulate the ventricle, us-

ing computer tomography data on an ImmersiveTouch

workstation. After a small incision is made and a bur

hole is drilled on a strategically chosen spot in the

patient’s skull, a ventriculostomy catheter is inserted,

Part B 15.3