Fuller S.H., Millett L.I. The Future of Computing Performance: Game Over or Next Level?

Подождите немного. Документ загружается.

The Future of Computing Performance: Game Over or Next Level?

x PREFACE

puters to the U.S. industrial and social infrastructure, economy, and

national security. The United States cannot afford to let this growth engine

stall out, and a concerted effort is needed to sustain it. Several centers

for parallel computing have already been established in leading research

universities. Those centers are a good start, and additional, strong actions

are required in many subdisciplines of computer science and computer

engineering. Our major goal for this study is to help to identify the actions

and opportunities that will prove most fruitful.

Samuel H. Fuller, Chair

Committee on Sustaining Growth

in Computing Performance

The Future of Computing Performance: Game Over or Next Level?

Acknowledgment of Reviewers

his report has been reviewed in draft form by individuals chosen

for their diverse perspectives and technical expertise, in accordance

with procedures approved by the National Research Council’s

(NRC’s) Report Review Committee. The purpose of this independent

review is to provide candid and critical comments that will assist the

institution in making its published report as sound as possible and to

ensure that the report meets institutional standards for objectivity, evi-

dence, and responsiveness to the study charge. The review comments

and draft manuscript remain conﬁdential to protect the integrity of the

deliberative process. We wish to thank the following individuals for their

review of this report:

Tilak Agerwala, IBM Research,

David Ceperley, University of Illinois,

Robert Dennard, IBM Research,

Robert Doering, Texas Instruments, Inc.,

Urs Hölzle, Google, Inc.,

Norm Jouppi, Hewlett-Packard Laboratories,

Kevin Kahn, Intel Corporation,

James Kajiya, Microsoft Corporation

Randy Katz, University of California, Berkeley,

Barbara Liskov, Massachusetts Institute of Technology,

Keshav Pingali, University of Texas, Austin,

James Plummer, Stanford University, and

Vivek Sarkar, Rice University.

The Future of Computing Performance: Game Over or Next Level?

xii ACKNOWLEDGMENTS

Although the reviewers listed above have provided many construc-

tive comments and suggestions, they were not asked to endorse the con-

clusions or recommendations, nor did they see the ﬁnal draft of the report

before its release. The review of this report was overseen by Butler Lamp-

son, Microsoft Corporation. Appointed by the National Research Council,

he was responsible for making certain that an independent examination

of this report was carried out in accordance with institutional procedures

and that all review comments were carefully considered. Responsibility

for the ﬁnal content of this report rests entirely with the authoring com-

mittee and the institution.

The Future of Computing Performance: Game Over or Next Level?

xiii

Contents

ABSTRACT 1

SUMMARY 5

1 THE NEED FOR CONTINUED PERFORMANCE GROWTH 21

Why Faster Computers Are Important, 22

The Importance of Computing Performance for the Sciences, 29

The Importance of Computing Performance for Defense and

National Security, 36

The Importance of Computing Performance for Consumer

Needs and Applications, 44

The Importance of Computing Performance for Enterprise

Productivity, 47

2 WHAT IS COMPUTER PERFORMANCE? 53

Why Performance Matters, 58

Performance as Measured by Raw Computation, 59

Computation and Communication’s Effects on Performance, 62

Technology Advances and the History of Computer Performance, 65

Assessing Performance with Benchmarks, 68

The Interplay of Software and Performance, 70

The Economics of Computer Performance, 75

The Future of Computing Performance: Game Over or Next Level?

xiv CONTENTS

3 POWER IS NOW LIMITING GROWTH IN COMPUTING 80

PERFORMANCE

Basic Technology Scaling, 83

Classic CMOS Scaling, 84

How CMOS-Processor Performance Improved Exponentially,

and Then Slowed, 87

How Chip Multiprocessors Allow Some Continued

Performance-Scaling, 90

Problems in Scaling Nanometer Devices, 94

Advanced Technology Options, 97

Application-Speciﬁc Integrated Circuits, 100

Bibliography, 103

4 THE END OF PROGRAMMING AS WE KNOW IT 105

Moore’s Bounty: Software Abstraction, 106

Software Implications of Parallelism, 110

The Challenges of Parallelism, 116

The State of the Art of Parallel Programming, 119

Parallel-Programming Systems and the Parallel

Software “Stack,” 127

Meeting the Challenges of Parallelism, 130

5 RESEARCH, PRACTICE, AND EDUCATION TO MEET 132

TOMORROW’S PERFORMANCE NEEDS

Systems Research and Practice, 133

Parallel-Programming Models and Education, 146

Game Over or Next Level? 150

APPENDIXES

A A History of Computer Performance 155

B Biographies of Committee Members and Staff 160

C Reprint of Gordon E. Moore’s “Cramming More Components

onto Integrated Circuits” 169

D Reprint of Robert H. Dennard’s “Design of Ion-Implanted

MOSFET’s with Very Small Physical Dimensions” 174

The Future of Computing Performance: Game Over or Next Level?

Abstract

nformation technology (IT) has the potential to continue to dramati-

cally transform how we work and live. One might expect that future IT

advances will occur as a natural continuation of the stunning advances

that IT has enabled over the last half-century, but reality is more sobering.

IT advances of the last half-century have depended critically on the

rapid growth of single-processor performance—by a factor of 10,000 in

just the last 2 decades—at ever-decreasing cost and with manageable

increases in power consumption. That growth stemmed from increasing

the number and speed of transistors on a processor chip by reducing

their size and—with improvements in memory, storage, and network-

ing capacities—resulted in ever more capable computer systems. It was

important for widespread IT adoption that the phenomenal growth in

performance was achieved while maintaining the sequential stored-program

model that was developed for computers in the 1940s. Moreover, computer

manufacturers worked to ensure that speciﬁc instruction set compatibil-

ity was maintained over generations of computer hardware—that is, a

new computer could run new applications, and the existing applications

would run faster. Thus, software did not have to be rewritten for each

hardware generation, and so ambition and imagination were free to drive

the creation of increasingly innovative, capable, and computationally

intensive software, and this in turn inspired businesses, government, and

the average consumer to buy successive generations of computer software

and hardware. Software and hardware advances fed each other, creating

a virtuous IT economic cycle.

The Future of Computing Performance: Game Over or Next Level?

2 THE FUTURE OF COMPUTING PERFORMANCE

Early in the 21st century, improvements in single-processor perfor-

mance slowed, as measured in instructions executed per second, and such

performance now improves at a very modest pace, if at all. This abrupt

shift is due to fundamental limits in the power efﬁciency of complemen-

tary metal oxide semiconductor integrated circuits (used in virtually all

computer chips today) and apparent limits in the efﬁciencies that can be

exploited in single-processor architectures. Reductions in transistor size

continue apace, and so more transistors can still be packed onto chips,

albeit without the speedups seen in the past. As a result, the computer-

hardware industry has commenced building chips with multiple proces-

sors. Current chips range from several complex processors to hundreds

of simpler processors, and future generations will keep adding more.

Unfortunately, that change in hardware requires a concomitant change in

the software programming model. To use chip multiprocessors, applica-

tions must use a parallel programming model, which divides a program into

parts that are then executed in parallel on distinct processors. However,

much software today is written according to a sequential programming

model, and applications written this way cannot easily be sped up by

using parallel processors.

The only foreseeable way to continue advancing performance is to

match parallel hardware with parallel software and ensure that the new

software is portable across generations of parallel hardware. There has

been genuine progress on the software front in speciﬁc ﬁelds, such as

some scientiﬁc applications and commercial searching and transactional

applications. Heroic programmers can exploit vast amounts of parallel-

ism, domain-speciﬁc languages ﬂourish, and powerful abstractions hide

complexity. However, none of those developments comes close to the

ubiquitous support for programming parallel hardware that is required

to ensure that IT’s effect on society over the next two decades will be as

stunning as it has been over the last half-century.

For those reasons, the Committee on Sustaining Growth in Com-

puting Performance recommends that our nation place a much greater

emphasis on IT and computer-science research and development focused

on improvements and innovations in parallel processing, and on making

the transition to computing centered on parallelism. The following should

have high priority:

· Algorithms that can exploit parallel processing;

· New computing “stacks” (applications, programming languages,

compilers, runtime/virtual machines, operating systems, and

architectures) that execute parallel rather than sequential pro-

grams and that effectively manage software parallelism, hard-

ware parallelism, power, memory, and other resources;

The Future of Computing Performance: Game Over or Next Level?

ABSTRACT 3

· Portable programming models that allow expert and typical pro-

grammers to express parallelism easily and allow software to be

efﬁciently reused on multiple generations of evolving hardware;

· Parallel-computing architectures driven by applications, includ-

ing enhancements of chip multiprocessors, conventional data-

parallel architectures, application-speciﬁc architectures, and radi-

cally different architectures;

· Open interface standards for parallel programming systems that

promote cooperation and innovation to accelerate the transition

to practical parallel computing systems; and

· Engineering and computer-science educational programs that

incorporate an increased emphasis on parallelism and use a vari-

ety of methods and approaches to better prepare students for the

types of computing resources that they will encounter in their

careers.

Although all of those areas are important, fundamental power and

energy constraints mean that even the best efforts might not yield a com-

plete solution. Parallel computing systems will grow in performance over

the long term only if they can become more power-efﬁcient. Therefore, in

addition to a focus on parallel processing, we need research and devel-

opment on much more power-efﬁcient computing systems at all levels

of technology, including devices, hardware architecture, and software

systems.

The Future of Computing Performance: Game Over or Next Level?

Summary

he end of dramatic exponential growth in single-processor perfor-

mance marks the end of the dominance of the single microprocessor

in computing. The era of sequential computing must give way to

a new era in which parallelism is at the forefront. Although important

scientiﬁc and engineering challenges lie ahead, this is an opportune time

for innovation in programming systems and computing architectures. We

have already begun to see diversity in computer designs to optimize for

such considerations as power and throughput. The next generation of dis-

coveries is likely to require advances at both the hardware and software

levels of computing systems.

There is no guarantee that we can make parallel computing as com-

mon and easy to use as yesterday’s sequential single-processor computer

systems, but unless we aggressively pursue efforts suggested by the rec-

ommendations below, it will be “game over” for growth in computing

performance. If parallel programming and related software efforts fail to

become widespread, the development of exciting new applications that

drive the computer industry will stall; if such innovation stalls, many

other parts of the economy will follow suit.

This report of the Committee on Sustaining Growth in Computing

Performance describes the factors that have led to the future limitations

on growth for single processors based on complementary metal oxide

semiconductor (CMOS) technology. The recommendations that follow

are aimed at supporting and focusing research, development, and educa-