Seibel Peter. Coders at Work

Подождите немного. Документ загружается.

Fran Allen

495

I always like to have a picture—a model. Often a flowchart and some

specification about the interfaces. We were heavy users, of course, of

flowcharts at that time because one didn’t get access to the machine that

often and that was a very nice model for thinking through how parts of the

system would interact, what would be done, and specifying what would be

done where, and the functionality of the various components. I don’t know

what the analog of that is now.

Seibel: Even with flowcharting, there are formal flowcharts produced as

documentation and then there are the flowcharts you draw on a blackboard

to try and understand something. Was it more of the former or the latter?

Allen: In some cases, it would be formal flowcharts. Often in the kernels of

things there were some very complex pieces and one would do that.

Otherwise, it was just informal and a way of working at solving a problem.

Blackboards would be covered and become the record for the month or

whatever period of time.

Seibel: So the big project you led was the PTRAN compiler project, which

was when you first started working on explicit concurrency, as opposed to

the implicit concurrency in the CPU pipelines and so forth. When you

started that, that was a new thing, both for you and for IBM.

Allen: It was new for IBM, but we were very, very late coming into it. The

great work that initiated it from a real pragmatic point of view was at Illinois

starting in ’69 and ’70.

Seibel: And what language did the PTRAN compiler compile? Was it

straight Fortran with no added constructs for parallelism?

Allen: That’s right, that’s where we started from. What I wanted to do was

to do the same thing we’d done for optimization: The user writes a

sequential code in the language in a way that’s natural for the application and

then have the compiler do the optimization and mapping it to the machine

and taking advantage of concurrency.

In PTRAN, the idea was still to take the “dusty decks,” as we called them,

referring to an existing code base, and to automatically take advantage of

the hardware’s parallel components.

Fran Allen

496

Seibel: So, essentially, that was targeting what today we would think of as

symmetric multiprocessors?

Allen: Yeah, could be. There are many, many models of parallelism, which

is one of the difficulties. I think that that can be greatly simplified. But

multicore is one of the things that really are particularly interesting, for me

at least. But there are many models of parallelism.

We actually built it from existing work, particularly Dave Kuck’s. Some

work from NYU. We hired a group of newly minted PhDs from these

places that already had built up a lot of expertise. We had a quite a lot of

significant results, both on the practical and on the theoretical side—we

worked on both at the same time. I’m a very strong believer that one wants

to be able to take the practice into identifiable algorithms, and theory, and

ways of thinking about how to solve the problems and also to take the

algorithms into practice to see how really valuable they are, and how they

apply. I think our field is best done when it works on both sides on the

same projects.

Seibel: On the PTRAN project, you were leading a team. Were you still

coding at all by then?

Allen: I wasn’t doing the coding, but I was very close to it. As an example,

when the Static Single Assignment work was done, I didn’t see how it could

be implemented in any reasonable time. I mean, it was a very good

algorithm, but I didn’t see an implementation that was bounded in time and

space in some real way. So I had that challenge out there. I had to see that

code. I needed it. It had to be implemented. It couldn’t be just a paper—a

very nice paper, a famous paper—that shows graphs and complexity

bounds.

If we can’t implement it in a real system, that challenge would still be out

there. It wasn’t going to be as useful as I wanted it to be. Finally, one of my

people had an encoding. And I walked through that, every piece of that

code, and looked at the data structures that were used. And it was

astounding. I said, “This is it. It works.”

Seibel: So you were looking at all of the pieces that were going to go into

the whole system?

Fran Allen

497

Allen: Yes, yes, yes.

Seibel: You also were managing all these people? Or were you a technical

architect with someone else managing the group?

Allen: No, I was the research manager for the group. There were about 10

or 12 core people and we divided the work up so that each person really

had ownership of a piece of it.

Seibel: People have been debating at least since Gerald Weinberg’s book

The Psychology of Computer Programming whether it’s better for people to

“own” code, so they take responsibility for it, or to have people work more

collaboratively so you avoid having silos that only one person understands.

It sounds like you thought dividing up the ownership was the way to go?

Allen: We worked collaboratively, but the collaboration was about the

state of the system, of the implementation. And some people were very

good at the implementation, and so they’d own a piece—some piece of the

optimizer or the intra-procedural analysis was definitely one or two people.

But also, there were a number of people that were doing a lot of the theory

work, or the abstract work of writing the papers, and writing a lot of the

papers and algorithms. And it was the bringing together of those two special

parts that I think really made the group so strong.

This was a period of a lot of work going on in the analysis and

transformations for parallelism. So what I tried to do was, for each of the

people that were doing theory to get them to write some code, express it

in code as a part of the system. And the people who were just doing the

other part, well I’d try to get them to write things up so they were more

generally available.

Seibel: A lot of programmers will do anything to avoid becoming a

manager. Was managing something that you also enjoyed?

Allen: Well, Research in the earlier times did not distinguish—doing

management was not a promotion, not a salary raise. It was just somebody

had to manage this piece of work, and “OK, don’t you want to do it?” Or,

“You’re the obvious one to manage this piece of work.” And it was

technical management; there wasn’t much people management involved. But

Fran Allen

498

in Research people are RSMs, Research Staff Members, the day they enter,

and for the rest of their career mostly. All my colleagues, that’s what we

are. So one moved in and out of management without any stigma attached.

Seibel: So presumably the people who got chosen to manage were the

ones who were actually good at it. How did you pick up those skills?

Allen: Well, I was sent to management school. Everybody was, back when I

was first appointed. But I think it goes back to when I grew up on the farm. I

was the oldest of six kids and there were five of us in a row, so my parents

were pretty overwhelmed. So it was a natural role for me in some ways.

Seibel: One of the difficulties of the kind of technical management that you

were doing is finding the balance between having your own technical

opinions about how stuff should be done and giving people room to put

their own ideas in.

Allen: I think I had some hard lessons on the Stretch project. I remember

some of the people on the project came in and said to me one day that we

ought to be using list and hash functions. Well, we knew about list programs

but hashing was new. So a couple of my people came and said they wanted

to use hashing for the symbol table. And I said, “No, we can’t do that. We

don’t know how to do that.” Yadda, yadda, yadda. The next Monday I came

in and they had done it. They had torn down the system and rebuilt it with

hashing. It worked, and it was much faster. So that was a big lesson to me. I

should be much more open to some brand-new ideas.

Seibel: So sometimes—maybe even often—your people actually know

what they’re talking about and you shouldn’t interfere too much because

you might stomp out a good idea. It’s trickier when you’re really right and

their idea really is a little bit flawed but you don’t want to beat up on them

too much.

Allen: There was some of that. It was often where somebody came in with

a knowledge of some area and wanted to apply that knowledge to an

ongoing piece of project without having been embedded in the project long

enough to know, and often up against a deadline.

Fran Allen

499

I ran into it big time doing some subcontracting work. I had a group of

people that was doing wonderful work building an optimizer based on the

work we’ve done here for PL/I, a big, different language. But one of the

people working for the subcontractor had just discovered object-oriented

programming and decided that he would apply it to the extreme. And I

couldn’t stop him, even though I was the contract overseer, and the project

was destroyed. Ultimately, what did it was PL/I has lots of pointers and

tracing a pointer is done all the time, and it took 11 instructions to trace the

value, to find the value of one pointer consistently.

Seibel: You mean in the generated code.

Allen: In the generated code and in the compiler itself because it was

bootstrapping the compiler. Every time you made a step you had to check

that the thing’s valid. And you were checking, and rechecking, and

rechecking. It still happens today. Some of these lessons we haven’t learned

well. And I guess I was not dealing with it very well, because I should have

just pointed out what the cost of applying object-oriented technology in that

kind of situation. It was just hopelessly slow, so the whole thing got

canceled.

Seibel: When were you most directly building a product for IBM, with

production deadlines that had to be met?

Allen: Certainly, the Stretch project was that way. And I’ve worked in

product development two or three times, and been in the situation where

one has week-by-week code reviews right up against deadlines. I have a lot

of respect for those processes and how important they are for the end

result and for the team that’s doing it. It can be very painful to sit there

every Friday and do code reads with people explaining why they’re doing

what they’re doing and finding other people’s errors.

Seibel: Painful, but worthwhile?

Allen: Absolutely worthwhile. On the PTRAN project, towards the end

when we were shipping a piece of it to the products, a half a day was

devoted to explaining errors in our code, their code, whatever it was, every

week. That went on for ten months, I think. Devoted Friday afternoon to it.

Fran Allen

500

Seibel: When you were working in those settings did you feel like you had

a process that let you estimate how long it’s going to take to build X

amount of software with any kind of accuracy?

Allen: Well, they did. The product-development people certainly did. It was

all tracked, and I’m sure it still is. Part of it would be to statistically get a

handle on the quality of the code. How many bugs showed up this week,

yeah. I liked the environment of a product lab because it’s sort of where

things get real.

Seibel: When you were hiring programmers what did you look for?

Allen: Well, I had a lot of connections in the universities. NYU had some

fabulous compiler-writer faculty—that group was really well-trained to

write compiler code.

Seibel: So you could hire people recommended by the professors you

knew and trusted. How about when you have to interview someone who

doesn’t come with a recommendation from someone you trust—how do

you figure out in the course of a couple of hours whether they are going to

be a good programmer?

Allen: I always start with interviewing anybody for IBM Research by trying

to find out what they’re excited about. That’s kind of a basic threshold for

me. And it doesn’t matter whether it has anything to do with programming

or computers. If they can’t get enthusiastic about something, they’re not

going to get charged up in a group.

Sometimes one takes a high risk. I took a high risk on a guy whose thesis

adviser wrote that he was very dyslexic. He didn’t work out so well here

because he didn’t fit in some ways, but he started his own company and I

still go to him for advice, for insights on how to do things technically. He

just knows things. So that wasn’t a mistake. It was a mistake in terms of the

project, but not in terms of his relationship with a lot of us.

Seibel: Lately you’ve been involved in mentoring—there’s a mentoring

award here at IBM that is named for you. Do you have any ideas about how

new programmers should be brought along into being better programmers?

Fran Allen

501

Allen: That aspect of mentoring, I don’t get very close to these days. What

I do do, though, is encourage a young person starting out to not jump into

becoming a manager, which is very tempting for people who have a talent in

that direction. Get a reputation for technical work. Whether it’s a nice

piece of science, an algorithm, or writing great code—whatever it is,

establish a strong reputation there first. That’ll serve you well if you do

want to go to managing projects and so forth, to have learned the discipline

of what it takes to do that and how to function in that way.

Seibel: Is it possible to have a great manager who’s actually not very

technically skilled, but is very good at organizing the efforts of others?

Allen: Yes, as long as he doesn’t think he’s good at the technology, and is

able to distinguish between who is and who isn’t in the people that work for

him.

Seibel: That’s probably the trickiest thing. For you, what distinguishes the

really best programmers?

Allen: Well, I always like to find the people who can turn a light bulb on

over my head. That’s really important for me because I spend a lot of time

thinking about systems, so I like to have people that will be able, at least in

Research, to show me something new and interesting or a new way of

looking at something—a new way of solving a problem.

Also, I rely on what other people think. I’ve been wrong plenty and it’s a

real learning experience when I find myself thinking more highly of

somebody than the group does. If you have a good group, it’s a very good

way of sorting out who’s doing good work.

Seibel: When do you think was the last time that you programmed?

Allen: Oh, it was quite a while ago. I kind of stopped when C came out.

That was a big blow. We were making so much good progress on

optimizations and transformations. We were getting rid of just one nice

problem after another. When C came out, at one of the SIGPLAN compiler

conferences, there was a debate between Steve Johnson from Bell Labs,

who was supporting C, and one of our people, Bill Harrison, who was

Fran Allen

502

working on a project that I had at that time supporting automatic

optimization.

The nubbin of the debate was Steve’s defense of not having to build

optimizers anymore because the programmer would take care of it. That it

was really a programmer’s issue. The motivation for the design of C was

three problems they couldn’t solve in the high-level languages: One of them

was interrupt handling. Another was scheduling resources, taking over the

machine and scheduling a process that was in the queue. And a third one

was allocating memory. And you couldn’t do that from a high-level language.

So that was the excuse for C.

Seibel: Do you think C is a reasonable language if they had restricted its

use to operating-system kernels?

Allen: Oh, yeah. That would have been fine. And, in fact, you need to have

something like that, something where experts can really fine-tune without

big bottlenecks because those are key problems to solve.

By 1960, we had a long list of amazing languages: Lisp, APL, Fortran,

COBOL, Algol 60. These are higher-level than C. We have seriously

regressed, since C developed. C has destroyed our ability to advance the

state of the art in automatic optimization, automatic parallelization,

automatic mapping of a high-level language to the machine. This is one of

the reasons compilers are . . . basically not taught much anymore in the

colleges and universities.

Seibel: Surely there are still courses on building a compiler?

Allen: Not in lots of schools. It’s shocking. there are still conferences going

on, and people doing good algorithms, good work, but the payoff for that is,

in my opinion, quite minimal. Because languages like C totally overspecify

the solution of problems. Those kinds of languages are what is destroying

computer science as a study.

Seibel: But most newer languages these days are higher-level than C.

Things like Java and C# and Python and Ruby.

Fran Allen

503

Allen: But they still overspecify. The core thing is that it specifies location

of data. If you look at these other languages, they stayed away from

specifying the location of data and how to move it, where to put it in the

machine. It was ultimately about its value at any point.

Seibel: But very few languages other than C and C++ have raw pointers

anymore. Java has garbage collection and the data moves around. Would

you say that’s still overspecified?

Allen: Yes. I believe that there’s an opportunity to do what we have done

with computation in the optimization world with data. We don’t manage

data very well. We don’t have good ways of managing data automatically—

establishing locality of data that’s going to be used together.

There are lots of threads of research now which are very exciting. But I

think what’s missing is the bigger, bolder concepts. A lot of this is happening

within a space that is bounded by what exists already or the current

thinking. It’s not going to change overnight by any means—there are millions

of lines of code out there. But we do need to start trying to break the

boundaries of, “This’ll be done here and that’ll be done there.”

Seibel: Your career has been largely in high-performance computing. Yet

by 2019, or whatever, we’re supposed to have 1,000 cores in a notebook

computer. Does that mean high-performance computing and everyday

computing will merge? Or will high-performance computing always be out

there doing things in a sufficiently different way?

Allen: Well, it kind of depends on where one is on the scale. To go to the

petaflop, which is our current goal in high-performance computing—I don’t

know how that’s going to go. Certainly the game in performance is going to

be at the multicore because it’s driven by reducing energy and lots of good

things and solving some problems with the basic physics.

And there’s a competitive element that’s just going to drive it. But

harnessing those multicores pushes the problem out of the hardware space

into the software space. And that is where we’re not prepared to make any

progress as far as I can see. To harness these multicores—I think that’s

where the new language levels are going to have to break in. We should do

an end-to-end look at it. But it’s going to take some very new thinking.

Fran Allen

504

I think these first 50 years—60 years maybe; ENIAC was in ’44, ’43—we’ve

built up not only a wonderful, amazing legacy—just astounding—but also

some artifacts that we need to get rid of. It’ll take a very long time to

replace these and I think it’s a little hard to predict how that will evolve. But

it’ll evolve very fast if we can get some new thinking going in the right

places. We know how to do computations on a lot of stuff. We don’t know

how to deliver the data to the computation elements in the machine.

Seibel: Can you give a simple example of what you mean by bringing the

data to the computation in contrast to what we know how to do now?

Allen: To me it means taking over the management of the data, Basically,

how we do it now is by reference—it’s moved by hardware, or by the

underlying operating systems and support systems. And often the references

are at an element level.

Seibel: You mean in the sense that you can have a pointer into the middle

of a struct or an array?

Allen: Yeah, into an element of it. And that brings, depending on the

protocols of the hardware and of the architecture itself, the value to where

it can be used as part of the computation.

But another way to do that would be to organize locations of data in their

relative positions as a target of optimization. The other part of it is that very

often what is good for one computation is poor for another. One

organization, even of simple things like matrices, is bad when you’re actually

accessing it in a different way. So it’s a combination of the order of the

accessing against the location. It may require some architectural work, and

hardware work, but I think that one can do this if we put some of the

referencing, addressing capabilities back out in the hardware itself. There

are machines where one has the ability, at the point data comes into the

memory, to do quite a lot of transformations. Mapping can happen there.

Computation speed is what we measure, mostly, in high-performance

computing so we go through all kinds of things to increase that speed.

Feeding that computational unit is one of the big issues that we face, but we

never made it a first-order problem to solve. We leave it to the hardware.