Seibel Peter. Coders at Work
Подождите немного. Документ загружается.
Peter Norvig
315
Sometimes I feel guilty about that. Is that a failure on my part? I didn’t
understand what the bug was. I didn’t find the bug. I just dropped a bomb on
the house and blew up all the bugs and built a new house. In some sense,
the bug eluded me. But if it becomes the right solution, maybe it’s OK.
You’ve done it faster than you would have by finding it.
Seibel: What about things like assertions or invariants? How formally do
you think about those kinds of things when you’re coding?
Norvig: I guess I’m more on the informal side. I haven’t used languages
where that’s a big part of the formal mechanism, other than just type
declarations. Like loop invariants: I’ve always thought that was more trouble
than it was worth. I occasionally have a problem where this loop doesn’t
terminate, but mostly you don’t, and I just feel like it slows you down to do
the formal part. And if you do have a problem, the debugger will tell you
what loop you’re stuck inside. I guess if you’re writing high-dependability
software that’s embedded in something that it’s really important that it
doesn’t fail, then you really want to prove everything. But just in terms of
getting the first version of the program running or debugging it, I’d rather
move fast towards that than worry about the degree of formal specification
you need later.
Seibel: Have you ever done anything explicit to try and learn from the bugs
that you’ve created?
Norvig: Yeah, I think that’s pretty interesting and I wish I could do more
with that. I’m actually in a discussion now to see if I can do an experiment,
company-wide and then maybe for the world at large, to understand more
some of these issues. How do you classify bugs, but also what are some
factors in terms of productivity? How do you know? Is there a certain type
of person? What are the factors of that person that makes them more
productive? And I think it’s more interesting what the controllable factors
are that make somebody do better. If giving them a bigger monitor increases
productivity by such and such a percent, then you should probably do it.
Seibel: People are going to hate you when you discover that actually, really
tiny monitors make people more productive.
Download at Boykma.Com
Peter Norvig
316
Norvig: Right. And if providing quiet is important, then you should
probably do that, but on the other hand, if providing communication
between team members is important, then you should do that, and how do
you balance those two?
I just started thinking about what’s the right way to do that. How do you set
up an experiment? What do you track? Do we have numbers already that
we could make use of just by adding in some kind of questionnaire? Do we
have to set up an experiment?
Seibel: It’s often claimed that there are orders of magnitude differences in
productivity between programmers. Yet I read somewhere some criticism
of those claims, saying the studies that found that were done quite some
time ago, and a lot of things have changed about programming since then
that could have accounted for the differences—such as some people in the
study were still using batch processing techniques while other people were
using timesharing programming environments.
Norvig: I don’t think that’s all of it, because I think there were some
differences within the same organization using more of the same tools. I also
remember there were criticisms of some of the studies of finding
correlation, but not knowing cause and effect. If you found that the
programmers in the big corner offices were more productive, is that
because you reward the good programmers with the offices, or is it because
the offices makes them better? You can’t really come to a conclusion.
Seibel: Do you still enjoy programming as much as when you were starting
out?
Norvig: Yeah, but it’s frustrating to not know everything. I’m not doing it
as much, so I forget a little bit. And there are all these new things. I really
should redesign my web site and it should have JavaScript on the client side.
It should have PHP or something like that, and I just haven’t gotten up the
momentum to learn all that stuff and be able to do it.
Seibel: Do you think programming is a young person’s game?
Norvig: I think it helps in some ways. We certainly have a range of people
here that are exceptionally good at all levels and all ages. I think the
Download at Boykma.Com
Peter Norvig
317
advantage of being young is it’s important to grasp the whole program, the
whole problem, in your head—being able to concentrate. And I think that’s
easier when you’re younger, because your brain is better at it, or maybe it’s
just that you have less distractions. If you have kids, and family, and so on,
you just can’t devote as many consecutive hours as when you don’t. So
that’s part of it. But on the other hand, you’ve got this range of experience,
so you can make up for that in some ways by being able to do more
because you know how to do it.
Seibel: One of the aspects of the modern style of programming, as you
were saying, is that programmers have to absorb things quickly. How do
you tackle the problem of understanding a big pile of code, none of which
you’ve ever seen before?
Norvig: I think you do a mix of statically and dynamically. You start reading
the code and trying to make sense of it and then you get some traces of
what calls what, and where most of time is spent, and what’s the flow
through it. Then try to do something. Say, “I’m going to make this trivial
little change.” Or go to the issues database and say, “I’ll take this one.” In
order to do that, I have to learn a little piece of it. There’s only a little piece,
but you get that done and you move on to the next one.
Seibel: Have you ever done literate programming a la Knuth?
Norvig: I never used his tools per se. I’ve certainly written macros and so
on. And I’ve used the Java docs and things like that. In many ways, Lisp
programming encourages you to make your own system as you go, and so it
ends up being literate in that way. You find your own macros for your own
application-specific programming, and part of that is the documentation, part
of it’s the data, and part of it’s the code, so I’ve certainly done that. Then
more recently, in whatever language I’m using, whether it’s Java, or Python,
or whatever, I’ve certainly been careful to write test cases, and document
around that.
You look at Knuth’s original Literate Programming, and he was really trying to
say, “What’s the best order for writing a book,” assuming that someone’s
going to read the whole book and he wants it to be in a logical order.
People don’t do that anymore. They don’t want to read a book. They want
an index so they can say, “What’s the least amount of this book that I have
Download at Boykma.Com
Peter Norvig
318
to read? I just want to find the three paragraphs that I need. Show me that
and then I’ll move on.” I think that’s a real change.
Seibel: I wonder if there isn’t a way to write modern-style literate
programming. Certainly Knuth’s tools give you an index and beautiful cross-
referencing. I wonder if perhaps a modern approach to literate
programming just would organize the book differently—both as a whole
program and as a bunch of pieces that you can understand in bits?
Norvig: I don’t know. I think he was solving a problem that doesn’t exist
anymore to a large degree. Part of it was because he wanted to put it in a
linear order rather than in a web-like or a searchable order. I think part of it
was the limitations. I think he was using Pascal originally. And there it’s
pretty strict in terms of what had to be declared first and not necessarily in
the order you want. Modern languages are more free in that order, so I
think it’s less of an issue now.
Seibel: You mentioned reading Strachey’s checkers code in Scientific
American. And in your “Teach Yourself Programming in Ten Years” essay,
you talk about the importance of reading code. What code did you read
coming up?
Norvig: I read a lot of the Symbolics’s code, because that was available
when I was at Berkeley.
Seibel: Was that just because it was available and was interesting? Or were
you reading it to try to understand some behaviors you were observing?
Norvig: Both. I sometimes just tried to figure out how things work and
sometimes I needed it to solve a problem.
Seibel: So when you’re reading just for general edification, how do you
approach that?
Norvig: I think it’s probably interest-driven. “Gee, this file system allows
you to read files across the net using the same protocol that you use locally
on your machine—I wonder how it does that?” And so you’ll say, “Maybe
it’s in the open function.” You look there, and say, “Oh, that calls this other
Download at Boykma.Com
Peter Norvig
319
thing.” And you look there, and eventually you say, “Oh, this is how it does
it.”
Seibel: Have you read any of Knuth’s literate programs in book form?
Norvig: I’ve certainly picked up the books and flipped through them. I
could say I glanced at them, but I haven’t studied them.
Seibel: What about The Art of Computer Programming? Some of the people
I’ve talked to on this have absolutely read it from cover to cover. Some
people have it on the shelf and use it as a reference. And some people just
have it on the shelf.
Norvig: At one point I had it as my monitor stand because it was one of
the biggest set of books I had, and it was just the right height. That was nice
because it was always there, and I guess then I was more prone to use it as
a reference because it was just right in front of me.
Seibel: But you had to lift up the monitor every time you wanted to look at
it?
Norvig: No, I had the box set. You had to pull hard, but you could pull one
of the box. Now I’m less likely to use any book for reference—I’m just
likely to do a search.
Seibel: Just because it’s more convenient?
Norvig: It’s convenient. I think it’s also that I’m probably more goal-
oriented. Knuth is good if you say, “I want to know everything about this
subject.” But usually I’m saying, “I want to know if A is better than B,” or, “I
want to know the asymptotic complexity of this, and once I’ve got that, I
don’t need all the details of how we got there.”
Seibel: As a programmer, do you consider yourself a scientist, an engineer,
an artist, or a craftsman?
Norvig: Well, I know when you compare the various titles of books and so
on, I always thought the “craft” was the right answer. So I thought art was a
little pretentious because the purpose of art is to be beautiful or to have an
emotional contact or emotional impact, and I don’t feel like that’s anything
Download at Boykma.Com
Peter Norvig
320
that I try to do. Certainly I want programs to be pretty in some ways, and
sometimes I feel like I spend too much time doing that. I’ve been in a
position where I’ve had the luxury to say, “Gee, I have time to go back and
pretty this up a little bit.” And places where I’ve been able to write for a
publication, you spend more time doing that than you would if it was just
for your own professional growth.
But I don’t think of that as art. I think craft is really the right word for it.
You can make a chair, and it’s good looking, but it’s mostly functional—it’s a
chair.
Seibel: How do you recognize a great programmer, particularly when
you’re hiring? You guys have hired a lot of programmers and you obviously
try to hire really good programmers. How do you do it?
Norvig: We still don’t know.
Seibel: Google is somewhat famous for asking puzzle questions in
interviews. Do you think that’s a good approach?
Norvig: I don’t think it’s important whether people can solve the puzzles
or not. I don’t like the trick puzzle questions. I think it’s important to put
them in a technical situation and not just chitchat and get a feeling if they’re
a nice guy. Though it is important to have someone that you can get along
with. But you really have to see, can they technically do what they said they
can do. And there are a lot of different ways to demonstrate that. And
many times you can see it from the résumé. I think our best signal is if
somebody has worked with one of our employees before and the employee
can vouch for them. But we still try to draw it out on-site during the
interview. It’s more you want to get a feeling for how this person thinks and
how they work together, so do they know the basic ideas? Can they say,
“Well, in order to solve this, I need to know A, B, and C,” and they start
putting it together. And I think you can demonstrate that while still failing on
a puzzle. You can say, “Well, here’s how I attack this puzzle. Well, I first
think about this. Then I do that. Then I do that, but geez, here’s this part I
don’t quite understand.” For some people that little part clicks and for some
it doesn’t. And you can do fine if it doesn’t click as long as you’ve
demonstrated the basic competency and fluency in how you think about it.
And then you really want to have people write code on the board if you’re
Download at Boykma.Com
Peter Norvig
321
interviewing them for a coding job. Because some people have forgotten or
didn’t quite know and you could see that pretty quickly.
Seibel: So is that just a negative indicator? If they can’t write reasonable
code, that’s a bad sign. But if they don’t stumble over that hurdle, it’s hard
to tell whether they’re actually going to write really good code in a larger
context.
Norvig: Right. You could tell to a certain degree but at other levels you
can’t. And we’ve studied this pretty carefully because we’ve gotten a lot of
applications and we look at it at two levels. One, we say, from all the
résumés we get, are we interviewing the right set of people? And then, from
the interviews we get, are we hiring the right set of people?
Seibel: So how do you measure that? You don’t know about the one you
didn’t talk to or didn’t hire.
Norvig: Yeah, so that’s hard. At both levels, you’ve only got half your
sample, so it’s this biased problem, but I guess basically what we’re doing is
saying, “Of the people that we interviewed and did well, what did their
résumé look like,” and try to find more of those. Is having so many years of
experience important? Is working on an open-source programming project
important? How does that compare to winning a programming contest?
Seibel: Do you really take all these things and shove them into a database?
Norvig: Yes, we do, and when we’re doing the hiring, we have these scores
that come up that say, “The résumé predictor says such and such, and the
interview predictor says such and such.” We don’t take them as gospel, but
they’re just another piece of input along with all the other feedback we
have.
Seibel: Do the people who are doing the interviews have those numbers
beforehand?
Norvig: No, we only see that when they’re in the hiring committee, once
we’ve gathered all the feedback. One of the interesting things we found,
when trying to predict how well somebody we’ve hired is going to perform
when we evaluate them a year or two later, is one of the best indicators of
Download at Boykma.Com
Peter Norvig
322
success within the company was getting the worst possible score on one of
your interviews. We rank people from one to four, and if you got a one on
one of your interviews, that was a really good indicator of success.
Seibel: But you had to do well enough on something else that you actually
got hired?
Norvig: Right, so that’s the thing. Ninety-nine percent of the people who
got a one in one of their interviews we didn’t hire. But the rest of them, in
order for us to hire them somebody else had to be so passionate that they
pounded on the table and said, “I have to hire this person because I see
something in him that’s so great, and this guy who thought he was no good
is wrong, and I’ve got to stand up for him and put my reputation on the
line.”
Seibel: So you’re surrounded by top-notch programmers here at Google.
Given how pervasive computers and software are in our society, do you
think everybody needs to understand a bit about programming just to get
along in or understand the world they live in?
Norvig: You probably want an educated person to understand how
software is made to the same degree they understand how a car is made.
The other interesting thing is, how much does an informed citizen have to
be a programmer? Certainly the average person now can do word
processing and many of them can do spreadsheets, and so if you’re a little
bit experienced with spreadsheets, you’re starting to be a programmer.
Lots of attempts at end-user programming, and programming for everyone,
haven’t been very successful. I don’t know how easy it is. Is there a way of
thinking that people have that we’ve gotten all the people that it’s easy to
teach, and the other ones are going to be really hard, or is it just that we’ve
missed the model and there’s some simple model in place of many people
that could influence programming if we created it?
Seibel: A lot of people that I have talked to for this book, and elsewhere,
got into computers both because they just enjoyed it and because they felt
like it would change the world. Some of the folks I talked to for this book
did that in the past and now are depressed by how little they feel the world
has changed as a result. How do you feel about that?
Download at Boykma.Com
Peter Norvig
323
Norvig: Well, I’m in the right place. We have hundreds of millions of users
and we can make a difference for them, and we can launch new services for
them quickly. I think that’s great. I can’t imagine anything else I could be
doing to have that level of impact.
Download at Boykma.Com
Download at Boykma.Com