Tabak J. Mathematics and the Laws of Nature: Developing the Language of Science

Подождите немного. Документ загружается.

190 MATHEMATICS AND THE LAWS OF NATURE

possible forecasts for ECMWF clients. She has published a number of

papers in the peer-reviewed literature on the diagnosis, post-processing,

and application of ensemble prediction systems. In conjunction with her

colleagues, she was awarded the Norbert Gerbier-Mumm International

Award 2006 by the World Meteorological Organization. The following

interview took place December 10, 2009.

J. T. Please tell me a little about your work at the Centre.

R. H. I started my work at ECMWF roughly nine years ago.

When I first joined ECMWF I worked on the DEMETER proj-

ect. That was a project on seasonal forecasting, and we developed

a so-called multi-model ensemble. That means we took seven

different models from different European weather and climate

research centers and combined them into a multi-model ensemble

prediction system. The aim of the project was to investigate

whether a multi-model forecast improved on the single model

forecasts—whether, basically, the multi-model can beat the best of

the single models.

J. T. When you say “multi-model” . . . were the equations in

the models different or did the models use the same equations and

different initial conditions?

R. H. The models used the same initial conditions but we had

different numerical modeling systems from different centers. Each

of these centers actually ran an ensemble of forecasts with their

numerical model system. That means they integrated (solved)

their models not only once, but each center performed nine

slightly perturbed model runs, and then in the end we combined

the seven times nine integrations (solutions) to get 63 ensemble

members. These 63 realizations (or ensemble elements) basically

represent the uncertainty of the prediction. This particular aspect

of numerical weather prediction—how to represent uncertain-

ties—is what I am mainly working on.

In general, the work at ECMWF has two aspects, the opera-

tional and the research aspects. On the one hand, ECMWF pro-

Afterword 191

duces operationally every day weather forecasts and disseminates

the results to its member states and users; on the other hand, the

researchers work constantly on trying to improve our model sys-

tems. When we talk about “improving the model” there are again

many aspects involved. First of all, we can work on improving the

numerical aspect of the model. That means how the dynamical

processes are set up in the numerics of the model, and how these

equations are solved. The second aspect is about the physics. That

means we have to parameterize certain physical processes of the

atmosphere. Because in the model, as you probably know, we

have to divide the atmosphere into grid boxes of a certain size,

and basically all the physical processes that are on a scale smaller

than the grid boxes must be parameterized. A whole part of our

research work is to try to improve these parameterizations. The

third aspect in the research department is the data assimilation

system—that is, we have to try to create the best initial conditions

for our model. Because the whole idea of numerical weather pre-

diction is that first of all you need the model, of course, you need

to describe the processes of the atmosphere, but then to start the

model you need the best initial conditions. The better the initial

conditions are—they are the starting point of your model—the

better your forecast also becomes. The initial conditions, also

called “the analysis,” are prepared with our data assimilation

system. In this system we collect all available observations from

satellites and other sources—like direct measurements at land

or sea—and combine these observations into our model system

to produce a dynamically balanced analysis. The fourth part of

the research at ECMWF is related to our Ensemble Prediction

System (EPS). The EPS has been developed because no matter

how good the analysis is that we produce, there will always be

a remaining uncertainty in the initial conditions and that means

that our forecasts will—to a certain degree—always be uncertain.

In the ensemble prediction section we work on how we can best

represent and predict this uncertainty.

Personally, my work in the EPS section is mainly concerned with

diagnostics and verification work—that is, trying to find deficien-

cies in the model. In order to correct for such deficiencies we not

192 MATHEMATICS AND THE LAWS OF NATURE

only try to improve the model directly by, for example, improving

the model formulation per se, but we also apply a so-called post-

processing. The principle of post-processing—or model output

statistic—is that we compare forecasts and observation from the

past in order to find any systematic model errors or deficiencies. If

we can find such systematic errors in the past we can correct these

errors in real forecasts of the future. That is my personal work

here—on diagnostics, verification, and post-processing.

J. T. First, could you explain what you said earlier. You men-

tioned that you have a certain grid size and when you look at

phenomena that are smaller than the grid size you “parameterize”

them. Could you explain exactly what that means?

R. H. Let’s assume that in our high-resolution model, we have

a horizontal grid size of 16 times 16 (or 256) square kilometers.

When you look into the atmosphere, you can see that there are

clouds, and these clouds are not 256 square kilometers big. They

can be much smaller, and in particular the processes inside the

clouds like evaporation and condensation or the formation of ice

crystals are very small-scale processes. They are happening on the

millimeter scale and below. So in the model we somehow have to

find a way to calculate the average effect of all these small scale

processes together in one grid box. In other words, we have to

find a relationship between the average conditions in a relatively

big grid box and sub-gridscale processes. For example, depending

on the average vertical wind speed and (water vapor) saturation

level certain amounts of cloud crystals or precipitation can form

in the grid box . . . so we represent processes that happen on a

much smaller scale by average conditions in the much bigger grid

box.

J. T. So each grid box might correspond to a pixel. Do I have

this right? You take each of these properties and average them out

over a grid box, and then assign that one value to that grid box?

R. H. Yes.

Afterword 193

J. T. When you do this averaging what sort of forecast do you get?

R. H. That depends on what quantity you look at, but you ask

exactly the correct question. When we want to judge a forecast,

then we usually compare a forecast with some sort of observation.

For example, when I look at precipitation, the rain gauge collects

the precipitation at a certain site. But one kilometer away there

could be a very different amount of precipitation, because precipita-

tion can be a quite small-scale process. Or even temperature, when

we have, for example, a station close to the sea we can experience

land-sea effects, or we can have observations influenced by other

complex orographic conditions. Such stations can have very par-

ticular circumstances—that is, the atmospheric conditions can vary

quite a lot at short distances. However, the model forecast gives the

average conditions in the 16 by 16 square kilometer grid box. And

that is not necessarily always directly comparable to what is mea-

sured at a particular station. To overcome that, there are two ways.

One way is that we do not compare our forecasts directly with

station observations, but we compare it with the so-called analysis

field. Our analysis represents all of the observations at the differ-

ent stations and brings it into our model grid. That is one way

that we compare—not to the real observations but to the analysis

values, which represent the same scales as the model.

On the other hand, one can go the other way around. One

can say, “OK, I want to correct my forecast, which only repre-

sents the average conditions in the grid box, to represent the

local conditions at the observation site.” This process is called

down-scaling. We distinguish between dynamical and statistical

downscaling. Statistical downscaling, which is closely related to

the post- processing mentioned earlier, is based on error statistics

of so-called training data sets. Basically, we compare forecasts and

station observations from the past in order to find out the system-

atic errors of the model at this particular station. Then we can

correct for the particular characteristics for this station.

J. T. You’re using training data? That must mean that you are

using some sort of machine learning—what is the right terminology?

194 MATHEMATICS AND THE LAWS OF NATURE

R. H. People always think of neural networks when the term

training data is mentioned.

J. T. Yes.

R. H. We do not necessarily use this particular technique, but

in a way you can compare it. There are different methods how to

learn from training data. In our case the learning process consists

of an iterative process to find the optimal correction for our model

forecasts at a certain location and for a certain lead time. Let’s say,

the model forecasts 20 degrees, but we know from our training

dataset that when the model forecasts 20 degrees it should be two

degrees less (at a particular site). Then there could be other situ-

ations—for example, when it is very cold—then the model may

always predict too high temperatures. So it is a learning process

how best to adjust for such systematic deficiencies. With this

training we try to learn or find out the errors of the model that are

in some way systematic.

J. T. So much analysis! For how large an area are you forecasting?

R. H. We have a global model, that means our predictions cover

the whole globe—

J. T. The whole globe with a 16 by 16 km grid size and so much

analysis! I’m surprised that you can do this in real time.

R. H. This has developed over time. When ECMWF started

30 years ago or so, we had a resolution of 200 by 200 (or 40,000)

square kilometers for each grid box. But with increasing computer

power and increasing efficiency of our numerical methods this is

now possible. I have to say that at the moment the highest resolu-

tion we run is actually 25 by 25 (or 625) square kilometers, but on

our next upgrade, which is planned for early next year, we will be

able to run our high-resolution model at 16 by 16 (or 256) square

kilometers. This deterministic model system is run twice per day,

one forecast started at 00:00 UTC and one at 12:00 UTC.

Afterword 195

If we go to our ensemble prediction system, we actually cannot

afford to run the model with such high resolution because in that

case we run the model 50 times instead of only once per start-

date. So we have two model systems. One is the deterministic

high-resolution system, in which we have the smallest possible

grid boxes to most accurately represent all the particular charac-

teristics of the complex orography and related physical processes.

In the second system, which is the ensemble prediction system,

we run 50 forecasts. In fact we perform 51 model integrations

since we do one forecast—the so-called control forecast—start-

ing from our best-guess initial conditions (the analysis) plus 50

integrations (solutions) from the “perturbed” or slightly altered

analysis. But since these are 51 forecasts, we cannot afford to run

the model on the high resolution, so for the ensemble we always

have a lower resolution than in our high resolution model, At

the moment the EPS resolution is 50 by 50 (or 2,500) square

kilometers.

J. T. To start with your deterministic model . . . How do you

evaluate the accuracy of your predictions? How, for example, can

you tell if your model is improving over time?

R. H. There are two different aspects. One aspect is that you

want to get the general circulation pattern right—you really want

to know where the low pressure systems are and where the high

pressure systems are. To assess our success in getting these more

large scale dynamical features right we compare our forecasts

with our analysis, in particular variables which describe the atmo-

spheric flow. In addition to predicting the general atmospheric

patterns, we also need to assess how well we predict the resulting

weather parameters like surface temperature, wind, or precipita-

tion because these are the parameters which are of real interest to

our users.

J. T. So you compute the (statistical) variance between your

observations and predictions to determine how well you have

done?

196 MATHEMATICS AND THE LAWS OF NATURE

R. H. Yes, we calculate how well our forecasts match our analysis

of the atmosphere. This is the deterministic aspect of our forecast

assessment, to study how close or far apart our deterministic fore-

cast is from the analysis or the observations.

If we come to the probabilistic forecasting system then we have

additional aspects, which we want to get right. In the probabilistic

forecasting system, basically what you predict is a certain probabil-

ity for a certain event. Say what you want to forecast is the prob-

ability that the wind speed will be above 20 meters per second.

Then what we do is count the number of ensemble members that

predict a wind speed above 20 meters per second, and if we have,

for example, 5 out of 50 predicting this event then we forecast

a 10 percent probability that this event will occur. What is very

important for a probabilistic forecasting system is that these prob-

abilities are reliable.

So suppose you have 100 cases in which our EPS predicts a 10

percent probability for a certain event. You might think that this is

a very low probability and therefore the event should not happen

in these 100 cases. But this is actually not true, because in a reliable

forecast system, a 10 percent probability for an event really means

that in 10 percent of the cases the event occurs—that is, when we

verify our 100 cases, we really want that in exactly 10 out of these 100

cases the event really occurred. It should not occur in more cases, but

it also should not occur in fewer cases. This example demonstrates

why we cannot verify a single probabilistic forecast, because the reli-

ability of a probabilistic forecast is a statistical feature and can only be

assessed in a statistical context—that is, with a bigger sample of cases.

J. T. OK. So you are running a probabilistic forecast and a

deterministic forecast, and you are making these many predictions

. . . how accurate are your forecasts?

R. H. This depends on many factors of your forecast like, for

example, the area or variable you are looking at, and of course the

lead time your forecast has. We are very good in forecasting the

global or general flow of the atmosphere. So we are quite good at

forecasting the general synoptic situation, which is the global dis-

Afterword 197

tribution of high- and low-pressure systems. So the general flow

is accurately forecast.

J. T. For how long?

R. H. Well, this depends on your definition of accuracy, but we

can say that our forecasts of the general flow can be useful for up

to eight to ten days. The problem is really with more local phe-

nomena and surface weather parameters. They are much more

difficult to predict—

J. T. When you say that you are accurate for up to ten days,

are you referring to the deterministic forecast, the probabilistic

forecast or both?

R. H. We actually run our probabilistic forecasts out to 15 days.

With the added information on the forecast uncertainty it can give

useful information up to that forecast range. It is difficult to say

how good the forecast is in general terms. You have to relate it to

something. You have to compare it to the quality of a reference

system or how much your forecasts have improved over time. At

ECMWF we have been doing forecasts operationally for more

than 20 years, and we do constantly “score” our forecasts. That

means that we compare them to observations for the last 20 years.

We definitely see an improvement in the forecast skill over the

years. It is not that everyday we improve, but on average one can

see that the level of quality we achieved for a five-day forecast 10

years ago we can achieve today with a six-day forecasts. So basi-

cally we extended the range of how far into the future we can

do useful predictions by one day, or in other words, we have an

improvement of one day lead-time per decade.

J. T. Why do you think that uncertainty is so much a part of mete-

orology? What are the factors that contribute to this uncertainty?

R. H. The main reason for this uncertainty is the “chaotic

nature” of the atmosphere. We speak of a chaotic system when

198 MATHEMATICS AND THE LAWS OF NATURE

it can be described by a nonlinear system of differential equa-

tions. Ed Lorenz was one of the first to discover and describe the

importance of this for numerical weather prediction and coined

the famous phrase, the “butterfly effect”—

J. T. The idea that the flapping of a butterfly’s wings can change

the weather—

R. H. And the point is really that a nonlinear system of equations

is extremely sensitive to the initial conditions. That is, when you

start your forecast from only slightly changed initial conditions

then after a while your forecast can end up somewhere totally dif-

ferent. And the interesting thing is that it depends on the initial

conditions themselves, whether or not your forecast is more pre-

dictable or less predictable. In certain situations . . . not getting

the initial conditions quite right does not affect the forecast too

much. In such cases the “spread” of the forecast—the variation

of the 50 ensemble members—is very small. Then we have other

situations where the weather is much more unpredictable, and tiny

variations in the initial conditions can lead to great deviations in

the forecasts, and we have a very large spread.

That is the crucial point of the ensemble prediction methodol-

ogy: We basically try to predict the uncertainty. We try to predict

whether our forecast is predictable or unpredictable, and I think

that this is very important information for a user. Because, for

example, if you have a very important thing to do—maybe you

are a builder and you ordered a lorry full of concrete to lay the

foundation of a house, but if the temperatures fall below freez-

ing level the concrete won’t set and you’ll lose a lot of money.

In this case, it is very important for you to know the uncertainty

of the weather forecast. Depending on your personal risk, or

what potentially you could lose, you might not want to take the

risk because it might cost you a lot of money. That is, it is very

important to know what the probability of a particular event is—

for example, whether there might be even the slightest chance

for temperatures below freezing level. On the other hand, if the

forecaster says we are very certain that this event will not hap-

Afterword 199

pen, then you can say, “OK, now I risk it—my big money or my

big event.” This (degree of uncertainty) is really important to

know.

J. T. Sure.

R. H. So when you ask, “How do you determine the success of

your forecasting system?” On the one hand, of course, we always

compare numbers: These are the observations; this is the fore-

cast; and we want the smallest difference possible. But in a way

this is only a theoretical assessment, if you like, this is more for

the model developers or for academic reasons. The real proof of

how good—or better how useful—our forecasts are, are really the

customers, the feedback that we get from our customers. Do they

make better decisions when they use our forecasts?

J. T. To return to the idea of uncertainty once more . . . Your

conservation laws are expressed as nonlinear equations, and this is

one source of uncertainty. But there must also be some difficulty

in obtaining accurate initial conditions to formulate your initial

analysis. If you are going to say that you know the state of the

weather around the globe at some instant of time, how certain are

you that you actually know the global weather at that time?

R. H. Let’s put it this way: ECMWF forecasts have enormously

improved over the last 20 years. This success has much to do with

how we have been able to include more and more data into our

initial analysis. That means all of the satellite data and other avail-

able observations. So in a way it is very crucial to get the initial

conditions as accurate as possible, but exactly as you say, we can

never achieve total accuracy because our observations do not cover

constantly the whole globe; there are always data gaps; there are

observational errors; satellite data have to be properly calibrated

and also do not always measure exactly what we need at the time

when we need it. We cannot observe the whole of the atmosphere

with total accuracy. Yes, we have uncertainty in the initial condi-

tions. We have little errors, and they lead to forecast errors.