Potter T.D., Colman B.R. (co-chief editors). The handbook of weather, climate, and water: dynamics, climate physical meteorology, weather systems, and measurements
Подождите немного. Документ загружается.
there has been real improvement in most aspects of weather forecasting as a result of
model outputs. However, it will be some time, if ever, before models will contain the
intelligence exhibited by local meteorologists (or local farmers) in local shor t-term
forecasts.
When assumptions go beyond the data in range, it is a guess. If anemometers are
calibrated to 50 m=s and report speeds of 70 m=s, it is impossible to know the
uncertainty of the measurem ent (unless some postanalysis is conducted). Extrapola-
tion beyond experience is often the only course available, but a footnote is required
to warn the user of that fact. Extrapolation between measurement points may not
agree with new measurements specifically located to test the extrapolation. The
measurement deserves the presumption of accuracy. There are instrument tests
that can confirm the instrument performance. The smoothed model value should
not be presumed to be correct.
It is true that caveats may adversely affect the literary quality of pronouncements.
Perhaps there should be a sharp division between the statements of ‘‘scientists’’ on
political issues and scientists reporting the results of their fundamental research. The
public is not capable of sorting the ‘‘results’’ from a reputable scientist that are
biased for advocacy from the results from a reputable scientist that contain all the
dull but critical caveats that qualify the data supporting the results. It used to be that
ethics took care of the problem. If ethics no longer apply, a method needs to be
found to warn the public of the advocacy role of ‘‘political’’ scientists.
All That Is Labeled Data Is Not Gold
Most of us are quite comfortable with the idea of evolution in the animal kingdom or in
climate, by which small departures from the norm can spread and eventually change the
entire picture. We are not as ready to accept a similar evolution in classical data sets, the
gold standard of climatology.
Recently, I have been using a small subset of the data generated by the GEOSECS
(Geochemical Ocean Sections) program (1973–1974), in particular, the ocean profiles
of oxygen-18 and deuterium measured by Harmon Craig at Scripps Institution of
Oceanography in La Jolla, Calif. Until recently, I was secure in the knowledge that the
data I was using, gathered from the National Climate Data Center repository, were
complete and unadulterated. Earlier this year, while presenting my results and using
these data for comparison, a member of the audience asked why I was not using the
complete data set. Confused, I defended myself vigorously. Later, it appeared that
although I had correctly presented the published data, there was an underground
version of unpublished data that was known only to a small circle of initiates. My hosts
were kind enough to include me among the cognoscenti and it was at this point that
things started to become interesting.
Examining the new version, I started to find many inconsistencies in the two data sets:
values different in one than the other, stations appearing in one or the other but not
both, different measurement depths, and even different positions for the stations.
Puzzled, I went further afield in search of information that could resolve the conflicts.
696
CHALLENGES OF MEASUREMENTS
Looking at the published tables, I worryingly found that the mistakes in station
positions seemed to be due to typographical errors.
Soon, though, I found someone else who had a subset of the unpublished data. This
surely would clear things up. Unfortunately, it made things even worse. Where this data
set purported to give the same data, it was different from both previous versions in
seemingly random ways.
In despair, I tried to track down the source of the unpublished data. Finally, I found
someone who had an old photocopy of an old-style computer printout whose
provenance was claimed to be Harmon Craig’s original measurements (including
repeats). At last, truth!
It became clear very quickly that this printout was the original version of the
unpublished data, and it was equally clear that the data had at some point been
typed in by hand. The errors were of the sort caused by inadvertent line skipping,
missing decimal points and minus signs, and incorrect averaging of repeat measure-
ments. However, it was also evident that this was the source for the published tables.
The published data though, had been subjected to some quality control; outliers had
been thrown out, repeat samples with too much dispersion ignored, and possibly,
further repeats had been performed. The measurement depths had clearly been refined
using more accurate information.
The origin of the third data set remained mysterious until I was informed that it had
been traced from a gra ph showing the data in an old article and then correlated to the
original stations. That explained the small but random differences. Only with all this
scientific detective work and the discovery of the ancestral data was I now able to
amalgamate the published and unpublished results into a consistent data set.
These data are only 25 or so years old and yet there are at least three (maybe more)
different and mutually inconsistent versions floating around. The lesson we should take
from this is that, if unchecked, small mutations will occur during transcriptions. Unless
we are vigilant, ‘‘data sets’’ will evolve and data that have been so painstakingly
collected and saved will be distorted and twisted beyond all usefulness. Our care in
using data should be a force analogous to natural selection, keeping data sets fit for the
purposes intended. If we do not take care, the gold standards of climatology may in
time turn to lead.
Gavin Schmidt, NASA Goddard Institute for Space Studies, New York; E-mail:
gschmidt@giss.nasa.gov.
2 GENERATION AND IMPLEMENTATION OF A CONSENSUS ON
STANDARDS FOR SURFACE WIND OBSERVATIONS
Generation
Robert L. Carnahan, the retired federal coordinator from the Office of the Federal
Coordinator for Meteorology (OFCM), contacted the author in 1992 regarding a
wind measurement question. Contact was arranged with David Rodenhuis, chairman
2 GENERATION AND IMPLEMENTATION OF A CONSENSUS ON STANDARDS 697
of the Ad Hoc Group and the Interdepartmental Committee for Meteorological
Services and Supporting Research (ICMSSR) on wind observing standards. The
author was asked to assist Rodenhuis in organizing a workshop and presiding at
the workshop; a plan was devised.
The purpose of the workshop was to seek a consensus standard method of
characterizing surface wind measurements that would serve all applications of
wind measurement. Two preparator y tasks were essential. First, a g roup of knowl-
edgeable attendees to represent all applications had to be identified. Second, a
syllabus needed to be prepared to set the background for the two-day workshop.
A list of 79 experts was assembled, and, of those, 40 attended; the 39 who could
not attend were sent all the materials and reports and had the opportunity to pa rti-
cipate by mail. The group included specialists from such applications as agriculture,
aviation, climatology, forecasting and warnings—National Weather Service (NWS),
manufacturers of instruments and systems, measurements, military, oceanography,
buoy operation, and hurricane research, insurance, transport and diffusion, air qual-
ity, network operations, and quality control, and wind energy. The names, affilia-
tions, and specialties of all participants are available from the OFCM.
The two-day workshop produced a group consensus, of which the most important
elements are listed below.
The metadata (information about the measurement system and the siting of
the sensors) must be available where data are archived. Included will be the
following: station name and identification number, station location in longitude
and latitude or equivalent, sensor type, first day of continuous operation, sensor
height, surface roughness analysis by sector and date, site photographs with
5-year updates, tower size and distance of sensor s from centerline, size and
bearing to nearby obstructions to flow, measurement system description with
model and serial numbers, date and results of calibrations and audits, date
and description of repairs and upgrades, data flowchart with sampling rates
and averaging methods, statement of exceptions to standard requirements, and
software documentation of all generated statistics.
The basic period of time for surface wind observations should be 10 min. This
period provides reasonable time resolution for continuous process analysis (air
quality, wind energy, and research) while providing the building blocks to
assemble longer averaging periods. Many current boundary layer models
require hourly data. However, there is a growing world consensus for
10-min data periods. The periods must be synchronous with the Universal
Time Clock (UTC) and labeled with the ending time. If a different time label is
used, it must be stated and attached to the data.
The operating range is either sensitive or ruggedized. The threshold and
maximum speed of the sensitive range is 0.5 and 50 m=s. The threshold and
maximum speed of the ruggedized range is 1.0 and 90 m=s.
The dynamic-response characteristics for either type are as follows. The
anemometer will have a distance constant of less than 5 m. The wind vane
698 CHALLENGES OF MEASUREMENTS
will have a damping ratio greater than 0.3 and a damped natural wavelength of
less than 10 m.
The measurement uncertainty for wind speed is 0.5 m=s below 10 m=s and
5% of the reading above 10 m=s. For wind direction, the accuracy to true north
is 5
.
The measurement resolution is 0.1 m=s for both average speed and the standard
deviation of the wind speed. Resolution for average direction is 1
and for the
standard deviation of wind direc tion is 0.1
.
The sampling rate will be from 1 to 3 s. Sampling for wind speed must be
capable of achieving a credible 3-s average.
The standard data output for archives will include the following:
Ten-minute scalar-averaged wind speed
Ten-minute unit vector or scalar-averaged wind direc tion
Fastest 3-s gust during the 10-min period
Time of the fastest 3-s gust
Fastest 1-min scalar-averaged wind speed during the period
Average wind direction for the fastest 1-min speed
Standard deviation of the wind speed samples about the 10-min mean speed
Standard deviation of the wind direction samples about the 10-min mean
direction.
Special guidance is offered for survivability to assure the support and power
will not fail before the sensor its elf fail s. A suggestion is offered for preserving
high-resolution data when the wind is above 20 m=s.
Quality assurance is required with documentation in the site log.
A specific method, based on Wieringa (1992) , was described.
The consensus was reached with caveats from some mem bers, generally prefer-
ring tighter requirements. However, this method was designed as a minimum that all
applications could use. Wind energy, for example, needs greater accuracy. There is
no reason why special networks or stations could not meet this specification and still
provide greater accuracy. Uncertainty is largely a result of calibration. The hope of
all the attendees was that the NWS would adopt the consensus and implement it in
the new ASOS (Automated Surface Observing System) deployment. Implementation
of new technology provides opportunities to improve the value of measurements.
Digital data systems can preprocess samples. There is value in the potential richness
of data that this consensus could provide as compared to manual observations.
Technology has also provided capability to archive the richer data. The National
Climatic Data Center (NCDC) representative joined the consensus and agreed that
the data could be archived. All that remained was implementation.
2 GENERATION AND IMPLEMENTATION OF A CONSENSUS ON STANDARDS 699
Implementation
There was immediate implementation by some members of the workshop. The
Oklahoma Mesonet used a version of the consensus in its new (at the time) statewide
network of automatic stations. Campbell Scientific Inc. designed an instr uction for
its data loggers that would generate the required outputs each 10 min. The American
Society of Civil Engineers (ASCE) adopted the 3-s average speed at 10 m as its basic
wind speed (since the fastest mile has been discontinued). It has been published in its
Minimum Design Loads for Buildings and Other Structures ANSI=ASCE 7-95.
Since there now was no expectation that the consensus would be used by the
NWS, its essence was introduced to the D22.11, Meteorology, subcommittee of the
American Society for Testing and Materials (ASTM). After the normal open process
of consideration, modification, and balloting, the Standard Practice for Characteriz-
ing Surface Wind Using a Wind Vane and Rotating Anemometer D 5741-96 was
adopted. As of 2001, this standard had not been adopted by the OFCM.
One element of the consensus is the definition of peak speed or gust. The ASOS
provides a peak 5-s average. This clock-driven averaging period provides a smaller
speed than the F420 anemometer attached to a dial or galvanometer recorder has
provided during the past several decades. This human observation was considered
‘‘instantaneous’’ since there was no intentional averaging between the generator in
the sensor and the dial or recorder in the office. The dial had to be seen at the
moment of maximum value. The strip chart record s the speed, but the galvanometer
does have a frequency response. The damping by the recorder has been estimated as
approximately equivalent with a 1- to 2-s average.
It became clear that, from a climate continuity standpoint, the change from the
conventional peak speed to the ASOS peak speed decreased the value reported. This
could have ramifications for storm or wind warnings. It is also clear that changing
from a clock 5-s average to a running 3-s average would move the peak speed value
closer to the conventional meas urements. The most compelling reason for adopting
the running 3-s average for peak speed is the fact that, without standards, the concept
of peak speed has no meaning.
In the past, technology was such that whatever one could get was acceptable. Any
measurement was better than no measurement. The totalizing anemometer had to be
read where the anemometer was mounted. Gears turned a scale not unlike an electric
meter. An interim step was the fastest-mile anemometer, which could drive an event
pen on a strip chart in the office, thereby recording the time of each mile (so many
revolutions of the cup) of air that passed. Time per mile could be converted to miles
per hour by using an overlay with different line spacings labeled as speed or by
counting the miles that passed in an hour for an hourly average.
With the advent of the cup anemometer driving an electric generator, the
measurement could be observed in real time in the office, first on a dial and then
on a dial and a chart recorder. When automatic weather stations came into use, the
questions of digital sampling and averaging had to be answered. The Federal Avia-
tion Administration (FAA) funded the Nationa l Oceanic and Atmospheric Admin-
istration (NOAA) and the NWS to make recommendations appropriate for aircr aft
700 CHALLENGES OF MEASUREMENTS
operations for the ASOS design. The 5-s average for peak speed came from this
study. Many other countries also considered the same questions. In the World
Meteorological Organization (WMO), the Commission for Instruments and Methods
of Observation (CIMO) adopted the 3-s average as the standard for peak wind speed.
It is hoped that our national network of surface weather stations will eventually
adopt the consensus standard. In the interim, a campaign is underway to change the
peak speed from the clock 5-s average to a running 3-s average on the basis of
climate continuity. The American Association of State Climatologists (AASC) voted
in their August, 1998, meeting to adopt the running 3-s average for climatological
purposes.
3 UNCERTAINTY, ACCURACY, PRECISION, AND BIAS
A critical component of a chapter of this kind is a careful consideration of the
meaning of the language used. There are two levels of language in this situation.
One is the precise and unambiguous words used by the professiona ls in the standards
field. The other is the routine communi cation used in the application of technology
in a regulated society. A convergence of these two is the goal of this chapter.
Accuracy is a word commonly used in the specification of regulations, procure-
ment documents, and manufacturer’s data sheets. According to ISO (1993), measure-
ment accuracy is simply the closeness of the agreement between the result of a
measurement and a true value of the measurand. Accuracy is therefore a qualitative
concept since the true value must remain indeterminate.
The concept of uncertainty, as carefully defined by standards professionals,
provides a method to quantitatively characterize all doubt about the validity of the
value of a measurement. There are two types of evaluation methods. Type A is an
evaluation of a statistical analysis of a series of observations of the measurand. Type
B is an evaluation of other information about the measurement. These two types of
evaluations provide what is called the ‘‘combined standard uncertainty.’’
This chapter includes an introduction to these concepts by listing a number of
definitions found in the 1993 report by the International Organization for Standar-
dization (ISO). Points that always need reinforcement are also discussed. These
include the practices of rounding and the display of significant figures. All of
these subjects will be discussed with examples drawn from the measurement of
air temperature at 1.5 m above the ground surface.
Background
An effort to find an international consensus on the expression of uncertainty in
measurements began in 1978. The world’s highest authority in metrology (mea sure-
ment science, not atmospheric science), the Comite´ International des Poids et
Mesures (CIPM) asked the Bureau International des Poids et Mesures (BIPM) to
make recommendatio ns. After consulting the national standards laboratories of 21
countries, a working group was established. This working g roup assigned the
3 UNCERTAINTY, ACCURACY, PRECISION, AND BIAS 701
responsibility to the ISO Technical Advisory Group on Metrology (TAG 4). It, in
turn, established Working Group 3 (ISO=TAG 4=WG 3) composed of experts nomi-
nated by BIPM IEC (International Electrotechnical Commission), ISO, OIML
(Internati onal Organization of Legal Metr ology), and appoin ted by the chairman
of TAG 4.
Working Group 3 was assigned the following terms of reference:
To develop a guidance document based upon the recommendations of the BIPM
Working Group on the Statement of Uncertainties which provide rules on the
expression of measurement uncertainty for use within standardization, calibration,
laboratory accreditation, and metrology services:
The purpose of such guidance is
to promote full information on how uncertainty statements are arrived at;
to provide a basis for the international comparison of measurement results.
ISO (1993) is the current guide from WG 3.
Definitions
The first paragraph of the introduction to the Guide (ISO, 1993) is reproduced to
explain the purpose of the Guide.
When reporting the result of a measurement of a physical quantity, it is obligatory that
some quantitative indication of the quality of the result be given so that those who use it
can assess its reliability. Without such an indication, measurement results cannot be
compared, either among themselves or with reference values given in a specification or
standard. It is therefore necessary that there be readily implemented, easily understood,
and generally accepted procedures for characterizing the quality of a result of a
measurement, that is, for evaluating and expressing its uncertainty.
In the scope of the Guide (ISO, 1993) is the following:
This document, hereafter called the Guide, establishes general rules for evaluating and
expressing uncertainty in physical measurement that can be followed at various levels
of accuracy and in many fields—from the shop floor to fundamental research. There-
fore, the principles of this Guide are intended to be applicable to a broad spectrum of
measurements, including those required for:
maintaining quality control and quality assurance in production;
complying with and enforcing laws and regulations;
conducting basic research, and applied research and development, in science and
engineering;
calibrating standards and instruments and performing tests throughout a national
measurement system in order to achieve traceability to national standards; and
702
CHALLENGES OF MEASUREMENTS
developing, maintaining, and comparing international and national physical reference
standards, including reference materials.
Annex B of the Guide (ISO, 1993) contains definitions of general metrological
terms that will be repeated here, with air temperature examples where appropriate.
B.1 (measurable) quantity
attribute of a phenomenon, body or substance that may be distinguished qualitatively
and determined quantitatively
EXAMPLE: the motion of air molecules interpreted as temperature.
B.2 value (of a quantity)
magnitude of a specific quantity generally expressed as a unit of measurement
multiplied by a number
EXAMPLE: the temperature of the boiling point of water (at 1 atmosphere) is
100
C.
B.3 true value (of a quantity)
value perfectly consistent with the definition of a given specific quantity
EXAMPLE: the theoretical absolute zero of 0 Kelvins.
B.4 conventional true value (of a quantity)
EXAMPLE: the temperature of air when there is no molecular motion is
273.15 0.01
C.
B.5 measurement
set of operations having the object of determining a value of a quantity
EXAMPLE: the function of a temperature measurement system or an observer
with a ther mometer taking a reading.
B.6 principle of measurement
scientific basis of a method of measurement
EXAMPLE: the thermoelectric effect applied to the measurement of temperature.
B.7 method of measurement
logical sequence of operations, in generic terms, used in the performance of measure-
ments according to a given principle
EXAMPLE: the resistance of a wire, exposed in an aspirated radiation shield,
determined by a substitution method.
B.8 measurement procedure
set of operations, in specific terms, used in the performance of particular measurements
according to a given method
EXAMPLE: a standard operating procedure used by an operator to obtain a
measurement or manage an automatic measurement system.
B.9 measurement process
all the information, equipment and operations relevant to a given measurement.
(NOTE—This concept embraces all aspects relating to the performance and
quality of the measurement; it includes for example the principle, method,
procedure, values of the influence quantities, and the measurement standards.)
3 UNCERTAINTY, ACCURACY, PRECISION, AND BIAS 703
B.10 measurand
specific quantity subject to measurement
EXAMPLE: The temperature of the air at a specific location, often called a
variable.
B.11 influence quantity
quantity that is not included in the specification of the measurand but that nonetheless
affects the result of the measurement
EXAMPLE: the effect of solar or long-wave radiation, wind speed, water (in any
phase), insects, environmental influence on measurement circuits, and the condi-
tion of any moving parts and connections.
B.12 result of a measurement
value attributed to a measurand, obtained by measurement
(NOTES—1. When the term ‘‘result of a measurement’’ is used, it should be made
clear whether it refers to: the indication; the uncorrected result; the corrected
result; and whether several values are averaged. 2. A complete statement of
the result of a measurement includes information about the uncertainty of
measurement.)
B.13 uncorrected result
result of a measurement before correction for the assumed systematic error
B.14 corrected result
result of a measurement after correction for the assumed systematic error
B.15 accuracy of measurement
closeness of the agreement between the result of a measurement and a true value of the
measurand
(NOTES—1. ‘‘Accuracy’’ is a qualitative concept. 2. The term ‘‘precision’’ should
not be used for ‘‘accuracy.’’)
B.16 repeatability (of results of measurement)
closeness of the agreement between the results of successive measurements of the same
measurand carried out subject to all the following conditions:
the same measurement procedure
the same observer
the same measuring instrument, used under the same conditions
the same location
repetition over a short period of time
B.17 reproducibility (of results of measurement)
closeness of the agreement between the results of successive measurements of the same
measurand, where the measurements are carried out under changed conditions such as:
principle or method of measurement
observer
measuring instrument
location
conditions of use
time
704
CHALLENGES OF MEASUREMENTS
B.18 uncertainty (of measurement)
a parameter, associated with the result of a measurement, that characterizes the
dispersion of the values that could reasonably be attributed to the measurand
(NOTES—1. The parameter may be, for example, a standard deviation (or a given
multiple of it), or the half-width of an interval having a stated level of confidence.
2. Uncertainty of measurement comprises, in general, many components. Some of
these components may be evaluated from the statistical distribution of the results
of series of measurements and can be characterized by experimental standard
deviations. The other components, which can also be characterized by standard
deviations, are evaluated from assumed probability distributions based on experi-
ence or other information. 3. It is understood that all components of uncertainty,
including those arising from systematic effects, such as components associated
with corrections and reference standards, contribute to the dispersion.)
B.19 error (of measurement)
result of a measurement minus a true value of the measurand
B.20 relative error (of measurement)
error of measurement divided by a true value of the measurand
B.21 random error
result of a measurement minus the mean result of a large number of repeated
measurement of the same measurand
B.22 systematic error
mean result of a large number of repeated measurements of the same measurand minus
a true value of the measurand.
B.23 correction
value that, added algebraically to the uncorrected result of a measurement, compensates
for an assumed systematic er ror
B.24 correction factor
numerical factor by which the uncorrected result of a measurement is multiplied to
compensate for an assumed systematic error
The annex to the Guide (ISO, 1993) provides additional examples and comments.
Annex C (not discussed here) provides the basic statistical terms and concepts for
making a type A evaluation of uncertainty.
Discussion
The traditional use of the term accuracy, among applied meteorologists, is under-
stood to include both a (const ant) bias term and a (variable) precision term. These
can only be estimated from comparisons with other measurements, calibration tests
at a laboratory with accepted authority, or reliance on the manufacturer’s data sheet
claims. These estimations do NOT include comparisons with the true value nor do
they incl ude the influence quantity.
Using the 1.5-m air temperature example, the bias term might be þ0.2
C, based
on either a calibration made by the National Institute of Standards and Technology
3 UNCERTAINTY, ACCURACY, PRECISION, AND BIAS 705