Steve M., Darby D.M., Geostatistics Explained - An Introductory Guide for Earth Scientists

Подождите немного. Документ загружается.

across the entire area being discussed (in this case the lake). A better design

would be to sample at several places chosen at random within the lake, as

shown in Figure 4.3(c).

Here is another example. A researcher sampled a peatland ecosystem by

dropping a 10 m

square frame, subdivided into a grid of 100 equal sized

squares, at random in one place only and then took one sample from each of

these smaller squares. Although these 100 replicates may very accurately

describe conditions within the sampling frame, they do not necessarily describe

the remaining 9990 m

of the wetland environment and would be pseudo-

replicates if the results were interpreted in this way. A more appropriate design

would be to sample 100 replicates chosen at random across the entire area.

4.2.4 The need to repeat the sampling on several occasions

and elsewhere

The results of sampling systems which are in considerable ﬂux (e.g. the

Holyoke estuary or the lake described above) can only conﬁdently be

discussed in relation to the area at the time of sampling. Therefore, you

100

19 *

103

(a)

(b)

(c)

Figure 4.3 Aerial view of a glacial lake showing mean sediment grain size (in

μm). (a) An unreplicated sample taken at only one place (*) would give a very

misleading indication of grain size within the entire lake. (b) Several replicates

taken close to one another (*****) would still give a very misleading indication.

indication.

4.2 Sampling: mensurative experiments 33

need to be cautious when interpreting results. Sampling the same area over

several diﬀerent years will strengthen the ﬁndings, and may be suﬃcient if

you are only interested in that area. Sampling more than one area

(e.g. estuary, lake or outcrop) will make the results more able to be general-

ized. Inappropriate generalization is another example of pseudoreplication

since data from one location (whether it is relatively unchanging or in a state

of ﬂux) may not hold in the more general case. At the same time, however,

even if your study is limited, you can still make more general predictions

from your ﬁndings provided these are clearly identiﬁed as predictions.

4.3 Manipulative experiments

4.3.1 Independent replicates

It is essential to have several independent replicates of any treatment used in

a manipulative experiment. We mentioned this brieﬂy when describing the

lead remediation experiment in Chapter 2 and said that for only one treated

and one untreated sample, any diﬀerence between them could have simply

been due to chance or some other unknown factor(s). As the number of

randomly chosen independent replicates increases, so does the likelihood

that any diﬀerence between the experimental group and the control group is

a result of the experimental treatment.

Here is an example of the need for replicates. Streams that drain from

catchments where gold is present often contain particles of alluvial gold

among the gravel on the stream bed. This gold can be recovered by “pan-

ning”–placing a few handfuls of stream gravel and sediment in the base of a

shallow pan, part-ﬁlling the pan with water and then gently swirling the

contents to wash away any ﬁne sediment, after which the gravel is spread

out within the pan and the gold grains picked out. A lot of experience is

needed before a person is skilled at doing this, and amateurs miss a lot of

gold. The standard gold pan is a wide and shallow metal bowl. One day you

notice that an older gold pan which has rusted and pitted on the inside

seems to give a more even spread of gravel, making it much easier to see the

gold. To test the hypothesis that a pitted pan gives better recovery you buy

two new standard gold pans, hammer 1000 pits into one and take both to a

gold bearing stream where you pan 16 ounces of gravel with the pitted one

while a friend pans another 16 ounces with the standard. To your delight

34 Introductory concepts of experimental design

you recover 0.025 ounces of gold, compared to 0.007 ounces for the stand-

ard one. The result is obvious – the pitted pan is better.

This result is consistent with the hypothesis but there is an obvious ﬂaw

in the experiment – with only one pan in each treatment, any diﬀerence

between them may be due to diﬀerences between the pans, the amount of

gold initially present in the gravel sample, the skill of the operators (includ-

ing that you might have concentrated more carefully on the contents of the

pitted pan because you hoped to recover more gold, or your friend did a

poor job out of disinterest), or all three. There is a need to replicate this

experiment and the replicates need to be truly independent – it is not

suﬃcient for you to use the pitted pan ten times while your friend does

the same with the standard pan, because any diﬀerences between treatments

may still be caused by operator skill. There will be more about this shortly.

4.3.2 Control treatments

Control treatments are needed because they allow the experimenter to

isolate the reason why something is occurring in an experiment by compar-

ing two treatments that diﬀer by only one factor. Frequently the need for a

rigorous experimental design makes it necessary to have several diﬀerent

treatments, more than one of which can be considered controls.

Here is an example. Ultramaﬁc rocks, and the soils produced from their

weathering, naturally contain high concentrations of metals such as chro-

mium, cobalt, manganese and nickel (that are toxic to plants) and low

concentrations of elements needed for plant growth. Therefore, very few

plant species grow in ultramaﬁc soils and the ones that do are sometimes so

distinctive that they can be mapped by remote sensing and used to identify

sites with particularly high concentrations of valuable metals, especially

nickel, for mining.

Some of these plant species also accumulate extremely high concentra-

tions of nickel in their leaves and stems, and it has recently been suggested

that these accumulators could be grown on contaminated land and the

mature plants later harvested and removed (and perhaps even smelted), to

reduce soil contamination. There is great interest in such bioremediation

and a environmental scientist did an experiment to trial it on a nickel-

contaminated mine tailing. The soil was plowed and planted with 10 000

seedlings of a nickel accumulator and the plants were left to grow for six

4.3 Manipulative experiments 35

months. At the end of the experiment the result was clear – less nickel was

present in the soil at the treatment site.

Unfortunately this experiment is not controlled. By plowing the site and

planting a nickel accumulator, more than one treatment has been applied.

Furthermore, six months have elapsed between measurements. So if you do

ﬁnd the nickel content of the soil has decreased, it may have been the result

of any of these factors and you cannot conﬁdently attribute it to the

presence of the nickel accumulator.

One essential improvement would be to have a control for the presence of

the plant, where a site was plowed but not planted, and then left for the same

amount of time. Another might simply be a control for time, where nothing

was done to the site. At this stage, by incorporating these two improve-

ments, you would have three treatments. Table 4.1 lists what these treat-

ments are doing to the tailings site.

For this design, if there is a reduction in nickel in the treatment with the

plants and no reduction in the other two, you might feel conﬁdent in claiming

the nickel accumulator had an eﬀect. Nevertheless, some mine site rehabil-

itators might say the design is still inadequate because the treatment in the

left-hand column of Table 4.1 is the only one in which any plant has been

grown. For example, the roots of the plants may simply have made the soil

more permeable to rainwater which leached some of the nickel, and the same

result might have been obtained by planting 10 000 seedlings of an inexpen-

sive non-accumulating species. This improvement has been listed in Table 4.2.

At this point you may be thinking that the above design is far too

complex, but experiments do have to have appropriate controls so that

the eﬀects of each potentially contributing factor can be isolated. For this

example, it would be extremely embarrassing to spend several thousand

dollars on expensive seedlings only to ﬁnd that any plant has the same eﬀect.

Table 4.1 Breakdown of three treatments in terms of their

eﬀect upon a nickel-rich site.

Planted nickel accumulator

Control for

disturbance

Control for

time

Nickel accumulator

Plowing Plowing

Time Time Time

36 Introductory concepts of experimental design

It is often diﬃcult to work out what control treatments you need for an

experiment. One way to clarify these is to list all of the things that are

actually happening in an experimental treatment and make sure you have

appropriate controls for each. Finally, the experimental treatments need to

be appropriately replicated – you cannot just have one replicate of each of

the four treatments in Table 4.2.

Here you may be wondering about the applicability of these concepts of

experimental design to geological systems, where you yourself do not (and

cannot) manipulate the experiment per se, because the “treatment” might be

some process or event that happened millions of years ago. Often earth

scientists can only do mensurative experiments, but a knowledge of the

essentials (and pitfalls) of doing these is becoming increasingly necessary if

you work in areas such as mine and hazardous waste site remediation.

4.3.3 Pseudoreplication

One of the nastiest pitfalls is appearing to have a replicated manipulative

experimental design that really is not replicated. This is another aspect of

“pseudoreplication” described by Hurlbert (1984) who invented the word –

before then it was just called “bad design.” Here is an example that relates

back to the use of bioaccumulators to rehabilitate mine sites.

A remediation geoscientist hypothesized that simply plowing the soil

would help decontaminate a mine site. They were aware of the need to

have appropriate treatments and replicates, so they chose two separate

tailing sites each 1000 m

in area. A toss of a coin decided which of these

was plowed, and the other site was left undisturbed. One hundred replicate

experimental units were taken at random within each site and analyzed for

nickel, which was high and similar at both. After six months the nickel

Table 4.2 Breakdown of three treatments in terms of their eﬀect

upon a nickel-rich site.

Planted nickel

accumulator

Control for

plant

Control for

disturbance

Control for

time

Nickel accumulator Nonaccumulator

Plowing Plowing Plowing

Time Time Time Time

4.3 Manipulative experiments 37

content in the soil was resampled and found to be much lower at the plowed

site. The scientist was delighted – an inexpensive experiment with 100

replicates of each treatment had produced a result consistent with the

hypothesis.

Unfortunately, there are not 100 truly independent replicates in each area

because the treated site is in a diﬀerent place to the control. All replicates of

the plowed treatment were in one tailing and all those from the undisturbed

one were in another. Therefore, any diﬀerence in nickel may, or may not,

have been due to the plowing – it could equally well have been due to some

other (perhaps unknown) diﬀ erence between the sites. The number of

replicates is the sites, not the number of experimental units within each,

so the experiment has no eﬀective replication at all and is essentially the

same as the (unreplicated) manipulative experiment on gold panning

described earlier in this chapter.

An improvement to the design would be to run each of the two treat-

ments at several tailing sites, but here too, it is still be necessary to have truly

independent replicates. If you do not, the experiment may still suﬀer from

apparent replication, and here are four examples.

(1) Treatments separate but clumped. Even if you have several separate

replicates of each treatment, the arrangement of these can lead to a lack

of independence. For example, you may have your treatments all

clumped together at one end of the mining lease and the controls at the

other, but this is no better than an unreplicated example (Figure 4.4(a)).

(2) Replicates placed alternately. If you decided to get around the clustering

problem by placing treatments and controls alternately (i.e. by placing,

from east to west, treatment #1, control #1, treatment #2, control #2,

treatment #3 etc.) there can still be problems. Just by chance all the

treatment sites (or all the controls) might be in line with an underlying

feature of the area (e.g. a regular alternation of grain size or soil

composition), or subject to some other regular feature you are not

even aware of (Figure 4.4(b)).

(3) Unavoidable segregation of replicates. Often, due to practical consid-

erations, you have to have all of your replicates of one treatment in only

one place, and all replicates of the control group in another.

Unfortunately, if there is something peculiar to one location, in addi-

tion to the variable you are intentionally manipulating, then either the

38 Introductory concepts of experimental design

experimental or control treatment may be aﬀected. For example, if you

were doing an experiment comparing crystallization at two high tem-

peratures you might have access to only two ovens, one set at 300 °C and

the other at 400 °C. Unfortunately these ovens may diﬀer in more ways

than their set temperature. One may vary in temperature by +/− 10 °C,

while the other might be more accurate and only vary by only +/− 1 °C.

This pattern is called “isolative segregation” (Figure 4.4(c)).

(4) The ﬁnal example is more subtle. Imagine you are an aqueous geo-

chemist interested in the hypothesis that “pH aﬀects the minerals that

crystallize from sulfate-rich water.” You set up ﬁve control beakers and

ﬁve experimental beakers and place them on the bench in a completely

randomized pattern to get around problems in (a), (b) and (c) above.

All the beakers have water constantly ﬂowing through them, so you set

up two storage tanks, one with water of pH 3.0 and the other with a

higher pH of 7.0. Water from each storage tank is piped into ﬁve

beakers as shown in Figure 4.5.

This looks ﬁne, but unfortunately all ﬁve beakers within each treatment

are sharing the same water. All in the “pH 7” treatment receive water from

Tank A and all beakers in the “ pH 3” treatment receive water from Tank B,

(c)

Drying oven 1 Drying oven 2

300

°C 400 °C

+/–

10 °C +/– 1 °C

(b)

(a)

T1 T2 T3 T4 T5 C1 C2 C3 C4 C5

C5T5C4T4C3T3C2T2C1T1

Figure 4.4 Three cases of apparent pseudoreplication. (a) Clustering of

replicates means that there is no independence among controls or treatments.

(b) A regular arrangement of treatments and controls may, by chance,

correspond to some feature of the environment that might aﬀect the results.

variance in temperature might be diﬀerent.

4.3 Manipulative experiments 39

so any diﬀerence in precipitated minerals between treatments may be due

either to the pH or some other feature of the supply tanks and circulation

system. Really, therefore, this design is little better than the case of isolative

segregation (example (c) above). Ideally, each beaker should have its own

separate and independent supply. Finally, the allocation of replicate beakers

to treatments should be done using a method that removes any possibility of

unintentional bias by the experimenter. (For example, the toss of a coin was

used to allocate paired heaps of soil to treated and untreated areas in the

experiment with lead and apatite described in Section 2.2.)

4.4 Sometimes you can only do an unreplicated experiment

Although replication is desirable in any experiment, there are some cases

where it is not possible. For example, when doing large-scale mensurative or

manipulative experiments on systems such as lakes or rivers, there may be

only one polluted lake or river available to study. Although you cannot

attribute the reason for any diﬀerence, or the lack of it, to the treatment

(e.g. a polluted versus a relatively unpolluted river) because you only have

one replicate of each, the results are still useful. First, they are still evidence

for or against your hypothesis and can be cautiously discussed in the light

of the lack of replication. Second, it may be possible to achieve replication by

analyzing your results in conjunction with those from similar studies

done elsewhere by other researchers. This is called a meta-analysis.

Finally, the results of a large-scale but unreplicated experiment may suggest

T1 C1 C2 T2 T3 C3 T4 C4 C5 T5

Tank A – pH 7.0

Tank B – pH 3.0

Figure 4.5 The positions of the treatment beakers are randomized, but all

tanks within a treatment share water from one supply tank.

40 Introductory concepts of experimental design

smaller-scale experiments that can be done with replication so that you can

continue to test the hypothesis.

4.5 Realism

Even an apparently well-designed mensurative or manipulative experiment

may still suﬀer from a lack of realism. Here are two examples.

The ﬁrst is a mensurative experiment on the occurrence of impact craters

on the Earth. It was once thought that meteor impacts on Earth were fairly

rare, and the contrast between the appearance of the Moon (which is pock-

marked with craters), and the Earth’s surface (which is almost lacking in

visible craters), supported the hypothesis “Impacts rarely occur on the

surface of the Earth compared to the Moon.” The null hypothesis (that

few people even considered to be a possibility) was that impacts occur at

about the same frequency on the surface of the Earth and the Moon.

All this changed with two big events during the 1960s. First, the return of

samples from manned lunar exploration revealed the relative ages of diﬀer-

ent sizes of lunar craters and supported the idea that impacts are occurring

continuously, although with a decreasing frequency, in our solar system.

Second, the theory of plate tectonics was introduced, which said that new

crust is constantly being created and subducted all over the Earth’s surface.

Suddenly it was apparent that the huge impact basins on the lunar surface

were created more than four billion years ago, and if comparable craters

existed on the Earth we would have no record of them. In the following

decades, studies of what is now known as Shoemaker Crater in Arizona

identiﬁed the characteristics of terrestrial impact craters (shocked minerals,

enrichment of trace elements only found in meteorites, etc.). Now, numer-

ous examples of impact craters have been found, particularly in older

terrains like the Canadian shield. So the naive assumption that impacts

never occur on the surface of the Earth turned out to be quite uninformed

and the comparison was unrealistic because surfaces of vastly diﬀerent ages

on the Moon and Earth were being compared.

Second, a geoscientist did an experiment to test the hypothesis that the

amount of olivine (Mg

SiO

) produced when MgO and SiO

react together

is dependent on temperature. They used equilibration temperatures of 700,

800, 900 and 1000 °C, and heated the mixtures for one week, after which the

treatment samples were removed from the furnaces and inspected for

4.5 Realism 41

olivine growth. The conclusion was that there was no eﬀect of temperature

on olivine production. Later, it was realized that these temperatures were

unrealistically low compared to those occurring within the Earth’s crust and

a subsequent experiment that included replicates at 1300, 1400 and 1500 °C

showed more olivine grew at the higher temperatures.

4.6 A bit of common sense

By now, you may be quite daunted by the challenge of being able to design a

good experiment. Provided, however, that you have appropriate controls,

replicates and have also thought about any obvious problems of pseudo-

replication and realism, you are well on the way to a good design.

Furthermore, the desire for a near-perfect design has to be balanced against

ﬁnancial constraints as well as space and time available to do the experi-

ment, so often it is not possible to have as many replicates as you would like.

It also depends on the type of science you do. For example, if you were doing

a precisely controlled standardization using several diﬀerent methods you

would be unlikely to set up diﬀerent replicates of each treatment in a

random pattern within the laboratory and to do this might grossly increase

the risk of making a procedural error (not to mention the cost of the

experiment!). Similarly, most experimental petrologists working with gas-

mixing furnaces are very careful to calibrate thermostats, identify temper-

ature gradients and locate furnace hotspots accurately, so that conditions

can be strictly controlled. They would never be concerned about clustering

of replicates or isolative segregation because they were conﬁdent that con-

ditions did not vary among furnaces or in diﬀ erent parts of the same one.

Most of the time they may be right, but considerations about experimental

design need to be borne in mind by all scientists, especially if you are

working in areas where conditions cannot be strictly maintained.

Sometimes you may not have the resources to do a large manipulative

ﬁeld experiment at more than one site. Certainly, in many geological

studies, there may well be only one accessible road-cut or outcrop suitable

for study. Although, strictly speaking, the individual results cannot be

generalized to other sites, they may nevertheless apply and with careful

interpretation and discussion of results you can make more general pre-

dictions. For example, the “apatite remediation” experiment described in

Chapter 2 was initially conceived in the laboratory, and repeated at

42 Introductory concepts of experimental design