Zelditch M.L. (и др.) Geometric Morphometrics for Biologists: a primer

Подождите немного. Документ загружается.

chap-07 4/6/2004 17: 24 page 188

chap-08 4/6/2004 17: 25 page 189

Computer-based statistical methods

Most active scientists are probably familiar with some computer-based statistical methods,

such as the bootstrap, jackknife, permutation or Monte Carlo simulations. The point of

this chapter is to present their underlying principles in a coherent fashion. The basic ideas

of all these methods appeared in the work of R. A. Fisher in the 1930s, but the ideas and

techniques were neither developed extensively nor used widely until recently. Perhaps the

best summary of the discipline is contained in the title of Efron’s (1979) paper, Computers

and the theory of statistics, thinking the unthinkable. The approach he outlined was indeed

unthinkable prior to the advent of computers, and it could not be used widely until com-

puters became inexpensive as well as fast, and generally available to researchers (which is

why there is such a long time-lag between the original development of the ideas and their

widespread application). Computer-based statistical methods are computationally inten-

sive because they replace the complex analytic mathematical methods of classical statistics

by an extensive use of randomization and repeated calculations. The enormous number of

calculations required by these methods makes them unthinkable without inexpensive (and

fast) computers.

Classical statistics is highly algebraic, relying on extended algebraic derivations of for-

mulae that are based on a limited number of well studied distributions, particularly the

normal (Gaussian), F-, gamma, chi-square, uniform, and Poisson distributions, among

others. Before the advent of personal computers, extensive calculations were expen-

sive (in both time or money) so statistical research relied primarily on analytic proofs

and mathematical approximations to minimize the number of calculations needed. Cur-

rently, most calculations are done by computers even when analytical statistical methods

are used; computers have altered even how classical statistical methods are used and

taught.

In this chapter, we present a brief discussion of some of the basic statistical concepts that

are needed to understand statistical methods in general (such as confidence intervals and

hypothesis testing), as well as the more specialized concepts that are needed to understand

computer-based statistical methods. We present four classes of these methods, including

the bootstrap, jackknife, and permutation tests, and Monte Carlo simulations. To illus-

trate these methods, we will focus on a few univariate statistical tests. The extension to

ISBN 0–12–77846–08 All rights of reproduction in any form reserved

chap-08 4/6/2004 17: 25 page 190

190 GEOMETRIC MORPHOMETRICS FOR BIOLOGISTS

multivariate statistics is not difficult, but it seems useful to focus on univariate statistics to

develop an intuitive understanding of how computer-based methods work. More complete

discussions of the topics presented in this chapter can be found in the texts by Efron and

Tibshirani (1993) and Manly (1997).

Basic statistical concepts

Human beings look for general patterns, such as connections between events. Statistics

provides tools to help identify these patterns as well as to verify that they are reliable,

given the information we have. Two statistical tasks related to studying patterns are the

estimation of confidence intervals, and hypothesis testing. To talk about these in any detail,

we need the basic concepts of:

•

samples

•

populations

•

variables

•

probability distributions

•

statistics

•

parameters.

A sample is a collection of individual observations made on a limited number of indi-

viduals representing a population. An individual observation is the smallest sampling unit

in the study, which might be an individual organism or one of its parts, or even a collection

of organisms such as a species or bacterial colony. The sample is drawn from a population,

which is the set of all individuals of a specific type, such as all members of a species, or

all teleosts in a given river basin, or all the leaves on an individual tree. In general it is not

possible to observe (much less measure) all the individuals in the population, so we rely

on the sample, from which we generalize to the population.

A variable is a measurement (or observation) made on individuals within the sample.

Univariate indicates that a single measurement was made on each individual; bivariate

indicates that two measurements were made, and multivariate indicates that three or more

measurements were made. Variables can be subdivided into two distinct types: discrete and

continuous. Discrete variables comprise integer-valued variables, which are ordered (e.g.

1 < 2 < 3), and categorical variables, which are not ordered (e.g. males versus females).

Categorical variables are sometimes represented by integers, but no ordering is thereby

implied. Continuous variables are real-valued, meaning that they are measured on an

infinitely divisible scale.

A probability distribution is a mathematical function that describes the probability of

a measurement taking on a particular value or a range of values, depending on whether

the variable is discrete or continuous, respectively. Since landmark coordinates and partial

warp scores are always continuous variables, we will focus on probability distributions

for continuous variables. If X represents a specific value of a measured variable and f (X)

is a probability distribution function, then the following relationship holds:

Probability (A ≤ X ≤ B) =



f (X)dx (8.1)

chap-08 4/6/2004 17: 25 page 191

COMPUTER-BASED STATISTICAL METHODS 191

which means that the probability that X lies between A and B is the sum of the probabilities

that X is equal to each value between A and B. Given the probability distribution of a

variable, or a function of a variable, we can determine (in principle) a large number of

useful results, such as the mean and the standard deviation of the variable, as well as the

probability of the different outcomes of different types of measurements.

Analytic statistical methods use a number of well-known, well-characterized mathemat-

ical models of populations, including the normal (or Gaussian) distribution:

f (X) =

√

2πσ

−(X−X

)

2σ

(8.2)

where X

is the mean of the distribution and σ is the standard deviation. Analytic statistical

methods start by assuming a specific distribution function for the variable, then use that

distribution function to derive results (analytically) about the probabilities of different

events. A commonly used result, based on the normal distribution, is that if X

is the mean

of the distribution and σ is the standard deviation, then the chance that X measured on

an individual drawn randomly from the sample exceeds X

+1.96σ is 2.5%. This result

is derived by integrating the Gaussian distribution function from 1.96σ to infinity, which

yields a probability of 0.025. As you might expect, the results based on analytic statistical

models are valid only when the mathematical model of the distribution matches the actual

distribution of the measured variables.

We would like to make statements about the population from which our sample is

drawn so we can generalize from our limited observations to the population as a whole.

That would not be necessary if we could measure every individual in a population, because

then we could determine the exact value of any statistic of interest. A statistic is any math-

ematical function calculated from all measured individuals, such as the mean, standard

deviation, variance, maximum, minimum, or range. Because we cannot measure all the

individuals in a population we must rely on our representative sample, which leads to

uncertainty about our conclusions. The true value of the statistic in the population is the

parameter, which is what we are trying to estimate from our sample.

Confidence intervals

The confidence interval tells us the range of values that a given statistic might have in

light of the uncertainties due to sampling. Whenever we present results based on samples,

such as the value of a mean, we must also include a statement about how accurately that

value is known. Otherwise, we cannot evaluate the meaning of the difference between

two measurements. For example, suppose you are told that John is 185 cm tall and Bob

is 190 cm tall: is it possible to say that Bob is taller than John? In this case, it would seem

easy to say that he is because these are two individuals – they are not samples representing

populations. However, even in this case, the height of an individual is a sample of heights

that we could measure for these individuals. Perhaps their heights were measured with

their shoes off, with a very precise measuring device, by a single technician practiced in

such measurements. Under those conditions, we might anticipate a small error, such as

1 cm or less. Considering that expected error, we could say that John’s height is 185 cm,

with a confidence interval of 184 cm to 186 cm, whereas Bob’s height is 190 cm, with a

chap-08 4/6/2004 17: 25 page 192

192 GEOMETRIC MORPHOMETRICS FOR BIOLOGISTS

confidence interval of 189 cm to 191 cm. Given that information, we can conclude that

Bob is taller than John. Now imagine measuring their heights using a much coarser device,

such as marks placed every 10 cm on a pillar in the hallway, and imagine replacing the

expert technician by an observer sitting on a chair on the opposite side of the hall who

notes the height of people passing by the marks. Furthermore, suppose that two different

observers estimate John’s and Bob’s heights on different days, sitting on chairs of dif-

ferent heights. Under these conditions, we might expect substantial measurement errors

for a large variety of reasons, including differences in posture between John and Bob,

the difficulty of measuring heights of moving objects, differences in the thickness of the

soles of their shoes, as well as differences between observers and the parallax introduced

by changing chair height, and so on. The measured heights might be off by as much as

5 cm, so John’s height should be given as 185 cm with a confidence interval ranging from

180 cm to 190 cm, while Bob’s height should be given as 190 cm with a confidence interval

ranging from 185 cm to 195 cm. Each estimate would be within the confidence interval

of the other estimate, so we could not say that Bob and John are significantly different

in height.

We are all used to thinking about numbers mathematically, so it may seem obvious that

190 is greater than 185. However, we need to bear in mind that scientific measurements are

not pure mathematical objects; rather, they result from human attempts to assign numbers

to physical quantities. Not all attempts are of equal quality, so we need to take that quality

into account when comparing measurements. That is part of what enters into confidence

intervals.

Most often, we are comparing statistics calculated over samples drawn from popula-

tions, rather than comparing observations on two individuals. Sampling introduces another

possible source of error. We wish to make statements about the population, so our con-

fidence interval must reflect not only the quality of the individual measurements but also

how well the sample is expected to reflect the population from which it is drawn. Statis-

tics calculated from large samples drawn from homogeneous populations will tend to

yield accurate estimates of the population’s parameters, whereas those calculated from

small samples drawn from heterogeneous populations will not. The confidence interval

expresses the uncertainty of sampling as well.

We need the confidence intervals as well as the estimated statistics to determine if pop-

ulations differ. The confidence intervals for many statistics can be calculated from the

probability distribution functions of the population. For example, if we assume that the

population follows a normal distribution, we can state the uncertainty of a measurement

by stating its standard deviation. In doing so, we imply that the measurement follows a

normal distribution. Another approach to stating the uncertainty of the measurement is to

give its 95% confidence interval. Doing so implies that repeating the measurement process

many times would result in 95% of the measured values falling within that interval (which

is equivalent to saying that there is a 95% chance that any single repetition would fall

within that interval), but it does not imply anything more about the distribution of values

within those limits.

Just as the value of the mean is an estimate from a sample, so is the confidence interval.

Different statistical approaches differ in how accurately they are able to estimate confidence

intervals. We return to this issue below when we talk about using computer-based methods

to estimate confidence intervals.

chap-08 4/6/2004 17: 25 page 193

COMPUTER-BASED STATISTICAL METHODS 193

Hypothesis testing

Statistics works with limited samples, so it is difficult to prove that a statement is true.

Statistics is more useful for proving that a statement is false within some level of confidence.

Our measurements will always have some degree of uncertainty, so we cannot prove the

truth of a statement like “the adult body mass of shallow water crabs is 40 g.” Even if

we could measure every crab in the population, we would still have some error in our

measurements, so we would still have uncertainty in our estimate of the mean. The true

mean body mass is likely to be somewhere within a confidence interval, but we cannot say

exactly where within that range it is. At best, we can say that the center of the interval is

the same as the hypothesized value. This is still not enough to claim that the hypothesis is

actually true. We can, however, disprove a statement like the hypothesis with some level of

confidence. Suppose that we found that the 95% confidence interval of mean adult body

mass is 37–39 g; this means the chance that the mean adult body mass of the shallow water

group is greater than 39 g is 2.5% ((100 −95)/2). By convention, a hypothesis is rejected

when there is only a 5% probability of being incorrect, so we would normally reject the

hypothesis that the mean weight is 40 g.

In general, when using a statistical approach to hypothesis testing, we state the hypoth-

esis we wish to disprove, which is called the null hypothesis. Usually, the null is the

hypothesis of “no difference” or “no effect.” The null hypothesis states that there is noth-

ing to explain – the apparent signal in the data is an outcome of chance. In our example

above, the null hypothesis is: “The adult body mass of the shallow water crabs is 40 g.”

We could state two alternatives: (1) “The adult body mass of the shallow water crabs is

less than 40 g” and (2) “The adult body mass of the shallow water crabs is greater than

40 g.” Based on the test of the null hypothesis, the chance that it is true is less than 2.5%.

The data are consistent with only one of the two alternatives – that the adult body mass

of the shallow water crabs is less than 40 g.

There are several subtleties to note about this approach to hypothesis testing. First, being

unable to reject the null hypothesis does not mean that the null is actually true. Instead, our

samples might be so small and the variability within a population so great that we do not

have enough evidence to reject a false null. When we cannot reject the null, all we can say

is that the data are consistent with it. Given more data (or more powerful tests), we might

be able to reject it. Another point to note is that we might have more than two alternative

hypotheses, and rejecting the null might not tell us which of them is most consistent with

the data. Additional statistical tests may be needed to rule out alternative nulls.

When carrying out statistical hypothesis testing, we can make two kinds of errors. Type

I error is rejecting a null hypothesis when it is true; the converse is Type II error – i.e. failing

to reject the null when it is not true. Thus, Type I error means that we improperly rejected

the null, and Type II error means that we have improperly failed to reject it (working

through the double negatives can be difficult, but we do not ever accept the null, so we

cannot avoid using the phrase “fail to reject”). The rate of Type I errors is controlled by

setting the alpha level of the test, which is the chance that the null hypothesis is true. The

alpha level typically favored is 5%, meaning that the null is rejected as “untrue” when the

estimated probability of the observed value of the test statistic (under the null hypothesis)

is 5% or less. For example, if the probability is less than 5% that two populations have

the same mean, we reject the null hypothesis of equality of means. The rate of Type II

chap-08 4/6/2004 17: 25 page 194

194 GEOMETRIC MORPHOMETRICS FOR BIOLOGISTS

error is more complex, being influenced by: (1) the nature of the null hypothesis and the

alternatives; (2) the statistical test used; (3) sample size; and (4) the alpha level used to

reject the null hypothesis. Higher alpha levels lower the rate of Type I error but increase

the rate of Type II error (for a more extensive discussion of the relationship between these

errors, see Sokal and Rohlf, 1995). Estimation and control of the Type II error rate is

rather difficult, and many workers focus on Type I error, neglecting Type II error, when

assessing results of statistical tests.

Why we need computer-based statistics: an example

To develop an intuition about the need for computer-based statistics, we will work through

an example. Suppose X is a set of 31 observations of a length:

X ={2, 2, 3, 4, 2, 5, 3, 2, 6, 2, 3, 4, 6, 2, 1, 4, 3, 7, 2, 3, 4, 4, 5, 8, 5, 2, 1, 3, 4, 4, 3} (8.3)

In this case, N =31. We can compute the mean (denoted <X> for “the expectation of

X”) by:

<X> =



i=1

(8.4)

where X

is the ith element in the list. In our example, <X> =3.52. Of course, we also

need to quantify our uncertainty in this value. If we assume that the distribution of X fits

the model of a normal distribution, then the standard error of the mean is given by the

standard deviation σ divided by the square root N (the number of observed individuals).

The standard deviation is:

σ =





i=1

−<X>)

N − 1



1/2

(8.5)

so the standard error of the mean (SEM) is:

SEM =

√





i=1

−<X>)

N(N − 1)



1/2

(8.6)

For our example, σ =1.69 and N =31, so SEM =1.69/(31)

1/2

=0.304.

The 95% confidence interval for the mean, assuming a normal distribution, ranges

from <X> −1.96(SEM) to <X> +1.96(SEM) because, for a normal or Gaussian distri-

bution, 95% of the values in the distribution lie within 1.96 standard deviations of the

mean. So, for our example, 1.96 SEM =0.304 and <X> =3.52, so the 95% confidence

interval is from 2.92 to 4.12. Suppose that we want to claim that the average body length

of this population is greater than 3.0 cm. Again, using the normal distribution, we can

calculate that the chance of the mean being less than or equal to 3.0 is 0.049%, so we

can reject the hypothesis that the mean is less than or equal to 3.0 at a 5% confidence

level (meaning that we accept a 5% chance of rejecting the null model when it was true

(Type I error)).

chap-08 4/6/2004 17: 25 page 195

COMPUTER-BASED STATISTICAL METHODS 195

What difficulties arise in this example? First, we have assumed that the distribution

is normal. This is important, even though statistics based on the normal distribution are

known to be robust to violations of the assumptions of normality. Nevertheless, as the

distribution departs further from normality, larger errors appear in the results, leading

to increased error rates. The validity of the normal distribution for our example has not

been determined. Is that assumption reasonable? If the distribution is normal, 1.9% of

the measurements will be less than or equal to zero (that is the expectation under the

model). Does that pose a problem? Yes, because we are measuring lengths, and none

can be less than zero, under any circumstances – in fact, the lower bound may be sub-

stantially larger than zero (due to physiological constraints on the size of the organism).

So we know that our distribution must deviate from the normal distribution. Perhaps

that deviation has only a small effect on our estimate of SEM, but we are relying on

the reputation of the normal distribution as a robust estimator to reassure ourselves

about that. We really do not know what effect that lower bound has on our statistical

inferences.

The other difficulty we face is the lack of an exact formula for the standard error of any

statistic other than the mean. Suppose we want to know the standard error in the median

of the distribution. We can calculate the median of our measurements of X, which equals

3.0, but can we actually conclude that the median of the population is greater than 2.0? We

do not really know the range of values that the median might take on for this distribution,

and the normal model provides no estimate of the uncertainty in the median. The standard

deviation and variance of populations are also of tremendous biological interest, but how

do we estimate the range of values for these statistics?

Resampling-based methods

Having noted that we can face serious difficulties when we assume a normal distribution

and rely on the theory based on it, we now examine methods that allow us to make

statistical inferences without assuming any distribution.

The bootstrap

We begin with the bootstrap because it is probably the easiest to understand. It was not the

first computer-based statistical method developed; in fact it is one of the more recent (it was

developed from jackknife and permutation methods). The term “bootstrapping” comes

from the novel Baron Münchausen’s Narrative of his Marvellous Travels and Campaigns

in Russia, by Rudolph Erich Raspé (1785), in which the Baron falls to the bottom of a

deep lake. He cannot figure out what to do until, at the last moment, he thinks to pull

himself up by his own bootstraps. This describes, fairly accurately, the approach used in

a bootstrap procedure: the observed data themselves are used as a basis for resampling;

we approximate the unknown statistical distribution from which the data were drawn by

(randomly) resampling our data.

A bootstrap set is a set of data of the same sample size as the original data set, whose

elements are randomly drawn with replacement from our original set of observations. To

randomly draw them (with replacement) from a set of N elements, a uniformly distributed

chap-08 4/6/2004 17: 25 page 196

196 GEOMETRIC MORPHOMETRICS FOR BIOLOGISTS

random number from 1 to N is generated by a random number generator. The corre-

sponding element from the original set of observations then forms the first element in the

bootstrap set. For example, given our 31 observations, we will construct a sample that

also has 31 observations. The number provided by the random number generator is 8, so

we take the value of the eighth individual of our sample as the first value in the bootstrap

set. This procedure is repeated N times. Note that a single value from the original data

set may appear multiple times in a bootstrap set (this is because we are sampling with

replacement, meaning that we do not remove an individual from the sample after we have

placed its value in the bootstrap set). Additionally, not all values in the original set need

appear in the bootstrap set.

To develop an understanding of how a bootstrap set is formed, we’ll consider an

abstract, symbolic example. Suppose C contains five values:

C ={C

, C

} (8.7)

To form a bootstrap version of C, we generate a list of five random numbers, each

independently chosen and ranging from 1 to 5 (because N =5):

L ={52435} (8.8)

The numbers in L are the ordinal positions of the elements of C; C

Bootstrap

contains the

corresponding values of C (e.g. L

=5, so it corresponds to the fifth element of C, which

is C

). Thus:

Bootstrap

={C

, C

} (8.9)

Note that C

does not appear in this bootstrap set, while C

appears twice.

Returning to the numerical example presented earlier:

X ={2, 2, 3, 4, 2, 5, 3, 2, 6, 2, 3, 4, 6, 2, 1, 4, 3, 7, 2, 3, 4, 4, 5, 8, 5, 2, 1, 3, 4, 4, 3} (8.10)

To form a bootstrap set, X

Boot

, from X, we generate the list, B, of 31 random numbers:

B ={30, 8, 19, 16, 28, 24, 15, 1, 26, 14, 20, 25, 29, 23, 6, 13, 29, 29,

13, 28, 2, 11, 26, 1, 5, 7, 7, 19, 9, 7, 1} (8.11)

We then select the elements of X corresponding to those ordinal values:

Boot

={4, 2, 2, 4, 3, 8, 1, 2, 2, 2, 3, 5, 4, 5, 5, 6, 4, 4, 6, 3, 2, 3, 2, 2, 2, 3, 3, 2, 6, 3, 2}

(8.12)

The first element of X

Boot

is the 30th element of X (because 30 is the first element of B),

and the seventh element of X appears three times in the bootstrap set (because 7 appears

three times in B). We can now calculate the mean, standard deviation and median of X

Boot

> =3.39, σ

Boot

=1.62, and median(X

Boot

) =3. These values are slightly different

from those of the original distribution, <X> =3.52; σ =1.69, and median(X) =3.0.

To arrive at an estimate of the confidence intervals for these statistics, we will compute

a large number (N

Bootstrap

) of bootstrap sets. We will then determine the 95% confidence

interval over the N

Bootstrap

sets, forming a bootstrap estimate of the confidence intervals

chap-08 4/6/2004 17: 25 page 197

COMPUTER-BASED STATISTICAL METHODS 197

on the mean, standard deviation and the median. If we generate 200 bootstrap sets based

on X, we find that the 95% confidence intervals for the mean is 3.00–4.10; for the stan-

dard deviation the confidence interval is 1.23–2.10, and for the median it is 3.00–4.00.

The normal model predicted a 95% confidence interval for the mean, 2.91–4.12, so the

two methods approximately agree. They appear to differ at the lower boundary (at small

lengths), which is where we expect departures from the normal distribution, for the reasons

discussed earlier.

The approach outlined here may be extended to virtually any statistic and to any

function, univariate or multivariate. We now use it to perform t-tests.

Using the bootstrap to conduct t-tests

The t-test is used to compare the means of two samples, the F-test to compare means of

two or more samples (both procedures are discussed at more length in the next chapter).

The question is whether the mean of one group differs from the mean of another in a

statistically significant way. It is possible that the observed difference in means is due to an

arbitrary division of one group into two, such that the variability within the two groups

gives rise to a difference between means solely by chance.

Let us look again at our sample of 31 measured lengths:

X ={2, 2, 3, 4, 2, 5, 3, 2, 6, 2, 3, 4, 6, 2, 1, 4, 3, 7, 2, 3, 4, 4, 5, 8, 5, 2, 1, 3, 4, 4, 3} (8.13)

and consider a second group of 18 lengths:

Y ={2, 2, 3, 2, 4, 2, 3, 2, 8, 9, 2, 9, 3, 2, 3, 3, 3, 9} (8.14)

Using the normal model, we find that <X> =3.52, σ

=1.69, and <Y> =3.94 and

=2.71. To test whether the means are different, we find the probability of statistic t:

t =

(

)









−1) +σ

−1)

−2







(8.15)

with degrees of freedom of (N

−2). For relatively large values of N

and N

, the

t-value will be normally distributed with a mean of zero and a standard deviation of one,

provided that the null hypothesis of equal means is true. If the absolute value of t exceeds

1.96, we may assert that, under the normal model, there is only a 5% chance of the mean

values being that different by chance. We can thus reject the null hypothesis at a 5% level

of confidence.

The problem is that the list of lengths contained in Y is highly non-normal. Most values

are close to 3, but there are several around 8 or 9, so Y appears to be rather bimodal.

Also, in a normal distribution with a mean of <Y> =3.94 and a standard deviation of

=2.71, we would expect that 7.3% of the measured lengths would be less than zero.

So the distribution of Y departs substantially from normality, more so than does the

distribution of X.