Drennan R.D. Statistics for Archaeologists: A Common Sense Approach

Подождите немного. Документ загружается.

PRINCIPAL COMPONENTS ANALYSIS 305

are strong and the signs of all four are the same. If energy invested in burials had

a strong negative loading and the fauna/sherd ratio had a strong positive loading,

this would mean that high energy investment in burials and low fauna/sherd ratios

corresponded to this component (and thus to each other). Since all the signs are

the same, we know that high energy investment in burials, high fauna/sherd rations,

high proportions of bowls, and high proportions of decorated ceramics all correlate

with each other on this component. None of the other variables show loadings of

much strength at all on this component, suggesting that none of them relates very

strongly to wealth (or whatever it is that this element of patterning in the dataset

may represent).

Three variables have strong loadings on the second component (again all with

the same sign): presence of a platform, presence of mace heads in burials, and the

shell/sherd ratio. We saw a similar relationship between platforms and mace heads

in the multidimensional scaling, and we also noted there a slightly more ambiguous

relationship between these two things and a high ratio of shell. The shell/sherd ratio

loads strongly on the second component here but its divided loyalties, so to speak,

are reﬂected in its loading on the third component, which is almost as strong. The

four variables that have such high loadings on the ﬁrst component have very low

loadings on the second, and the three variables with high loadings on the second

component have very low loadings on the ﬁrst. The message for us in this obser-

vation is that the ﬁrst two components are quite independent of each other. We

drew this same conclusion from the fact that these two elements appeared in the

multidimensional scaling as two gradients that were perpendicular to each other.

The two variables with the strongest loadings on the third component are the

proportions of kiln wasters and obsidian. We saw that high proportions of these two

things formed clusters that overlapped in one view of the multidimensional scaling

conﬁguration, but not so much in another view in which they also appeared. The

shell/sherd ratio also loads fairly highly on this third component. It is interesting

to note that obsidian also has a moderate loading on the second component, par-

alleling the moderate relationship we saw in the multidimensional scaling between

high proportions of obsidian and the gradient interpreted there as prestige. Finally,

though, all four variables that did not have extremely high loadings on the ﬁrst two

components, have moderate to strong loadings on the third component. In part, this

is produced by the fact that household units with high values for these four variables

tend to share lower values for the six variables that have very high loadings on the

ﬁrst two components.

The only loading of much strength on the fourth component is for the proportion

of debitage. It is in this way that this variable’s particular lack of connection to others

is shown in the principal components results. In the multidimensional scaling, both

obsidian and kiln wasters stood partly apart and partly together, and both obsidian

and shell were connected in their tendency to be at the high end of the gradient

interpreted as prestige. Debitage, however, formed a cluster with little indication

of overlap or relationship to anything else in the multidimensional scaling, and it

stands apart in the principal components analysis as well.

306 CHAPTER 24

Table 24.3. Component Loadings (with Orthogonal Rotation) for Analysis of the

Ixcaquixtla Household Dataset

Components

12345

Energy in Burials −0.960 −0.008 −0.044 0.068 0.047

Fauna/Sherds −0.950 0.045 −0.061 0.020 0.068

Bowls % −0.937 −0.027 0.129 −0.127 0.151

Decoration % −0.854 −0.128 −0.164 0.022 −0.272

Shell/Sherds 0.002 0.946 −0.058 −0.151 0.103

Mace Heads 0.068 0.908 0.099 0.087 0.023

Platform −0.008 0.597 0.743 − 0.087 −0.156

Obsidian % 0.089 −0.088 0.934 0.166 −0.238

Debitage % 0.004 0.051 −0.097 −0.970 0.169

Wasters % 0.041 − 0.086 0.374 0.224 −0.874

There is considerable difference of opinion about rotation of loadings in prin-

cipal components analysis. Once the components have been extracted, they can be

thought of as coordinate axes in a space of multiple dimensions. This whole set

of extracted axes can be rotated around in the space to maximize various differ-

ent criteria of relationships with the original variables. They can be subjected to

orthogonal rotation in which the axes are rotated as a set and all are kept at right

angles to each other (i.e., they are uncorrelated with each other). Or they can be

rotated individually in oblique rotation, in which they lose the property of being

uncorrelated with each other. Within each of these two large families of rotation

there are several variants.

Orthogonally rotated component loadings for the Ixcaquixtla household analysis

are given in Table

24.3. As usual with orthogonal rotation the contrasts between high

loadings and low loadings on each component are maximized. The same four vari-

ables that showed strong loadings with the same sign on the ﬁrst component in the

unrotated components behave the same in the rotated components. The same three

variables load strongly on the second component in both sets as well. In the rotated

components obsidian appears more strongly linked to platforms by virtue of their

strong loadings on the third component. The relationship between obsidian and kiln

wasters that we saw in the unrotated components and in the multidimensional scal-

ing has disappeared, and both debitage and kiln wasters stand strongly apart from

the other variables. The two sets of component loadings correspond quite well in

regard to the main elements of patterning, and sometimes these main elements show

up more sharply in rotated component loadings. The discrepancies in the indica-

tions of some of the lesser elements in the patterning exemplify what worries some

analysts about component rotation.

There is no easy answer, and certainly no consensus answer, to the question about

the wisdom of rotating components. Large statpacks all do principal components

and factor analysis, and they all offer rotation as an option. There is certainly no

harm in looking at both rotated and unrotated components. If the important ele-

ments of patterning in both sets of loadings correspond well to each other, then it

PRINCIPAL COMPONENTS ANALYSIS 307

Statpacks and Reporting Results

Principal components analysis is almost certainly a more straightforward

proposition with your statpack than either of the other two approaches to

multivariate analysis discussed here. Although a couple of other options are

possible, principal components analysis almost always is based on correla-

tion coefﬁcients between variables. If you don’t say otherwise, this is what the

reader will assume. Even though little has been made here of the difference

between principal components analysis and factor analysis, some readers will

be much more concerned about the distinction, so it is important to pay atten-

tion to the options in your statpack and be sure to report accurately which you

did. Providing the actual list of component loadings is an essential part of pre-

senting the results. Just a description of the patterns observed on them is not

enough.

As with multidimensional scaling, it is a good idea to limit the number of

variables in a principal components analysis to no more than about half the

number of cases. If the number of variables is much larger than this, there is

substantial risk of ﬁnding spurious patterns that are no more than the product

of random noise in the data.

really doesn’t matter which you choose. If some details differ between the two, then

perhaps these details should be taken with a grain of salt.

Principal components analysis is extremely powerful, just as regression analy-

sis is. Much of this power comes from the speciﬁcity and rigidity of the model of

relationships between variables, just as it does in regression analysis. Except for

the possibility of inexplicable discrepancies between unrotated and rotated compo-

nents, the results of principal components analysis tend to be clear and unequivocal.

A principal components analysis will always produce components of some sort,

whether there is much strong patterning in the dataset or not. In this sense it cannot

fail, as multidimensional scaling can fail to produce any intelligible patterns. The

signs that there just may not be much useful patterning in a dataset are easier to

overlook in principal components analysis. If labeling the cases provides intuitive

knowledge, then ﬁnding patterning in a multidimensional scaling is likely to be quite

straightforward. On the other hand, if looking at the behavior of variables provides

the most intuitive knowledge, as it does in the Ixcaquixtla household analysis, then

presenting the multidimensional scaling in an intelligible way requires a good bit

more work. With principal components analysis, it is just the reverse. If the behavior

of variables provides the most direct route into identifying meaningful patterns, then

principal components is certainly less work than multidimensional scaling because

its natural way of presenting its results is organized around the variables.

Chapter 25

Cluster Analysis

Single Linkage Clustering ......................................................................... 310

Complete Linkage Clustering ...................................................................... 312

Average Linkage Clustering ....................................................................... 313

Which Linkage Criterion to Choose ............................................................... 315

How Many Clusters to Deﬁne ..................................................................... 316

Clustering by Variables............................................................................. 316

Clustering the Ixcaquixtla Household Data ....................................................... 318

Cluster analysis is perhaps the most familiar of all approaches to exploratory

multivariate analysis, although it is not always thought of as a multivariate technique

parallel to, for example, multidimensional scaling or principal components analysis.

It is like such approaches, though, in seeking structure in the relationships among

cases characterized by a number of variables. Cases that are strongly similar to each

other, in terms of their values for a number of variables, wind up in the same groups

or clusters, while those that are more different from each other wind up in differ-

ent clusters. Cluster analysis mimics one of the human mind’s fundamental ways

of dealing with complicated variability: categorizing, or putting things into groups.

Artifact typology in archaeology is a very familiar example of such categorizing.

Recognizing that no two artifacts are likely to be identical, but that some pairs are

more similar than others, we put the more similar ones together subjectively into

what we then deﬁne as types. Our artifact typologies are hierarchical in that they

group artifacts ﬁrst according to broad classes like ceramics, ﬂaked stone, textiles,

etc., and then, within these broad classes, into more speciﬁc types at perhaps several

levels. Flaked stone, for example, may be divided into tools and debitage; tools, in

turn, into unifaces and bifaces; unifaces, into scrapers, blades, burins, etc.; scrapers,

into endscrapers and sidescrapers; and so on.

This kind of hierarchical clustering can also be accomplished by statistical

(as opposed to purely subjective) means. The ﬁrst step in a hierarchical cluster anal-

ysis is usually the same as the ﬁrst step in multidimensional scaling: measuring the

similarities between each pair of cases in the dataset. The coefﬁcients of similarity

(or dissimilarity) that were discussed in Chapter

22 are just as suitable for cluster-

ing. Once the similarities (or dissimilarities or distances) have been measured, the

clustering can begin.

R.D. Drennan, Statistics for Archaeologists, Interdisciplinary Contributions

to Archaeology, DOI 10.1007/978-1-4419-0413-3

25,

 Springer Science+Business Media, LLC 2004, 2009

309

310 CHAPTER 25

Hierarchical cluster analysis is most often agglomerative because it usually

proceeds by combining individual cases together to form larger and larger clus-

ters. The procedure begins with each case considered as a separate entity. At the

ﬁrst step of this multistep procedure, the two most similar cases are combined into

a cluster. At the next step, either two other cases are combined to begin a second

cluster or else a third case is added to the existing cluster. Step by step, the cluster-

ing procedure continues to higher and higher (or more and more inclusive) levels in

the hierarchy of agglomeration until ﬁnally all cases are combined into a single big

cluster.

There are three major variations on this basic theme of hierarchical clustering,

involving different speciﬁc clustering criteria. They arise because, as individual

cases are gradually combined into more and more inclusive clusters, the proce-

dure must choose between combining two individual cases to initiate a new cluster,

adding a case to an existing cluster, or joining together two already existing clusters.

While similarities have already been measured for all possible pairs of individ-

ual cases, the question arises of just how to use these in measuring the similarity

between a case and an already existing cluster or between two already existing clus-

ters. There may, for example, be a very strong similarity between a case in one

cluster and a case in another cluster but very little similarity between the other pairs

of cases involved in the two clusters.

SINGLE LINKAGE CLUSTERING

The simplest approach is single linkage clustering, in which the strongest single sim-

ilarity score between cases governs each step in clustering. For example, using the

matrix of similarity coefﬁcients in Table

25.1, the sequence of single link clustering

would go as follows:

1. The strongest single similarity in the matrix is 0.96 between Cases 4 and 6, so

these two cases would be combined into a cluster.

2. The next strongest similarity in the matrix is 0.95 between Cases 1 and 3, so

these two cases would be combined into a second cluster.

Table 25.1. Matrix of Similarity Coefﬁcients for Seven Cases

1234567

11.00

2 0.34 1.00

3 0.95 0.22 1.00

4 0.69 0.04 0.11 1.00

5 0.87 0.90 0.75 0.63 1.00

6 0.12 0.15 0.37 0.96 0.27 1.00

7 0.86 0.76 0.32 0.59 0.43 0.49 1.00

CLUSTER ANALYSIS 311

3. The next strongest similarity in the matrix is 0.90 between Cases 2 and 5, so

these two cases would be combined into a third cluster.

4. The next strongest similarity in the matrix is 0.87 between Cases 1 and 5. Case 1

already belongs to a cluster (with Case 3) and Case 5 already belongs to a cluster

(with Case 2), so these two clusters would be combined to form a cluster of four

cases.

5. The next strongest similarity in the matrix is 0.86 between Cases 1 and 7. Case 1

already belongs to a cluster (with Cases 2, 3, and 5), so Case 7 would be added

to that cluster, enlarging it to ﬁve cases.

6. The next strongest similarity in the matrix is 0.76 between Cases 7 and 2. These

two already belong to the same cluster so there is no additional joining at this

point.

7. The next strongest similarity in the matrix is 0.75 between Cases 3 and 5. These

are also already members of the same cluster.

8. The next strongest similarity in the matrix is 0.69 between Cases 1 and 4. Case

1 already belongs to a cluster (with Cases 2, 3, 5, and 7), and Case 4 already

belongs to a cluster (with Case 6), so these two clusters would be joined together

to form a cluster of seven cases.

At this point all the cases have been joined into a single cluster, and the procedure is

ﬁnished. The dendrogram in Fig.

25.1 provides a complete account of the entire pro-

cedure. The joining steps can be read from left to right. That is, the leftmost vertical

line joining two cases represents the ﬁrst joining step, and each step can be read in

succession by selecting vertical joining lines in a steady rightward progression. The

strength of similarity at any particular step in joining can be read from the horizontal

scale at the top of the dendrogram.

Single linkage clustering often joins clusters together even though a number of

individual cases from the two clusters show very little similarity to each other. That

is, a single strong similarity for one pair of cases can cause two clusters to be joined

even if all the other cases involved are quite different from each other. In the example

used above, for instance, the cluster consisting of Cases 1 and 3 was joined with the

cluster consisting of Cases 2 and 5 at Step 4 because of the strong similarity (0.85)

Figure 25.1. Single linkage clustering of the similarity scores from Table 25.1.

312 CHAPTER 25

between Cases 1 and 5, even though the new cluster formed incorporated cases that

were often not measured to be very similar. The similarity between Cases 1 and 2

is only 0.34 and between Cases 2 and 3 is only 0.22. Yet these pairs of cases were

included together in the newly formed cluster. This would seem to make little sense,

so clustering criteria other than single linkage are often used.

COMPLETE LINKAGE CLUSTERING

Complete linkage clustering prevents the undesirable joining of clusters with dis-

similar members. With complete linkage, no two clusters are joined together unless

even the weakest similarity between any two of the cases involved is stronger than

any other “unused” similarity score in the matrix. The sequence of complete linkage

clustering of the similarity scores from Table

25.1 would be as follows:

1.–3. The ﬁrst three steps would be the same as with single linkage.

4. The next strongest similarity in the matrix is 0.87 between Cases 1 and 5.

Case 1 already belongs to a cluster (with Case 3) and Case 5 already belongs

to a cluster (with Case 2), so we would need to examine the other similarities

involved in combining these two clusters. Cases 3 and 5 have a similarity of

0.75; Cases 1 and 2, of 0.34; and Cases 2 and 3, of 0.22. The two clusters will

not be combined unless all the other possible combinations at this stage would

require combining even less similar cases. All the possible combinations (and

the weakest similarities each would incorporate) are as follows:

1/3 with 7 (a similarity of 0.32 between 3 and 7);

1/3 with 4/6 (a similarity of 0.11 between 3 and 4);

1/3 with 2/5 (a similarity of 0.22 between 2 and 3);

2/5 with 7 (a similarity of 0.43 between 5 and 7);

2/5 with 4/6 (a similarity of 0.04 between 2 and 4); and

4/6 with 7 (a similarity of 0.49 between 6 and 7).

Combining 7 with the cluster of 4 and 6 puts together less dissimilar cases

than any other available combination, so step 4 makes the cluster 4/6/7.

5. At this point there are three clusters, made up of Cases 1/3, 2/5, and 4/6/7.

Cluster 1/3 is held back from joining Cluster 2/5 by a weak similarity of

0.22 between Cases 2 and 3. The only other two possible joining steps, how-

ever, are impeded by even weaker similarities. The similarity of 0.11 between

Cases 3 and 4 holds Clusters 1/3 and 4/6/7 apart, and the similarity of 0.04

between Cases 2 and 4 holds Clusters 2/5 and 4/6/7 apart. Thus, the proce-

dure would work its way down to the similarity of 0.22 as the next strongest

combination, and would put Clusters 1/3 and 2/5 together as the ﬁfth step.

6. The last joining would unite Cluster 1/2/3/5 with Cluster 4/6/7 at the weakest

level of 0.04 (the similarity between Cases 2 and 4).

This history of clustering and its outcome are represented in the dendrogram of

Fig.

25.2. Complete linkage clustering can be seen to represent the opposite extreme

CLUSTER ANALYSIS 313

Figure 25.2. Complete linkage clustering of the similarity scores from Table 25.1.

to single linkage clustering. Instead of relying on the single strongest link between

clusters (as with single linkage), it relies on the single weakest link to determine

what to join. The effects of the two criteria can be appreciated by imagining each

cluster as a club whose members are deciding which applicants to accept. The effect

of complete linkage clustering is the same as the system of “blackballing” applicants

for membership. Any case can prevent the joining of any other case with which it has

a very low similarity, just as in a club where any single member can veto a disliked

applicant. Single linkage clustering, however, is just the opposite: a new member

is accepted into the club on the basis of a strong similarity score from any current

member, regardless of how low other members’ scores for that applicant may be.

AVERAGE LINKAGE CLUSTERING

Just as clubs may adopt procedures for accepting new members that fall somewhere

between the two extremes, average linkage clustering has been proposed as a happy

medium between single and complete linkage. In average linkage clustering, after

each joining step a new matrix of similarities is calculated, treating each existing

cluster as if it were a single case. The similarity between an existing cluster and

another case is the average of the similarities between that case and each member of

the cluster. For the similarity coefﬁcients from Table

25.1, average linkage clustering

would proceed as follows:

1. Cases 4 and 6, with a similarity of 0.96, would join. The similarity matrix would

be recalculated, with the result in Table

25.2. The similarity score between Clus-

ter 4/6 and Case 1, for example, would be the average of the similarity scores

between Cases 1 and 4 and Cases 1 and 6, or: 0.69+ 0.12

2 = 0.41.

2. Cases 1 and 3, with a similarity of 0.95, would combine, and the similarity matrix

would be recalculated again, with the result in Table

25.3. The similarity between

Clusters 1/3 and 4/6 would be the average of the similarities between the four

pairs of cases involved (1 and 4, 1 and 6, 3 and 4, and 3 and 6).

3. Cases 2 and 5, with a similarity of 0.90, would combine, and the similarity matrix

would be recalculated again, with the result in Table

25.4. (The ﬁrst three steps,

314 CHAPTER 25

Table 25.2. Matrix of Similarity Coefﬁcients after the First Step in

Average Link Clustering

1234/657

11.00

2 0.34 1.00

3 0.95 0.22 1.00

4/6 0.41 0.10 0.24 1.00

5 0.87 0.90 0.75 0.45 1.00

7 0.86 0.76 0.32 0.54 0.43 1.00

Table 25.3. Matrix of Similarity Coefﬁcients after the

Second Step in Average Link Clustering

1/3 2 4/6 5 7

1/3 1.00

2 0.28 1.00

4/6 0.32 0.10 1.00

5 0.81 0.90 0.45 1.00

7 0.59 0.76 0.54 0.43 1.00

Table 25.4. Matrix of Similarity Coefﬁcients after

the Third Step in Average Link Clustering

1/3 2/5 4/6 7

1/3 1.00

2/5 0.56 1.00

4/6 0.32 0.27 1.00

7 0.59 0.60 0.54 1.00

Table 25.5. Matrix of Similarity Coefﬁcients

after the Fourth Step in Average Link

Clustering

1/3 2/5/7 4/6

1/3 1.00

2/5/7 0.56 1.00

4/6 0.32 0.36 1.00

Table 25.6. Matrix of Similarity Coefﬁcients after the

Fifth Step in Average Link Clustering

1/2/3/5/7 4/6

1/2/3/5/7 1.00

4/6 0.35 1.00

CLUSTER ANALYSIS 315

Figure 25.3. Average linkage clustering of the similarity scores from Table 25.1.

then, turn out to be the same for average linkage as they were for both single and

complete linkage.)

4. Case 7 would join with Cluster 2/5, based on their similarity score of 0.60, and the

similarity matrix would be recalculated yet again, with the result in Table

25.5.

5. The highest similarity score in the new matrix, 0.56, would cause Cluster 1/3 to

join with Cluster 2/5/7, and the similarity matrix would be recalculated one last

time, with the result in Table

25.6.

6. The ﬁnal two clusters would be joined together at a similarity level of 0.35.

This clustering sequence and its outcome are illustrated in the dendrogram in

Fig.

25.3. Minor variations in average linkage clustering can be produced by calcu-

lating the new similarity matrix in different ways – by using the median similarity

coefﬁcient instead of the average, for example, or by calculating some other index

of the center of the scores involved (sometimes referred to as a centroid).

WHICH LINKAGE CRITERION TO CHOOSE

Comparison of the three dendrograms in Figs. 25.1–25.3 shows the different con-

sequences of the three different linkage criteria. Sometimes the difference is quite

dramatic. This naturally raises the question of which linkage criterion should be

chosen. There is no simple principle that can be routinely applied to answer this

question.

Occasionally the nature of the dataset suggests a particular linkage criterion as

the most appropriate. For example, hierarchical clustering is frequently used in raw

material sourcing studies. Here the cases are typically artifacts that have been sub-

jected to some form of chemical analysis, and the variables are some measure of the

abundance of different chemical elements or other constituents. Hierarchical clus-

tering may be used to delineate groups of artifacts that are presumed to be made

of materials from the same source location. If we expect the raw material involved

to have pretty much the same composition in a given source, then complete link-

age clustering makes good sense. This is because complete linkage clustering will

prevent assigning an artifact to a cluster unless it is broadly similar to each of the