Kline R.B. Principles and Practice of Structural Equation Modeling

Подождите немного. Документ загружается.

364 ADVANCED TECHNIQuES, AVOIDING MISTAKES

tic information about possible sources of model–data disagreement. Any respeciﬁcation

suggested by the diagnostics must be theoretically justiﬁable. If such respeciﬁcations

result in subsequently passing the chi-square test, then diagnostic information about

the ﬁt of the respeciﬁed model is still needed. This is because a passing model can still

have ﬁt problems concerning its explanations of the observed covariances. If this diag-

nostic assessment indicates a problem, then the model should be rejected despite pass-

ing the chi-square test.

40. Rely solely on suggested thresholds for approximate ﬁt indexes to justify retaining

the model. Results of recent studies indicate that rules of thumb from the 1980s–1990s

about cutoffs for approximate ﬁt indexes that supposedly indicate “acceptable” ﬁt are not

trustworthy. This mistake is compounded when (a) the model fails the chi-square test

when the sample size is not large and (b) the researcher neglects to report diagnostic

information about model ﬁt.

41. Interpret “closer to ﬁt” as “closer to truth.” Close model–data correspondence

could reﬂect any of the following (not all mutually exclusive) possibilities: (a) the model

accurately reﬂects reality; (b) the model is an equivalent or near-equivalent version of

one that corresponds to reality but itself is incorrect; (c) the model ﬁts the data in a non-

representative sample but has poor ﬁt in the population; or (d) the model has so many

freely estimated parameters that it cannot have poor ﬁt even if it is grossly misspeciﬁed.

In a single study, it is usually impossible to determine which of these scenarios explains

the good ﬁt of the researcher’s model. This is another way of saying that SEM is more

useful for rejecting a false model than for somehow “proving” whether a given model is

true (point 5).

42. Interpret good ﬁt as meaning that the endogenous variables are strongly predicted.

If the exogenous and mediator variables account for a small proportion of the variances

of the ultimate outcome variables and a model accurately reﬂects this lack of predictive

validity, then the overall ﬁt of the model may be good. Fit statistics in SEM indicate

whether the model can reproduce the observed covariances, not whether substantial

proportions of the variance of the endogenous variables are explained.

43. Rely solely on statistical criteria in model evaluation. Other important consid-

erations include model generality, parsimony, and theoretical plausibility. As noted by

Robert and Pashler (2000), good statistical ﬁt of a model indicates little about (a) theory

ﬂexibility (e.g., what it cannot explain), (b) variability of the data (e.g., whether the data

can rule out what the theory explain cannot explain), and (c) the likelihood of other out-

comes. These authors also suggest that a better way to evaluate a model is to determine

(a) how well the theory limits outcomes (i.e., whether it can accurately predict), (b) how

closely the actual outcome agrees with those limits, and (c) if plausible alternative out-

comes would have been inconsistent with the theory (Sikström, 2001). That is, whether

a model is statistically beautiful involves not just numbers, but ideas, too.

44. Rely too much on statistical tests. This entry covers several kinds of errors. One is

to interpret statistical signiﬁcance as evidence for effect size (especially in large samples)

or for importance (i.e., substantive signiﬁcance). Another is to place too much emphasis

on statistical tests of individual parameters that may not be of central interest in hypoth-

How to Fool Yourself with SEM 365

esis testing (e.g., whether an error variance differs statistically from zero). A third is

to forget that statistical tests of individual effects tend to result in rejection of the null

hypothesis too often when non-normal data are analyzed by methods that assume nor-

mality. See point 27 for related misuses of statistical tests in SEM.

45. Interpret the standardized solution in inappropriate ways. This is a relatively com-

mon mistake in multiple-sample SEM—speciﬁcally, to compare standardized estimates

across groups that differ in their variabilities. In general, standardized solutions are ﬁne

for comparisons within each group (e.g., the relative magnitudes of direct effects on the

same endogenous variable), but only unstandardized solutions are usually appropriate

for cross-group comparisons. A related error is to interpret group differences in the stan-

dardized estimates of equality-constrained parameters: the unstandardized estimates of

such parameters are forced to be equal, but their unstandardized counterparts are typi-

cally unequal if the groups have different variabilities.

46. Fail to consider equivalent or near-equivalent models. Essentially all structural

equation models have equivalent versions that generate the same predicted correlations

or covariances. For latent variable models, there may be inﬁnitely many equivalent mod-

els. There are probably also near-equivalent versions that generate almost the same cova-

riances as those in the data matrix. Researchers must offer reasons why their models are

to be preferred over some obvious equivalent or near-equivalent versions of them.

47. Fail to consider (nonequivalent) alternative models. When there are competing

theories about the same phenomenon, it may be possible to specify alternative models

that reﬂect them. Not all of these alternatives may be equivalent versions of one another.

If the overall ﬁts of some of these alternative models are comparable, then the researcher

must explain why a particular model is to be preferred.

48. Reify the factors. Believe that constructs represented in your model must corre-

spond to things in the real world. Perhaps they do, but do not assume it.

49. Believe that naming a factor means that it is understood (i.e., commit the naming

fallac y). Factor names are conveniences, not explanations. For example, if a three-factor

ﬁts the data, this does not prove that the verbal labels assigned by the researcher to the

factors are correct. Alternative interpretations of factors are often possible in many, if

not most, factor analyses.

50. Believe that a strong analytical method like SEM can compensate for poor study

design or slipshod ideas. No statistical procedure can make up for inherent logical or

design ﬂaws. For example, expressing poorly thought out hypotheses with a path dia-

gram does not give them more credibility. The speciﬁcation of direct and indirect effects

in a structural model cannot be viewed as a replacement for an experimental or longi-

tudinal design. As mentioned earlier, the inclusion of a measurement error term for an

observed variable that is psychometrically deﬁcient cannot somehow transform it into

a good measure. Applying SEM in the absence of good design, measures, and ideas is

like using a chain saw to cut butter: one will accomplish the task, but without a more

substantial base, one is just as likely to make a big mess.

51. As the researcher, fail to report enough information so that your readers can repro-

duce your results. There are still too many reports in the literature where SEM was used

366 ADVANCED TECHNIQuES, AVOIDING MISTAKES

in which the authors do not give sufﬁcient summary information for readers to re- create

the original analyses or evaluate models not considered by the authors. At minimum,

authors should generally report all relevant correlations, standard deviations, and

means. Also describe the speciﬁcation of the model(s) in enough detail so that a reader

can reproduce the analysis.

52. Interpret estimates of relatively large direct effects in a structural model as “proof” of

causality. As discussed earlier, it would be almost beyond belief that all of the conditions

required for inference of causality from covariances have been met in a single study. In

general, it is better to view structural models as being “as if” models of causality that

may or may not correspond to causal sequences in the real world.

suMMarY

So concludes this journey of discovery about SEM. As on any guided tour, you may have

found some places along the way more interesting than others. Also, you may decide

to revisit certain sites by using some of the related techniques in your work. Overall, I

hope that reading this book has given you new ways of looking at your data and testing

a broader range of hypotheses. Use SEM to address new questions or to provide new

perspectives on older ones, but use it guided by your good sense and knowledge of your

research area. Use it also as a means to reform methods of data analysis in the behavioral

sciences by focusing more on models instead of speciﬁc effects analyzed with traditional

statistical signiﬁcance tests. The American politician Ivy Baker Priest once said: The

world is round and the place which may seem like the end may also be the beginning.

Go do yourself (and me, too) proud!