Kline R.B. Principles and Practice of Structural Equation Modeling

Подождите немного. Документ загружается.

354 ADVANCED TECHNIQuES, AVOIDING MISTAKES

suMMarY

In moderated path analysis, the interactive effects of observed variables are represented

by product terms included in the model along with the main effect terms. Path coef-

ﬁcients for direct effects of the product terms estimate the corresponding interaction

effects. One way to interpret an interaction between two continuous predictors is to gen-

erate the simple regressions of the outcome variable on one predictor at different levels of

the other predictor. One of the ﬁrst approaches to estimating interactive effects of latent

variables is the Kenny–Judd method, which features all possible product indicators of

latent product variables, the imposition of nonlinear constraints, and the assumption

of multivariate normality. Some alternative methods do not require all possible prod-

uct indicators, nonlinear constraints, or the normality assumption. Large samples are

needed in all these methods regardless of the estimation method.

Researchers who know something about both SEM and multilevel modeling can

test an even wider range of hypotheses compared with those who know about one

technique, but not the other. The convergence of the two techniques in the form of

multilevel SEM offers the capability to (1) calculate correct standard errors in hier-

archical datasets; (2) analyze predictors from both the individual level and the group

level (contextual effects) in the same analysis; (3) take account of unreliability when

latent variables are represented as measured by multiple indicators; and (4) estimate

both direct and indirect effects when structural models are analyzed. The increasing

availability of SEM computer tools that directly support multilevel analyses in complex

sampling designs is making it easier for researchers to actually reap these potential

beneﬁts.

reCoMMended readIngs

Interaction effects

Preacher et al. (2007) describe the estimation and interpretation of conditional indirect effects

in moderated path models. The edited volume by Schumaker and Marcoulides (1998) is a

good resource in this area, and Marsh, Wen, and Hau (2006) describe additional estimation

methods.

Marsh, H. W., Wen, Z., & Hau, K.-T. (2006). Structural equation modeling of latent interaction

and quadratic effects. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation model-

ing: A second course (pp. 225–265). Greenwich, CT: Information Age Publishing.

Preacher, K. J., Rucker, D. D., & Hayes, A. F. (2007). Addressing moderated mediation hypoth-

eses: Theory, methods, and prescriptions. Multivariate Behavioral Research, 42, 185–

2 2 7.

Schumacker, R. E., & Marcoulides, G. A. (Eds.). (1998). Interaction and nonlinear effects in

structural equation modeling. Mahwah, NJ: Erlbaum.

Interaction Effects and Multilevel SEM 355

Multilevel analysis

Bickel (2007) introduces multilevel analysis by comparing it with standard multiple regression.

A more advanced work for readers with strong quantitative backgrounds is Raudenbush and

Bryk (2002). Stapleton (2006) offers a clear introduction to multilevel analysis in SEM.

Bickel, R. (2007). Multilevel analysis for applied research: It’s just regression! New York: Guil-

ford Press.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models (2nd ed.). Thousand

Oaks, CA: Sage.

Stapleton, L. M. (2006). Using multilevel structural equation modeling techniques with complex

sample data. In G. R. Hancock and R. O. Mueller (Eds.), Structural equation modeling: A

second course (pp. 345–383). Greenwich, CT: Information Age Publishing.

356

How to Fool Yourself with SEM

The family of techniques that make up SEM provide researchers with a marvelously

ﬂexible set of tools for hypothesis testing. But as with any complex set of procedures,

its use must be guided by reason. Although many ways to mislead yourself with SEM

were mentioned in previous chapters, they are discussed altogether here. Potential

pitfalls are considered under four categories that correspond to major phases of the

analysis: speciﬁcation, data preparation, analysis and respeciﬁcation, and interpreta-

tion. These categories are not mutually exclusive, but they correspond to the usual

sequence in which researchers should address these issues. You are encouraged to

use the points presented next as a checklist to guide the conduct of your own analyses.

Here they are: 52 ways to take leave of your senses in SEM. This list is not exhaustive,

but it contains many of the more common mistakes. I thank members of SEMNET for

their comments on an earlier version of this list.

trIPPIng at the startIng lIne: sPeCIFICatIon

Despite all the statistical machinations of SEM, speciﬁcation is the most important part

of the process. Occasionally, however, researchers spend the least amount of time on it.

Listed here are several ways not to do your homework in this critical area:

1. Specify the model after the data are collected rather than before. This case concerns

the speciﬁcation of a model for an archival data set. Potential problems caused by plac-

ing the data cart before the theory horse are described under points that follow, but

they include the realization that key variables are omitted or the model is not identiﬁed.

With the data already collected, it may be too late to do anything about the identiﬁca-

tion problem. Also, adding exogenous variables in one way to remedy an identiﬁcation

problem for a nonrecursive structural model and adding indicators can help to identify

a measurement model.

How to Fool Yourself with SEM 357

2. Omit causes that are correlated with other variables in a structural model. If an

omitted cause is uncorrelated with causes already represented in the structural model,

then estimates of direct effects are not biased because of this omission. It is rare, how-

ever, that the types of causal variables studied by behavioral scientists are independent.

Depending on the pattern of correlations between an omitted variable and those in the

model, estimates of causal effects can be too high or too low.

3. Fail to include unanalyzed associations between pairs of measured exogenous vari-

ables in path models or exogenous factors in structural regression models. Recall that these

associations are represented by the symbol

in model diagrams and reﬂect the fact

that the model offers no explanation of why exogenous variables covary (and vary, too,

represented by the symbol

). Because most causal variables are correlated, omitting

these paths is a speciﬁcation error. Even if these intercorrelations are relatively small,

their omission could adversely affect the estimation of other model parameters (i.e.,

error propagation).

4. Use psychometrically inadequate measures. The analysis of variables with a lot of

measurement error in the scores (e.g., unreliability) can lead to inaccurate results. The

general effect of measurement error is to underestimate causal effects, but—depending

on the intercorrelations—estimates can be too high. Although measurement error is

taken into account in the analysis of a measurement model or structural regression

model, estimates about latent variables are more precise when the indicators are psycho-

metrically sound.

5. Fail to give careful consideration to the question of directionality. Directionality

is a critical feature not only of structural models but also of measurement models. In

structural models, speciﬁcations of direct and indirect effects are statements about the

expected sequence of causality, or effect priority. Assumptions about directionality in

measurement models are expressed by the speciﬁcation of observed variables as either

effect indicators (reﬂective measurement) or cause indicators (formative measurement).

If solid reasons cannot be provided for the speciﬁcation of directionality, then either use

another type of statistical procedure (e.g., multiple regression) or test alternative models

with different causal sequences. However, some alternative models may be equivalent

or near-equivalent models that generate the same or very similar predicted covariances,

which means that there is no way to statistically distinguish them. This is why SEM is

described as being good for estimating effects when the underlying causal mechanism

is known. It is also more a disconﬁrmatory method than a conﬁrmatory one because

SEM helps researchers to detect false models, but it basically never conﬁrms whether a

retained model is true.

6. Specify feedback effects in structural models (e.g., Y



) as a way to mask

uncertainty about directionality. Not only do feedback relations have their own assump-

tions (e.g., equilibrium), but their presence also makes a structural model nonrecur-

sive, which introduces potential problems (e.g., identiﬁcation) in analyzing the model. A

related mistake is to fail to rationally evaluate the equilibrium assumption for a feedback

loop.

7. Add disturbance or measurement error correlations without substantive reason. This

358 ADVANCED TECHNIQuES, AVOIDING MISTAKES

is a variation on the previous point. When there is justiﬁcation (e.g., repeated measure-

ment, two indicators share a common method), speciﬁcation of these types of unana-

lyzed associations is appropriate. Otherwise, they can be a way to improve ﬁt simply by

making a model more complex. This is especially true if the researcher initially speciﬁes

uncorrelated residuals because of the assumption of conditional independence of the

indicators for a latent variable model. If the model in fact does not adequately explain

the observed associations among the indicators, adding measurement error correlations

without substantive reason can be a way to mask this shortcoming.

8. Fail to include design-driven correlated residuals. This mistake is the ﬂip side of

the one just described, and it is the failure to include theoretically justiﬁable correlated

residuals, given measurement theory or research design. Cole, Ciesla, and Steiger (2007)

remind us that the omission of such residual correlations may not in some cases harm

model ﬁt, but their omission could change the meaning of latent variables and thus lead

to inaccurate results. In other disciplines such as econometrics, the speciﬁcation of cor-

related residuals is more routine.

So the speciﬁcation of correlated residuals should not

be seen as a necessary evil. Instead, they should be included in the model when there is

justiﬁcation. In this sense, correlated residuals are no different from any other type of

effect that could potentially be included in structural equation models.

9. Overﬁt the model (i.e., forget the goal of parsimony). Any model, even theoretically

nonsensical ones, will perfectly ﬁt the data if they are speciﬁed to be just as complex

as possible (i.e., df

= 0). But models just as complex as the data test no particular

hypothesis. It is only more parsimonious models, in which some effects are intentionally

constrained to zero (df

> 0), that allow tests of speciﬁc ideas. The goal of parsimony

is also important in respeciﬁcation. Here one must be careful not to modify the model

solely for the sake of improving ﬁt. These comments on parsimony are not intended to

dissuade you from analyzing complex models per se. This is because a phenomenon that

is complex in the real world may require a relatively complex statistical model in order

to reﬂect it basic essence. The main point is that the model should be as parsimonious as

possible while respecting theory and results of prior empirical studies. Models that are

complex without theoretical justiﬁcation are probably so speciﬁed in order to maximize

ﬁt (i.e., it capitalizes on sample-speciﬁc variance), but such models are unlikely to repli-

cate.

10. Fail to have sufﬁcient numbers of indicators of latent variables. Measurement mod-

els with more than one factor typically require only two indicators per factor for iden-

tiﬁcation. However, having only two indicators per factor may lead to problems. Such

models may be more likely to be empirically underidentiﬁed than models with at least

three indicators per factor. Other problems, such as nonconvergence of iterative estima-

tion, are more likely to occur for models with only two indicators per factor, especially

in small samples. It may be difﬁcult to estimate measurement error correlations for

factors with only two indicators, which can result in a speciﬁcation error. Parts of the

model where some factors have only two indicators are not “self-sufﬁcient” in terms of

W. Wothke, personal communication, November 24, 2003.

How to Fool Yourself with SEM 359

identiﬁcation, which means that they have to “borrow” covariance information from

other parts of the model. This may result in propagation of error from one part of the

model to another. Suppose that the measurement error correlation for a factor with just

two indicators is really substantial, but it cannot be estimated because of identiﬁcation.

This speciﬁcation error may propagate to estimation of the factor loadings for this pair

of indicators.

Two indicators per factor is the technical minimum, but at least three or

four indicators per factor is a better target.

11. Specify that indicators load on > 1 factor without a substantive reason. The speci-

ﬁcation that an indicator depends on more than one factor may be appropriate if you

really believe that it measures more than one construct. Just like measurement error

correlations, though, adding factor loadings makes a measurement model less parsimo-

nious.

12. Specify that a set of effect indicators with low intercorrelations loads on a com-

mon factor. The speciﬁcation of effect indicators implies reﬂective measurement, which

assumes that a set of effect indicators all measure the same underlying factor. This in turn

means that their intercorrelations should all be positive and relatively high (e.g., > .50).

If the assumptions of reﬂective measurement are not tenable, then consider the speciﬁ-

cation of formative measurement where indicators are speciﬁed as causes of composites.

Of course, the speciﬁcation of reﬂective versus formative measurement requires a theo-

retical basis.

13. In a complex sampling design, assume that the within-group model and the between-

group model are the same without veriﬁcation. One lesson of multilevel modeling is that

different models may describe covariance patterns at different levels of analysis, within

versus between. Without a basic understanding of statistical techniques for hierarchical

data, including multilevel SEM, the researcher could miss this possibility.

14. Forget that the main goal of speciﬁcation is to test a theory, not a model. The model

analyzed in SEM represents predictions based on a particular body of theory or results

of prior empirical studies. Outside this role, the model has no intrinsic value. That is, it

provides a vehicle for testing ideas, and the real goal of SEM is to evaluate these ideas in

a scientiﬁcally meaningful and valid way. Whether or not a model is retained is inciden-

tal to this purpose.

IMProPer Care and FeedIng: data

The potential missteps presented in this section involve leaping before you look, that is,

not carefully screening the data before analyzing them:

15. Don’t check the accuracy of data input or coding. Data entry mistakes are so easy

to make, whether in recording the raw data or in typing the values of a correlation or

B. Muthén, personal communication, November 25, 2003.

360 ADVANCED TECHNIQuES, AVOIDING MISTAKES

covariance matrix. Even machine-based data entry is not error free (e.g., smudges on

forms can “fool” an electronic scanner, software errors can result in the calculation of

incorrect scores). Mistaken speciﬁcation of codes in statistical programs is also common

(e.g., “9” for missing data instead of “–9”).

16. Ignore whether the pattern of missing data loss is random or systematic. This point

assumes that there are more than just a few missing scores. Classical statistical methods

for dealing with incomplete data, such as case deletion or single-imputation methods,

generally assume that the data loss pattern is missing completely at random, which is

unlikely in perhaps most data sets analyzed in the behavioral sciences. These classical

techniques have little basis in statistical theory and take little advantage of structure in

the data. More modern methods, including those that impute multiple scores for miss-

ing observations based on predictive theoretical distributions, generally assume that

the data loss pattern is missing at random, a less strict assumption about randomness.

But even these methods may generate inaccurate results if the data loss mechanism is

systematic. If so, then (1) there is no “statistical ﬁx” for the problem, and (2) you need to

explicitly qualify the interpretation of the results based on the data loss pattern.

17. Fail to examine distributional characteristics. The most widely used estimation

methods in SEM, including maximum likelihood (ML), assume multivariate normal dis-

tributions for continuous endogenous variables. Although values of parameter estimates

are relatively robust against non-normality, statistical tests of individual parameters

tend to be positively biased (i.e., Type I error rate is inﬂated). If the distributions of con-

tinuous endogenous variables are severely non-normal, then use an estimation method

that does not assume normality or use corrected statistics (e.g., robust standard errors,

corrected model test statistics) when normal theory methods such as ML estimation

are used. If the distributions are non-normal because the indicators are discrete with

a small number of categories (i.e., they are ordered-categorical variables), then use an

appropriate method for this type of data, such as robust weighted least squares (WLS).

18. Don’t screen for outliers. Even a few extreme scores in a relatively small sample

can distort the results. If it is unclear whether outlier cases are from a different popula-

tion, the analysis can be run with and without these cases in the sample. This strategy

makes clear the effect of outliers on the results. This same strategy can be used to evalu-

ate the effects of different methods to deal with missing data.

19. Assume that all relations are linear. A standard assumption in SEM is that variable

relations are linear. Curvilinear or interactive relations can be represented with product

terms but, in general, such terms must be created by the researcher and then included

in the model. Simple visual scanning of scatterplots can detect bivariate relations that

are obviously curvilinear, but there is no comparably easy visual check for interaction

effects. Model test statistics, including

, are generally insensitive to serious interac-

tion misspeciﬁcation (i.e., there is real interaction, but the model has no corresponding

product terms that represent these effects).

20. Ignore lack of independence among the scores. This problem may arise in two

contexts. First, the scores are from a repeated measures variable. The ability to specify

a model for the error covariances addresses this ﬁrst context. The second context refers

How to Fool Yourself with SEM 361

to hierarchical data structures in which cases are clustered within larger units, such as

employees who work under the same manager. Scores within the larger unit are proba-

bly not independent. The analysis of nested data with statistical techniques that assume

independence may not yield accurate results. Awareness of the possibility to incorporate

multilevel modeling in an SEM analysis helps to avoid this mistake.

CheCkIng CrItICal JudgMent at the door:

analYsIs and resPeCIFICatIon

The potential pitfalls described next concern the analysis and interpretation stages.

However, problems at earlier stages may make these problems more likely to happen:

21. When identiﬁcation status is uncertain, fail to conduct tests of solution uniqueness.

The identiﬁcation of only some types of models can be determined by heuristics with-

out resorting to algebraic manipulation of their equations. If it is unknown whether a

model is theoretically identiﬁed but an SEM computer program yields a converged and

an admissible solution, then the researcher should conduct empirical tests of the solu-

tion’s uniqueness. These tests do not prove that a solution is truly unique, but if they

lead to the derivation of a different solution, then the model is not identiﬁed.

22. Fail to recognize empirical underidentiﬁcation. Estimation of models that are

identiﬁed can nevertheless fail because of data-related problems, including extreme col-

linearity or estimates of key parameters that are close to zero or equal to each other.

Modiﬁcation of a model when the data are the problem may lead to a speciﬁcation

error.

23. Ignore the problem of start values. Iterative estimation may fail to converge

because of poor initial estimates, which is more likely with nonrecursive structural

models or measurement models where some indicators load on multiple factors and with

error correlations. Although many SEM computer programs can automatically generate

their own start values, these values do not always lead to converged admissible solu-

tions, especially for complex models. When this happens, the researcher should try to

generate his or her own initial estimates.

24. Fail to check accuracy of computer syntax. Just as with data entry, it is easy to

make an error in computer syntax that misspeciﬁes the model or data. Although SEM

computer programs have become easier to use, they still cannot generally detect a mis-

take that is logical rather than a syntax error. A logical error does not cause the analy-

sis to fail but instead results in an unintended speciﬁcation (e.g., Y

→ Y

is speciﬁed

when Y

→ Y

is intended). Carefully check to see that the model analyzed was actually

the one that you intended to specify. This is where LISREL’s unique capability to auto-

matically draw the model speciﬁed in your syntax comes in handy: inspection of the

computer-generated diagram gives an opportunity to verify the syntax.

25. Fail to carefully inspect the solution for admissibility. The presence of Heywood

cases or other kinds of illogical results indicates a problem in the analysis. That is, the

362 ADVANCED TECHNIQuES, AVOIDING MISTAKES

solution should not be trusted. For the same reason, avoid making interpretations about

otherwise sensible-looking results in an inadmissible solution.

26. Interpret results from a nonconverged solution or one where the computer imposed a

zero constraint to avoid a Heywood case. This mistake is related to the one just described.

Output from a nonconverged solution is not trustworthy. The same is true when the

computer forces some estimates, such as for error variances, to be at least zero in order

to avoid an illogical result, such as a negative variance estimate. Such solutions are also

untrustworthy.

27. Respecify a model based entirely on statistical criteria. A speciﬁcation search

guided entirely by statistical criteria such as modiﬁcation indexes is unlikely to lead

to the correct model. Use your knowledge of relevant theory and research ﬁndings to

inform the use of such statistics.

28. Analyze a correlation matrix without standard deviations when it is clearly inap-

propriate. These situations include the analysis of a model across independent samples

with different variabilities, longitudinal data characterized by changes in variances over

time, or a type of SEM that requires the analysis of means (e.g., a latent growth model),

which needs the input of not only means but covariances, too.

29. Estimate a covariance structure with a correlation matrix without using proper

methods. Standard ML estimation assumes the analysis of unstandardized variables and

may yield incorrect results when a model is ﬁtted to a correlation matrix without stan-

dard deviations. Appropriate procedures such as the method of constrained estimation

should be used to analyze a correlation matrix when it is not inappropriate to do so (see

the previous point).

30. Fail to check for constraint interaction when testing for equality of loadings across

different factors or of direct effects on different endogenous variables. If the results of the

chi-square difference test for the equality-constrained parameters depend on how the

factors are scaled (i.e., unstandardized vs. standardized), there is constraint interaction.

In this case, it makes sense to analyze the correlation matrix using the method of con-

strained estimation, assuming it is appropriate to analyze standardized variables.

31. Analyze variables so highly correlated that a solution is unstable. If very high cor-

relations (e.g., r > .85) do not cause the analysis to fail or yield a nonadmissible solution,

then extreme collinearity may cause the results to be statistically unstable.

32. Estimate a complex model within a small sample. This is a related problem. As

the ratio of cases to the number of parameters is smaller, the statistical stability of the

estimates becomes more doubtful. Cases-to-free parameter ratios less than 10:1 may be

cause for concern, as are sample sizes < 100. These recommendations assume ML esti-

mation. Some special methods, such as asymptotic distribution free (ADF) methods that

make no distributional assumptions, may require very large samples.

33. Report only standardized estimates. This mistake concerns the possible fooling of

others; that is, always report the unstandardized estimates in a primary analysis. Other-

wise, it may be difﬁcult to compare the results to those from later studies where either

the same or a similar model is estimated in different samples. Along the same lines,

do not associate information about statistical signiﬁcance with standardized estimates

How to Fool Yourself with SEM 363

unless you used a method that calculates correct standard errors in the standardized

solution.

34. Set scales for latent variables inappropriately. In multiple-sample SEM, the tactic

of standardizing factors by ﬁxing their variances to 1.0 is incorrect if groups differ in

their variabilities. Fixing the loading of an indicator to 1.0 (i.e., the factor is unstandard-

ized) is preferable, but note that (a) the same factor loading must be ﬁxed for all groups

and (b) indicators with ﬁxed loadings are assumed to be invariant across all samples. In

single-sample analyses, ﬁxing to 1.0 the variances of factors measured over time is also

inappropriate if factor variability is expected to change (see point 28).

35. Fail to separately evaluate the measurement and structural portions of a structural

regression model. Two-step (or four-step) estimation of structural regression models can

help determine whether the source of poor ﬁt of the original model lies in the measure-

ment component or in the structural component. These sources of poor ﬁt are con-

founded in one-step estimation.

36. Estimate relative group mean or intercept differences on latent variables without

establishing at least partial measurement invariance of the indicators. If the observed vari-

ables do not have the same basic factor structure across all groups, then it makes lit-

tle sense to evaluate relative group mean differences in regression slopes or intercepts

among the latent variables.

37. Analyze parcels of items with Likert-type scales as continuous indicators without

checking to see whether items in each parcel are unidimensional. If a set of items assigned

to the same parcel do not measure one common domain, then analysis of the total score

across the items may not be very meaningful.

the garden Path: InterPretatIon

Potential mistakes described in this section concern the (mis)interpretation of the out-

put from an SEM computer program. Some of these errors may be consequences of mis-

takes listed in earlier sections of this chapter:

38. Look only at values of ﬁt statistics; ignore other types of information about ﬁt. It is

possible that the ﬁt of some portion of the model is poor despite seemingly impressive

values of its average correspondence to the data. Inspection of the magnitudes, direc-

tions, and pattern of correlation residuals can help to spot particular observed associa-

tions that are poorly explained by the model. A related mistake is selective reporting

of ﬁt statistics—speciﬁcally, reporting values of only those ﬁt statistics that favor the

researcher’s model when results on other statistics are clearly less favorable.

39. Ignore a failed model chi-square test. In the past, it was common for researchers

to ignore a statistically signiﬁcant discrepancy between the model and the covariance

data, yet justify retention of the model based on values of approximate ﬁt indexes. This

is bad practice, one that is no longer tolerable. Speciﬁcally, a failed chi-square test should

result in (a) the tentative decision to reject the model and (b) the inspection of diagnos-