Kline R.B. Principles and Practice of Structural Equation Modeling

Подождите немного. Документ загружается.

184 CORE TECHNIQuES

a model with a corrected normal theory method. The program ends with an

error message. Why?

7. Use an SEM computer tool to ﬁt the nonrecursive model in Figure 7.2. to the

data summarized in Table 7.6. Was it necessary for you to specify start val-

ues?

8. Interpret these results: The observed correlation between a pair of endogenous

variables is .41, and the estimated disturbance correlation is .38.

185

APPENDIX 7.A

Start Value Suggestions for Structural Models

These recommendations concern structural models, whether those models are path models or

part of a structural regression model. First, think about the expected direction and magnitude of

standardized direct effects. For example, in some research areas, an absolute standardized direct

effect < .10 may be considered a “smaller” effect; values around .30 a “typical” or “medium” effect;

and values > .50 a “larger” effect. If the numerical values just stated do not match the qualitative

interpretations for “smaller,” “typical,” or “larger” effects, then substitute the appropriate values

(e.g., Bollen, 1989, pp. 137–138). A meta-analytic study is a good way to gauge the magnitude

of “typical” versus “smaller” or “larger” effect sizes in a particular research area. Suppose that a

researcher believes that the direct effect of X on Y is positive and of standardized magnitude, .30.

Thus, a reasonable start value for the unstandardized coefﬁcient for the path X → Y would be .30

(SD

/SD

Start values for disturbance variances can be calculated in a similar way, but now think about

standardized effect size in terms of the proportion of explained variance (i.e., R

). For example,

in some research areas a proportion of explained variance < .01 may indicate a “smaller” effect;

values of about .10 a “typical” or “medium” effect; and values > .30 a “larger” effect. Again, the

numerical values just stated can be adjusted up or down for a particular research area. Suppose that

a researcher believes that the magnitude of the predictive power of all variables with direct effects

on Y (including X) is “typical.” This corresponds to a proportion of explained variance of about

.10 and a proportion of unexplained variance to 1 – .10, or .90. Thus, a reasonable start value for

the disturbance variance would be .90 (

186

APPENDIX 7.B

Effect Decomposition in Nonrecursive Models

and the Equilibrium Assumption

The tracing rule does not apply to nonrecursive structural models. This is because variables in

feedback loops have indirect effects—and thus total effects—on themselves, which is apparent

in effect decompositions calculated by SEM computer programs for nonrecursive models. Con-

sider the reciprocal relation Y



. Suppose that the standardized direct effect of Y

on Y

.40 and that the effect in the other direction is .20. An indirect effect of Y

on itself would be the

sequence

→ Y

which is estimated as .40 × .20, or .08. There are additional indirect effects of Y

on itself through

, however, because cycles of mutual inﬂuence in feedback loops are theoretically inﬁnite. The

indirect effect

→ Y

is one of these, and its estimate is .40 × .20 × .40 × .20, or .0064. Mathematically, these terms head

fairly quickly to zero, but the total effect of Y

on itself is an estimate of all possible cycles through

. Indirect and total effects of Y

on itself are similarly derived.

Calculation of indirect and total effects among variables in a feedback loop as just described

assumes equilibrium (Chapter 5). It is important to realize, however, that there is generally no

statistical way to directly evaluate whether the equilibrium assumption is tenable when the data

are cross-sectional; that is, it must be argued substantively. Kaplan, Harik, and Hotchkiss (2001)

note that rarely is this assumption explicitly acknowledged in the literature on applications of SEM

where feedback effects are estimated with cross-sectional data. This is unfortunate because the

results of a computer simulation study by Kaplan et al. (2001) indicate that violation of the equi-

librium assumption can lead to severely biased estimates. They also found that the stability index

did not accurately measure the degree of bias due to lack of equilibrium. This index is printed in

the output of some SEM computer programs when a nonrecursive model is analyzed. It is based on

certain mathematical properties of the matrix of coefﬁcients for direct effects among all the endog-

enous variables in a structural model, not just those involved in feedback loops. These properties

concern whether estimates of the direct effects would get inﬁnitely larger over time. If so, the sys-

tem is said to “explode” because it may never reach equilibrium, given the observed direct effects

among the endogenous variables. The mathematics of the stability index are complex (e.g., Kaplan

et al., 2001, pp. 317–322). A standard interpretation of this index is that values less than 1.0 are

taken as positive evidence for equilibrium but values greater than 1.0 suggest the lack of equilib-

rium. However, this interpretation is not generally supported by Kaplan and colleagues’ computer

simulation results, which emphasize the need to evaluate equilibrium on rational grounds.

187

APPENDIX 7.C

Corrected Proportions of Explained Variance

for Nonrecursive Models

Several authors have noted that

smc

calculated as one minus the ratio of the disturbance variance

over the total variance may be inappropriate for endogenous variables involved in feedback loops.

This is because the disturbances of such variables may be correlated with one of their presumed

causes, which violates the least squares requirement that the residuals (disturbances) are uncor-

related with all predictors (causal variables). Some corrected R

statistics for nonrecursive models

are described next:

1. The Bentler–Raykov corrected R

(Bentler & Raykov, 2000) is based on a respeciﬁca-

tion that repartitions the variance of endogenous variables controlling for correlations between

disturbances and causal variables. This statistic is automatically printed by EQS for nonrecursive

models.

2. The LISREL program prints a reduced-form R

(Jöreskog, 2000) for each endogenous

variable in a structural model. In reduced form, the endogenous variables are regressed on the

exogenous variables only. This regression also has the consequence that all direct effects of distur-

bances on their respective endogenous variables are removed or blocked, which also removes any

contribution to all other endogenous variables (Hayduk, 2006). For recursive models, the value of

the reduced-form R

can be substantially less than that

smc

for the same variable.

3. Hayduk (2006) describes the blocked-error-R

for variables in feedback loops or with

correlated errors. It is calculating by blocking the inﬂuence of the disturbance of just the variable

in question (the focal endogenous variable). An advantage of this statistic is that it equals the value

smc

for each endogenous variable in a recursive model. The blocked-error-R

is not yet auto-

matically printed by SEM computer programs, but Hayduk (2006) describes a method for doing

so using any program that reports the model-implied covariance matrix when all parameters are

ﬁxed to equal user-speciﬁed values.

Depending on the model and data, the corrected R

s just described can be either smaller or

larger than that of

smc

for endogenous variables in feedback loops. For example, reported next are

values of

smc

, the Bentler–Raykov R

, and the reduced-form R

for the variables mother education

and maternity age in Figure 7.2:

Endogenous variable

smc

BR R

RF R

Mother Education .161 .162 .055

Maternity Age .097 .100 .137

Note. BR, Bentler–Raykov; RF, reduced form.

188 CORE TECHNIQuES

The values of

smc

and the Bentler–Raykov R

are similar and indicate proportions of explained

variance of about .16 and .10 for, respectively, mother education and maternity age. However,

proportions of explained variance estimated by the reduced-form R

are somewhat different. Spe-

ciﬁcally, they are about .06 and .14, respectively, for the same two endogenous variables. Both sets

of results just described are equally correct because they represent somewhat methods to cor-

rect for model-implied correlations between disturbances and causal variables. In written reports,

always indicate the particular R

statistic used to estimate the proportions of explained variance for

endogenous variables in nonrecursive models.

189

Hypothesis Testing

Outlined in this chapter are methods and strategies for (1) evaluating whether a struc-

tural equation model is consistent with the sample data and (2) hypothesis-testing strate-

gies in SEM. Two related topics are (3) statistical power analysis and (4) consideration

of equivalent models or near-equivalent models that ﬁt the same data just as well as

the researcher’s preferred model or nearly so. There is an emerging consensus—one

expressed in a recent special issue on SEM in the journal Personality and Individual

Differences (Vernon & Eysenck, 2007)—that standard practice about model ﬁt evalu-

ation has been lax. Accordingly, next I describe an even more rigorous approach to

model testing compared with the one presented in the previous edition of this book.

This modiﬁed approach includes the reporting of diagnostic information about speciﬁc

sources of model misﬁt. Because the issues discussed here generalize to most SEM

analyses, they warrant careful study.

eYes on the PrIze

Newcomers to SEM sometimes mistakenly believe that “success” means that, at the end

of the analysis, the researcher will have a model that ﬁts the data. However, this outcome

by itself is not very impressive. This is because any model, even one that is grossly mis-

speciﬁed, can be made to ﬁt the data simply by adding free parameters (i.e., reduce df

If all possible free parameters are estimated (df

= 0), ﬁt will likely be perfect, but such

a model would have little scientiﬁc value.

Hayduk, Cummings, Boadu, Pazderka-Robinson, and Boulianne (2007) remind us

that the real goal is to test a theory by specifying a model that represents predictions of

that theory among plausible constructs measured with the appropriate indicators. If

such a model does not ultimately ﬁt the data, then this outcome is interesting because

there is value in reporting models that challenge or debunk theories. But the story is

hardly over if the researcher happens to retain a model. This is because there could be

190 CORE TECHNIQuES

equivalent or near-equivalent models that explain the same data just as well. Among

plausible models with equal or near-equal ﬁt, the researcher must explain why any one

of them may actually be correct. This includes (1) directly acknowledging the exis-

tence of equivalent or near-equivalent models and (2) describing what might be done

in future research to differentiate between any serious competing models. So success in

SEM is determined by whether the analysis dealt with substantive theoretical issues regard-

less of whether a model is retained. In contrast, whether or not a scientiﬁcally meaningless

model ﬁts the data is irrelevant (Millsap, 2007).

state oF PraCtICe, state oF MInd

For at least 30 years the literature has carried an ongoing discussion about the best ways

to test hypotheses and assess model ﬁt. This is also an active research area, especially

concerning computer simulation (Monte Carlo) studies. Discussion and research about

this topic are likely to continue because there is no single, black-and-white statistical

framework within which we can clearly distinguish correct from incorrect hypotheses

in SEM. Nor is there ever likely to be such a thing. Part of the problem is that behavioral

scientists typically study samples, not whole populations, so the problem of sampling

error looms over analyses conducted with sample data. (This is not unique to SEM.)

Another problem is the philosophical question of whether correct models really exist.

The recognition of this possibility is based on the view that basically all statistical mod-

els are wrong to some degree; that is, they are imperfect reﬂections of a complex reality.

Speciﬁcally, a statistical model is an approximation tool that helps researchers to struc-

ture their thinking (i.e., generate good ideas) in order to make sense of a phenomenon

of interest (Humphreys, 2003). If the approximation is too coarse, then the model will

be rejected. Otherwise, the failure to reject a model must not provide unjustiﬁed enthu-

siasm over the implied accuracy of that model; that is, a retained model is not proved

(Chapter 1). MacCallum and Austin (2000) put it this way:

With respect to model ﬁt, researchers do not seem adequately sensitive to the fundamental

reality that there is no true model . . . , that all models are wrong to some degree, even in

the population, and that the best one can hope for is to identify a parsimonious, substan-

tively meaningful model that ﬁts observed data adequately well. At the same time, one

must recognize that there may well be other models that ﬁt the data to approximately the

same degree. Given this perspective, it is clear that a ﬁnding of good ﬁt does not imply that

a model is correct or true, but only plausible. These facts must temper conclusions drawn

about good-ﬁtting models. (p. 218)

A related complication is that there is no statistical “gold standard” in SEM that

automatically and objectively leads to the decision about whether to reject or retain a

particular model.

Researchers typically consult various statistical measures of model–data corre-

Hypothesis Testing 191

spondence in the analysis, but, as explained in the next section, no set of ﬁt statistics is

deﬁnitive. This means that ﬁt statistics in SEM do not generally provide a simple yes or

no answer to the question, should this model be retained? Various guidelines about how

to interpret various ﬁt statistics as providing something like a yes-or-no answer have

been developed over the years, but these rules of thumb are just that. The fact that some

of these interpretive guidelines probably do not apply to the whole range of structural

equation models actually analyzed by researchers is becoming ever more clear. It is also

true that we in the SEM community have collectively relied too much on unsubstanti-

ated principles about what ﬁt statistics say about our models. However, there is no dis-

guising the fact that decisions about the viability of hypotheses and models in SEM are

ultimately a matter a judgment. This judgment should have a solid basis in theory (i.e.,

think like a researcher) and a correct appreciation of the strengths and limitations of ﬁt

statistics. There is also no need to apologize about the role of human judgment in SEM

or science in general. As Kirk (1996) and others note, a scientiﬁc decision is ultimately a

qualitative judgment that is based on the researcher’s domain knowledge, but it will also

reﬂect the researcher’s personal values and societal concerns. This is not “unscientiﬁc”

because the evaluation of all ﬁndings in science involves some degree of subjectivity. It is

better to be open about this fact, however, than to base such decisions solely on statistics

that seem to offer absolute objectivity, but do no such thing. As aptly put by Huberty

and Morris (1988, p. 573), “As in all statistical inference, subjective judgment cannot be

avoided. Neither can reasonableness!”

Described in this chapter is what I believe is a rigorous approach to hypothesis test-

ing that addresses problems seen in too many published reports of SEM analyses. Not all

experts in SEM may agree with each and every speciﬁc detail of this approach, but most

experts would likely concur that authors of SEM studies need to give their readers more

information about model speciﬁcation and its correspondence with the data. Speciﬁ-

cally, I want you to be hardheaded in the way you test hypotheses by being your model’s

toughest critic and by holding it to higher standards than have been applied in the past.

But I do not want you to be bullheaded and blindly follow the method described here as

though it were the path to truth in SEM. Instead, you should use your good judgment

about what makes the most sense in your research area at every step of the process at the

same time you follow a rigorous method of hypothesis testing. To paraphrase Millsap

(2007), this is SEM made difﬁcult, not easy. The hard part is thinking for yourself in a

lucid, disciplined way instead of hoping that ﬁt statistics can somehow make decisions

for you.

a healthY PersPeCtIve on FIt statIstICs

There are dozens of ﬁt statistics described in the SEM literature, and new ones are being

developed all the time. Evaluation of the statistical properties of ﬁt statistics in com-

puter simulation studies is also an active research topic; thus, the state of knowledge is

continually changing. It is also true that SEM computer programs usually print in their

192 CORE TECHNIQuES

output the values of many more ﬁt statistics than are typically reported for the analysis,

which presents a few problems. One problem is that different ﬁt statistics are reported

in different articles, and another is that different reviewers of the same manuscript may

request statistics that they know or prefer (Maruyama, 1998). It can therefore be difﬁ-

cult for a researcher to decide on which particular statistics and which values to report.

There is also the possibility for selective reporting of values of ﬁt statistics. For example,

a researcher keen to demonstrate acceptable model ﬁt may report only those ﬁt statistics

with favorable values. A related problem is “ﬁt statistic tunnel vision,” a disorder appar-

ent among practitioners of SEM who become so preoccupied with overall model ﬁt that

other crucial information, such as whether the parameter estimates actually make sense,

is overlooked. Fortunately, there is a cure, and it involves close inspection of the whole

computer output (Chapter 7), not just the section on ﬁt statistics.

A more fundamental issue is the ongoing debate in the ﬁeld about the merits of the

two main classes of ﬁt statistics described in the next section: model test statistics and

approximate ﬁt indexes. To anticipate some of this debate now, some methodologists

argue strongly against what has become a routine—and bad—practice for researchers to

basically ignore model test statistics and justify retention of their preferred model based

on approximate ﬁt indexes. Others argue that there is a role for reasoned use of approxi-

mate ﬁt indexes in SEM, but not at the expense of what test statistics say about model

ﬁt. I will try to convince you that (1) there is real value in the criticisms of those who

argue against the uncritical use of approximate ﬁt indexes, and (2) we as practitioners

of SEM need to “clean up our act” by taking a more skeptical, discerning approach to

model testing. That is, we should walk disciplined model testing as we talk it (practice

the rigor that we as scientists preach).

The main beneﬁt of hypothesis testing in SEM is to place a reasonable limit on

the extent of model–data discrepancy that can be attributed to mere sampling error.

Speciﬁcally, if the degree of this discrepancy is less than that expected by chance, there

is initial support for the model. This support may be later canceled by results of more

speciﬁc diagnostic assessments, however, and no testing procedure ever “proves” mod-

els in SEM (Chapter 1). Discrepancies between model and data that clearly surpass the

limits of chance require diagnostic investigation of model features that might need to be

respeciﬁed in order to make the model consistent with the evidence.

Before any individual ﬁt statistic is described, it is useful to keep in mind the fol-

lowing limitations of basically all ﬁt statistics in SEM:

1. Values of ﬁt statistics indicate only the average or overall ﬁt of a model. That is,

ﬁt statistics collapse many discrepancies into a single measure (Steiger, 2007). It is thus

possible that some parts of the model may poorly ﬁt the data even if the value of a ﬁt

statistic seems favorable. In this case, the model may be inadequate despite the values of

its ﬁt statistics. This is why I will recommend later that researchers report more speciﬁc

diagnostic information about model ﬁt of the type that cannot be directly indicated by

ﬁt statistics alone. Tomarken and Waller (2003) discuss potential problems with models

that seem to ﬁt the data well based on values of ﬁt statistics.

Hypothesis Testing 193

2. Because a single statistic reﬂects only a particular aspect of ﬁt, a favorable value

of that statistic does not by itself indicate acceptable ﬁt. That is, there is no such thing as

a magical, single-number summary that says everything worth knowing about model ﬁt.

3. Unfortunately, there is little direct relation between values of ﬁt statistics and

the degree or type of misspeciﬁcation (Millsap, 2007). This means that researchers can

glean relatively little about just where and by how much the model departs from the data

from inspecting values of ﬁt statistics. For example, ﬁt statistics cannot tell whether you

have the correct number of factors (3, 4, etc.) in a measurement model. Other kinds of

diagnostic information, such as covariance residuals and correlation residuals, speak

more directly to this issue.

4. Values of ﬁt statistics that suggest adequate ﬁt do not also indicate that the pre-

dictive power of the model is also high as measured by statistics for individual endog-

enous variables such as

smc

. In fact, overall model ﬁt and

smc

for individual outcomes

are relatively independent characteristics. For example, disturbances in structural mod-

els with perfect ﬁt can still be large (i.e.,

smc

s are low), which means that the model

accurately reﬂects the relative lack of predictive validity.

5. Fit statistics do not indicate whether the results are theoretically meaningful.

For instance, the sign of some path coefﬁcients may be unexpectedly in the opposite

direction (e.g., Figure 7.1). Even if values of ﬁt statistics seem favorable, results so anom-

alous require explanation.

tYPes oF FIt statIstICs and “golden rules”

Described next are the two broad categories of ﬁt statistics and the status of interpreta-

tive guidelines associated with each. Each category actually represents a different mode

or contrasting way of considering model ﬁt.

Model test statistics

These are the original ﬁt statistics in SEM. A model test statistic is a test of whether

the covariance matrix implied by the researcher’s model is close enough to the sample

covariance matrix that the differences might reasonably be considered as being due to

sampling error. If not, then (1) the data covariances contain information that speak

against the model, and (2) this outcome calls for the researcher to explain model-data

discrepancies that exceed those expected by chance.

Most model test statistics are generally scaled as “badness-of-ﬁt” statistics because

the higher their values, the worse the model’s correspondence with the data. This

means that a statistically signiﬁcant result (e.g., p < .05) indicates problematic model–

data correspondence. That is, it is the failure to reject the null hypothesis (e.g., p ≥ .05)

that the model-implied covariance matrix is identical to the population covariance

matrix that generated the sample covariance matrix that supports the researcher’s

model. This logic is “backward” from the usual reject–support context for statisti-