Alfred DeMaris - Regression with Social Data, Modeling Continuous and Limited Response Variables

Подождите немного. Документ загружается.

I refer to ∆ as the explained risk of an event and show that it is bounded by 0 and

1 (DeMaris, 2002c). Like P

, ∆ reﬂects the model’s ability to account for the

response. When the model aﬀords no improvement in predicting the event of inter-

est compared to using the marginal probability to predict Y, ∆ will be 0. At the other

extreme, if all conditional probabilities are either 0 or 1, the occurrence/nonoccur-

rence of an event is predicted with certainty. In this case, the conditional variance of

is 0 for all i, and ∆ equals 1. A consistent estimator of ∆ is

∆

⫽1 ⫺

ᎏ

冢

冱

i⫽1

(

⫺

1⫺

πˆ

)冟x

冣

ᎏ

where the numerator of the second expression to the right of the equals sign is just the

average estimated conditional variance of Y over all n cases. Like R

, ∆

has an

explained variance interpretation and is bounded by 0 and 1 but is not necessarily

nondecreasing in x. My simulation results showed that ∆

was the best estimator of ∆

among the several R

analogs investigated (DeMaris, 2002c). Both R

and ∆

require

casewise calculations and are not currently available in conventional software. (A

program in SAS that calculates these as well as six other R

analogs is available from

the author by request, however.) For the logit model in Table 7.1, ∆

⫽ .051.

Three other measures are worth mentioning because they are all nondecreasing in

x as well as bounded by 0 and 1. All are based on the likelihood function, and as

such, have much broader applicability than R

and ∆

. These can be employed with

any model employing maximum likelihood estimation. They are also readily calcu-

lated from standard logistic regression output. The ﬁrst is the likelihood-ratio index

(Long, 1997):

⫽ ,

the numerator of which is just the model χ

, described above. As mentioned above,

⫺2logL

is analogous to TSS and ⫺2 log L

is analogous to SSE; hence R

is a direct

counterpart of the OLS R

. However, it does not have an explained variance inter-

pretation. Rather, it represents the proportionate reduction in minus twice the log-

likelihood (a measure of total uncertainty in Y ) when the likelihood function is

evaluated at the MLEs for the parameters rather than the MLE for an intercept-only

model. I have found that R

is a comparatively good estimator of explained risk

(DeMaris, 2002c). For the logit model in Table 7.1, R

⫽⫽.060.

The second measure is the generalized R

(Allison, 1995; Maddala, 1983). Its for-

mula is

⫽ 1 ⫺

冢

ᎏ

冣

2/n

3249.56 ⫺ 3054.132

ᎏᎏᎏ

3249.56

⫺2 log L

⫺ (⫺2 log L

)

ᎏᎏᎏ

⫺2 log L

EMPIRICAL CONSISTENCY AND DISCRIMINATORY POWER 275

c07.qxd 8/27/2004 2:54 PM Page 275

It can be shown that R

is identically equal to the OLS R

in a linear regression

model with normally distributed errors. (A proof is available from the author on

request.) Because the likelihood functions in this equation may be diﬃcult to work

with, the measure can also be computed as (Allison, 1995)

⫽1 ⫺ exp

冢

ᎏ

⫺

ᎏ

冣

where in this case, χ

is the model χ

discussed above. For the logit model in Table

7.1, R

⫽ 1 ⫺ exp

冢

ᎏ

⫺1

394

ᎏ

冣

⫽ .0466.

It turns out that R

has an upper bound of less than 1. In particular, the upper

bound is 1 ⫺ (L

)

2/n

, suggesting the scaled measure, R

GSC

, proposed by Cragg and

Uhler (1970) as well as others (Maddala, 1983; Nagelkerke, 1991):

GSC

⫽

ᎏ

1 ⫺

)

2/n

ᎏ

or in terms of ease of computation, we have

GSC

⫽ .

The scaling ensures that R

GSC

lies between 0 and 1. Although both R

and R

GSC

are comparable to R

in the linear regression model, they do not have an explained

variance interpretation in logistic regression. For the logit model in Table 7.1,

GSC

⫽⫽

ᎏ

⫽ .085.

All ﬁve R

analogs just discussed are shown in Table 7.1, at the bottom of

the logit model column (R

, R

, and R

GSC

are also shown for probit). Which to

employ depends on how couple violence should be conceptualized. Physical

aggression should probably be considered a continuous variable that ranges from

displacement aggression (e.g., kicking in doors) to relatively minor violence (e.g.,

pushing and shoving), to severe acts (e.g., beating someone up or attacking a per-

son with a weapon). Due to the crudeness of measurement in this case, however,

this range of acts is simply mapped into violence, a binary yes–no response. In

that the continuous response of physical aggression is really the focus, however,

is the preferred measure. It indicates that about 13% of the variance in phys-

ical aggression is accounted for by the logistic regression. Were we to be solely

interested in the event of whether or not a couple is reported as being violent, how-

ever, ∆

suggests that only about 5% of this phenomenon is accounted for by the

model.

.0466

ᎏᎏᎏᎏ

1 ⫺ exp[(2/4095)(⫺1624.763)]

1 ⫺ exp(⫺χ

/n)

ᎏᎏᎏ

1 ⫺ exp[(2/n)log L

]

276 REGRESSION WITH A BINARY RESPONSE

c07.qxd 8/27/2004 2:54 PM Page 276

EXERCISES

7.1 The following ( x,y ) pairs were obtained for ﬁve individuals: (1.5, 1), (2, 1),

(1.5, 0), (1.75, 0), (4, 0). Maximum likelihood estimates for a logistic

regression of Y on X produced the following equation: ln O

⫽ 1.7184 ⫺

1.0627X. Using this equation and a calculator, give the log of the like-

lihood function for this model, evaluated at the MLEs of the parameter

estimates.

7.2 Give the model χ

for the model in Exercise 7.1.

7.3 Give R

, R

, and R

GSC

for the model in Exercise 7.1.

7.4 Using a calculator and the equation in Exercise 7.1, give R

and ∆

for the

model in Exercise 7.1.

7.5 For the low-risk couple described in this chapter, give the probability of vio-

lence, according to the scobit model in Table 7.2.

7.6 For the low-risk couple described in this chapter, give the probability of vio-

lence according to the complementary log-log model in Table 7.2.

7.7 For the 4095 couples in the NSFH, deﬁne a high-risk couple as a cohabiting,

minority couple in which at least one partner has a problem with alcohol or

drugs, who is 1 standard deviation below the mean on relationship duration

and female’s age at union, and 1 standard deviation above the mean on male’s

isolation and economic disadvantage. Give the estimated probability of vio-

lence for such a couple based on the logit model in Table 7.1.

7.8 Give the estimated probability of violence for the high-risk couple in Exercise

7.7 based on the probit model in Table 7.1.

7.9 Give the partial derivative of the probability of violence, with respect to rela-

tionship duration, based on both logit and probit, for the high-risk couple in

Exercise 7.7.

7.10 Recall that a “baseline” couple (having all covariates in Table 7.1 set to zero)

is estimated to have odds of violence equal to .1164 and a probability of violence

of .1043. Give the change in probability of violence expected for an addi-

tional year of being together, according to the logit model in Table 7.1.

7.11 Based on the probit model in Table 7.1, give the baseline probability of vio-

lence, as well as the change in the probability of violence for an additional

year of being together, for a baseline couple.

EXERCISES 277

c07.qxd 8/27/2004 2:54 PM Page 277

7.12 Use partial derivatives to estimate the changes in probability calculated in

Exercises 7.10 and 7.11. How close are the approximations in this case?

7.13 A simpliﬁed logit model for couple violence for the 4095 NSFH couples is

ln O

⫽⫺1.948 ⫹ 1.0563 alcohol/drug problems ⫹ .0307 economic disadvan-

tage.

(a) Interpret all three of the parameter estimates in terms of the odds of vio-

lence, recalling that economic disadvantage is a centered variable.

(b) Show that equations (7.15) and (7.16) imply that the model is interactive

in the regressors in their eﬀects on π

violent

. This can be shown by showing

that a 1-unit increase in economic disadvantage has a diﬀerent impact on

violent

for those with alcohol/drug problems than for those without such

problems. (Hint: Use those without alcohol/drug problems, who have

mean economic disadvantage, as the baseline group.)

Use the following additional analyses of couple violence for the 4095 NSFH couples

for Exercises 7.14 to 7.20:

Predictor Logit 1 Logit 2 Probit 1 Probit 2

Intercept .0011 ⫺.4657* ⫺.1295 ⫺.3810***

Average age ⫺.0449*** ⫺.0332*** ⫺.0231*** ⫺.0168***

First union ⫺.2394* ⫺.2212* ⫺.1268* ⫺.1146*

Number of children ⫺.0403 ⫺.1027* ⫺.0255 ⫺.0569**

Female traditional ⫺.0773 ⫺.1472 ⫺.0439 ⫺.0747

Male traditional .2444* .1898 .1391* .1071

Both traditional ⫺.0528 ⫺.0822 ⫺.0251 ⫺.0348

Conﬂict over money .3220*** .1812***

Conﬂict over time .1942*** .1082***

Conﬂict over sex .1735*** .0951**

⫺2 ln L 3120.892 2973.667 3122.895 2976.173

Partial covariance matrix for logit 2:

7.14 Excluding the intercept, interpret all regressor coeﬃcients in logit 1 with

respect to the odds of couple violence.

7.15 Give model χ

’s for all four models and test whether the addition of the three

conﬂict variables adds signiﬁcantly to model ﬁt for both the logit and probit

analyses.

7.16 Give R

, R

, and R

GSC

for all four models.

Money Time Sex

Money .0027 ⫺.0008 ⫺.0005

Time .0025 ⫺.0010

Sex .0027

278 REGRESSION WITH A BINARY RESPONSE

c07.qxd 8/27/2004 2:54 PM Page 278

7.17 Based on logit 1, give:

(a) The impact on the odds of violence for a 10-year increase in average age.

(b) The impact on the odds of violence for having three additional children.

traditional vs. couples in which only the male is traditional.

7.18 Find the predicted probability of violence for a couple whose average age is

25, who is in their ﬁrst union, who has three children, and among whom only

the male is traditional, based on both logit 1 and probit 1.

7.19 What is the diﬀerence in π

violent

, compared to the couple in Exercise 7.18, for

having been in a prior union, based on both logit 1 and probit 1?

7.20 Using the results for logit 2, test whether the eﬀects of conﬂict over money,

time, and sex on the log odds of violence are diﬀerent from each other.

Use software along with the couples dataset to answer Exercises 7.21 to 7.26.

7.21 The variable FAGRESS represents whether or not the female partner has

hit, shoved, or thrown things at the male partner in the past year, and is

coded 1 (“yes”) and 2 (“no”). Regress this variable on SEPARATE, CHIL-

DREN, MAGUNION, CONFLICT (the average of MFIGHTS and

FFIGHTS), and DURYRS, using both logistic and probit regression. Show

the logit and probit estimates and model χ

’s, indicating significance levels

with asterisks. Interpret the effect of relationship duration in the logit and

probit models.

7.22 Find the predicted probability of female aggression at 1 standard deviation

above the mean of conflict, for those who never lived apart because of dis-

agreements, and who are average on all other predictors, for both the logit and

probit models in Exercise 7.21.

7.23 What is the change in probability of female aggression for a standard devia-

tion increase in the level of conflict, at the π

calculated in Exercise 7.22, based

on both logit and probit?

7.24 Based on the logit results in Exercise 7.21: What is the percent difference in

the odds of female aggression for those who have lived apart for a time

because of disagreements vs. those who have not? By what percent do cou-

ples’ odds of female aggression change for a standard deviation increase in

the male’s age at union formation?

7.25 What is the discriminatory power of the model, based on both R

and R

for both the logit and probit models in Exercise 7.21?

EXERCISES 279

c07.qxd 8/27/2004 2:54 PM Page 279

7.26 Using the couples dataset, create a marital status variable with the following

levels (dummy names):

• Couples currently cohabiting unmarried (COHABIT)

• Married couples, both in a ﬁrst marriage (FIRSTMAR)

• Married couples, husband married before (HUSBRE)

• Married couples, wife married before (WIFERE)

• Married couples, both married before (BOTHRE)

(Hint: You can create MARHIST using SAS code, as in: IF MARCOHAB ⬎ 1

THEN MARHIST ⫽ 1; * cohabitors; IF MARCOHAB ⫽ 1 AND MTI-

MARR ⬎ 1 AND FTIMARR ⫽ 1 THEN MARHIST ⫽ 2; * husband remar-

ried; and so on.) Then, using logistic regression:

(a) Test whether marital status has a signiﬁcant eﬀect on the probability of

female aggression, ignoring other covariates.

(b) Test all possible contrasts between pairs of marital status categories,

again, ignoring other covariates. Show the odds ratios (of female aggres-

sion) for each of these pairs. (Hint: This is most easily done via repeated

estimation of the model after changing the contrast category for the marital-

status dummies.)

ling for the covariates in the model of Exercise 7.21.

Use the following data for Exercises 7.27 to 7.30. These data are from Spector and

Mazzeo (1980), as reported in Aldrich and Nelson (1984). They are from a study that

examined the effect of a teaching method (PSI) on performance in an intermediate

macroeconomics class. The other regressors are entering GPA (GPA) and entering

knowledge of the material (TUCE).

OBS GPA TUCE PSI GRADE OBS GPA TUCE PSI GRADE

1 2.66 20 0 C 17 2.75 25 0 C

2 2.89 22 0 B 18 2.83 19 0 C

3 3.28 24 0 B 19 3.12 23 1 B

4 2.92 12 0 B 20 3.16 25 1 A

5 4.00 21 0 A 21 2.06 22 1 C

6 2.86 17 0 B 22 3.62 28 1 A

7 2.76 17 0 B 23 2.89 14 1 C

8 2.87 21 0 B 24 3.51 26 1 B

9 3.03 25 0 C 25 3.54 24 1 A

10 3.92 29 0 A 26 2.83 27 1 A

11 2.63 20 0 C 27 3.39 17 1 A

12 3.32 23 0 B 28 2.67 24 1 B

13 3.57 23 0 B 29 3.65 21 1 A

14 3.26 25 0 A 30 4.00 23 1 A

15 3.53 26 0 B 31 3.10 21 1 C

16 2.74 19 0 B 32 2.39 19 1 A

280 REGRESSION WITH A BINARY RESPONSE

c07.qxd 8/27/2004 2:54 PM Page 280

7.27 Estimate the probability of getting an A (π

) as a linear function of GPA,

TUCE, and PSI, using OLS. Interpret the coeﬃcients. Give π

for OBS 11.

7.28 Estimate π

as a linear function of GPA, TUCE, and PSI, using WLS. This is

done as follows: From the OLS regression in Exercise 7.27, save the ﬁtted

values (i.e., the π

). Recode any π

ⱕ 0 as .001. Recode any π

ⱖ 0 as .999.

Create the estimated conditional variance of Y for each case as π

(1 ⫺ π

The weight for WLS is then the reciprocal of this estimated conditional vari-

ance. Give π

for OBS 11, based on the WLS estimates.

7.29 Estimate π

as a function of GPA, TUCE, and PSI, using logistic regression.

Interpret all estimates in terms of odds. Give the π

for OBS 11. Give R

for

the model.

7.30 Estimate π

as a function of GPA, TUCE, and PSI, using both probit and

complementary log-log models. Give π

for OBS 11, based on each set of

estimates.

EXERCISES 281

c07.qxd 8/27/2004 2:54 PM Page 281

282

Regression with Social Data: Modeling Continuous and Limited Response Variables,

By Alfred DeMaris

CHAPTER 8

Advanced Topics in Logistic

Regression

CHAPTER OVERVIEW

In Chapter 7 we saw that the logistic regression model was particularly advantageous

with respect to the interpretation of coeﬃcients. In this chapter, then, we consider the

logistic regression model in greater detail, continuing our investigation of couple vio-

lence in the NSFH. I begin by outlining the techniques used to investigate interaction

eﬀects. First, I sketch out how we compare models across groups: in this case, minor-

ity vs. nonminority couples. This involves a logistic regression analog of the Chow test

that was covered in Chapters 3 and 4. I then consider modeling interaction eﬀects

involving speciﬁc regressors, using cross-product terms. From this I move to a discus-

sion of modeling nonlinearity in the relationship between regressors and the logit,

detailing a general means of assessing whether linearity is plausible. I then address the

testing of coeﬃcient changes across models, demonstrating an analog of the procedure

discussed in Chapter 3. I conclude the discussion of binary logistic regression by rein-

vestigating the discriminatory power and empirical consistency of the ﬁnal model of

interest. The narrative then moves to an explication of logistic regression with multin-

omial responses. First, I develop the multinomial logistic regression model for the case

in which the categories of the response are not ordered, and consider both limited- and

full-information maximum likelihood approaches. Finally, I take advantage of the ordi-

nal nature of the response and introduce the ordered logit model.

MODELING INTERACTION

An artifact of the logistic regression model, as well as the other probability mod-

els considered, is that the regressors are automatically interactive with respect to

c08.qxd 8/27/2004 2:54 PM Page 282

probabilities. (Showing this was the theme of Exercise 7.13.) However, to model

interaction over and above what is incorporated into the nature of the logit link, or

to model interaction in the odds or the log odds, we have two choices. Either we

can compare the model over levels of an explanatory variable or we can utilize

cross-product terms, similar to the approaches taken in linear regression.

Comparing Models across Groups

Until now I have been utilizing a combined sample of minority and nonminority cou-

ples to investigate models of couple violence. The dummy variable reﬂecting minor-

ity status in the model simply allows minorities to have a diﬀerent baseline log odds

of violence than that for nonminorities. Otherwise, the eﬀects of the other regressors

are assumed to be the same for minorities, as opposed to nonminorities. Although I

have no reason to suspect otherwise, it would be fruitful to provide a statistical

justiﬁcation for combining the models for both groups. In Chapters 3 and 4 we saw

that the Chow test is used for this purpose in linear regression. There is an analog of

this test for logistic regression, recently outlined by Allison (1999).

Recall that the Chow test in linear regression assumes that the equation error vari-

ance is the same across groups. The following explication similarly assumes equal

error variance. In this case, however, the error variance in question pertains to the

latent-scale formulation of the logistic regression model. That is, if

⫽

冱

⫹ ε

is the equation that underlies the binary response for the ﬁrst group, and

⫽

冱

⫹ υ

underlies the binary response for the second group, the assumption is that V(ε

) ⫽

V(ν

). [It is not necessary to assume that a latent variable underlies the response in

order to motivate the homogeneity-of-variance assumption; see Allison (1999) for

details.] Unfortunately, there is no simple means of testing this assumption [see

Allison (1999) for suggested techniques, however]. In the present example, we sim-

ply assume that the error variances are equal. The Chow test analog for logistic

regression involves estimating the model for the combined sample and then for each

sample separately. For two groups, the test statistic is then χ

⫽⫺2lnL

⫺ [⫺2ln

⫹ (⫺2lnL

)], where ln L

is the ﬁtted log-likelihood for the combined sample,

ln L

the ﬁtted log-likelihood for group 1, and ln L

the ﬁtted log-likelihood for

group 2. Under the null hypothesis that

γγ

⫽

ββ

, that is, that regressor eﬀects are the

same across groups, χ

has a chi-squared distribution with degrees of freedom equal

to the diﬀerence in the number of parameters estimated in the combined versus the

separate sample approaches. As with linear regression, the intercept can be constrained

to be the same by omitting the dummy for group in the combined-sample model.

Or, it can be allowed to diﬀer across groups by including the group dummy in the

combined-sample model.

MODELING INTERACTION 283

c08.qxd 8/27/2004 2:54 PM Page 283

Example. Table 8.1 presents the results of estimating logistic regression models for

couple violence using the combined sample of 4095 couples, as well as using sepa-

rate samples of 1216 minority and 2879 nonminority couples. The ﬁrst model is just

a repeat of the logit model in Table 7.1 and represents a model in which the intercept

is unconstrained across groups. The second model constrains the intercept, in addi-

tion to the regressors, to be the same for minorities and nonminorities. The third and

fourth columns represent the models for minorities and nonminorities, respectively.

A glance at the coeﬃcients in the last two columns suggests that there are some

diﬀerences in regressor eﬀects across groups. But diﬀerences in coeﬃcients are to be

expected, due purely to sampling variability. The test statistic for coeﬃcient invari-

ance across groups, allowing the intercept to be unconstrained, is χ

⫽ 3054.132 ⫺

(1006.921 ⫹ 2043.172) ⫽ 4.039. The degrees of freedom for the test are ﬁgured as

follows. Including the intercept, we estimate eight parameters in the combined model,

whereas we estimate seven parameters in each of the submodels. The test therefore

has (7 ⫹ 7) ⫺ 8 ⫽ 6 degrees of freedom. The result is nonsigniﬁcant (p ⫽ .671). If

we allow the intercept to be constrained, we have χ

⫽ 3058.240 ⫺ (1006.921 ⫹

2043.172) ⫽ 8.147. With seven degrees of freedom, this test is also nonsigniﬁcant

(p ⫽ .32). Based on either result, I conclude that there is insuﬃcient evidence to sug-

gest that regressor eﬀects are any diﬀerent for minorities versus nonminorities. As

was the case in linear regression, an equivalent test can be fashioned using cross-prod-

uct terms. Testing the diﬀerence between the model in the ﬁrst column in Table 8.1

versus a model in which the cross-products of minority status with all 6 other regres-

sors has been added gives us the unconstrained-intercept test above. Adding the

dummy for minority status plus the six cross-product terms to the model in the sec-

ond column of Table 8.1, and testing the diﬀerence in resulting models, produces the

constrained-intercept test.

284 ADVANCED TOPICS IN LOGISTIC REGRESSION

Table 8.1 Regression Results for the Logit Model in Table 7.1 Estimated for the

Combined Sample as Well as Separately for Minority and Nonminority Couples

Combined Sample

Intercept Intercept Minority Nonminority

Predictor Unconstrained Constrained Couples Couples

Intercept ⫺2.151*** ⫺2.081*** ⫺1.892*** ⫺2.154***

Relationship duration ⫺.046*** ⫺.046*** ⫺.039*** ⫺.050***

Cohabiting .810*** .836*** .706* .960**

Minority couple .221*

Female’s age at union ⫺.027*** ⫺.026*** ⫺.017 ⫺.033***

Male’s isolation .020* .019* .027* .017

Economic disadvantage .023* .030*** .017 .032*

Alcohol/drug problem 1.029*** 1.011*** 1.244*** .946***

⫺2 log L 3054.132 3058.240 1006.921 2043.172

Number of parameters 8 7 7 7

n 4095 4095 1216 2879

* p ⬍ .05. ** p ⬍ .01. *** p ⬍ .001.

c08.qxd 8/27/2004 2:54 PM Page 284