Alfred DeMaris - Regression with Social Data, Modeling Continuous and Limited Response Variables

Подождите немного. Документ загружается.

U 

冤冥

6.14 (a)Give φ for R

(b)Give R

(c)Give R

1

. [Hint: for parts (b) and (c) you may want to use a matrix program

such as SAS IML or MATLAB to perform the necessary calculations.]

6.15 (a) Show the decomposition of V(b

) after the fashion of equation (6.14),

assuming that V(ε

∗

)  σ

(b) Give the variance proportions p

, p

, and p

6.16 For the following data:

Assume that the model for the squared errors is ε

 σ

exp(δ

 δ

X)u, and

using a calculator, ﬁnd the weights to be used in a WLS regression of Y on X.

6.17 Using the couples dataset, perform a WLS regression of COITFREQ on

MALEAGE via an OLS regression of 兹w

苶

COITFREQ on 兹w

苶

and 兹w

苶

MALEAGE, as discussed in the chapter (remember to specify a regression

through the origin). Conﬁrm that the resulting coeﬃcients for 兹w

苶

and for

MALEAGE are the intercept and slope in the last column of Table 6.1. (Note:

The solution is also in this column.)

6.18 Use the kids dataset to test for heteroscedasticity in the regression of PER-

MISIV on ADVENTRE, FSTYLE1, MSEXATT, and FSEXATT. Do both

White’s test and the Breusch–Pagan test. Also, obtain the White het-

eroscedasticity-robust estimates of the OLS standard errors (using appropri-

ate software).

6.19 Use the kids dataset to estimate a WLS regression for the model in Exercise

6.18, employing an exponential function of the predictors to model the error

variance (as in Exercise 6.16). Be sure to provide the correct R

for the WLS

analysis.

6.20 Using the students dataset, test whether the error variance for the model

EXAM1  β

 β

SCORE  β

COLGPA  β

STATMOOD  ε is diﬀerent

Case XY Case XY

1 0 0 4 10 12.5

2 0 5 5 15 7.5

3 10 2.75 6 15 27.5

.3199

.8055

.4988

.0014

.7549

.1015

.6480

.0017

.0061

.0093

.0082

.9999

.5725

.5837

.5756

.0137

EXERCISES 245

c06.qxd 8/27/2004 2:53 PM Page 245

for males and females. Then test for gender diﬀerences in the model’s

coeﬃcients using both OLS and WLS approaches. Missing imputation:

Substitute the parenthetical values for missing data on each variable indi-

cated: SCORE (40.9358974) and COLGPA (3.0827835).

6.21 Using the students dataset, and the results from Exercise 6.20, test for gender

diﬀerences in EXAM1 scores at college GPAs of 2.5, 3.0, and 3.5 employing

the WLS approach, in which the weights are functions of each gender’s error

variance.

For Exercises 6.22 to 6.25, use the faculty salary dataset and note that “the model”

refers to the regression of AYSALARY on YRDG, YRBG, PRIOREXP, R1, R2, R4,

YRRANK, SALFAC, GRAD, ADMIN, and EMINENT. This model should be regarded

as having heuristic rather than substantive value. First, YRDG, YRBG, and PRIOR-

EXP in ordinary circumstances would be exactly linearly dependent, since PRIOR-

EXP  YRDG  YRBG. However, in this case, due to inaccuracies in either recall or

recording of dates, some values of PRIOREXP were negative. These were simply

recoded to zero, a procedure which, as in the case of missing imputation, nulliﬁes

the exact linear dependency. Second, it is probably not realistic to attempt to esti-

mate the impact of each of YRDG, YRBG, and PRIOREXP while holding the other

two variables constant, given how much information these variables have in com-

mon. Nevertheless, teasing out the separate impacts of each of these factors might

be of interest.

6.22 Examine the inﬂuence diagnostics for the model. Describe the characteristics

of the most inﬂuential observation(s) in the data. Is there any justiﬁcation for

dropping any of the cases from the analysis?

6.23 Examine collinearity diagnostics for the model. Which variables appear to be

tied together by near-linear dependencies? How severe is the collinearity?

Which coeﬃcients are aﬀected, and by how much?

6.24 Using appropriate software, perform a ridge regression for the model. Use the

ridge traces of YRDG, YRBG, and PRIOREXP to choose the best ridge con-

stant.

6.25 Using appropriate software, perform a principal components regression for

the model, leaving out the last component of R

1

. Which set of coeﬃcients is

more appealing: those from the ridge regression or those from the principal

components regression? Why?

246 ADVANCED ISSUES IN MULTIPLE REGRESSION

c06.qxd 8/27/2004 2:53 PM Page 246

CHAPTER 7

Regression with a Binary Response

CHAPTER OVERVIEW

In the social sciences some of the more interesting response variables are binary, or

dichotomous. In certain instances, such variables arise due to the crudeness of meas-

urement. For example, the NSFH asks respondents involved in marriages or cohab-

iting relationships whether they and their partner experienced any “physical

arguments” in the past year, with possible responses of “yes” and “no.” Although

physical aggression might be treated as a continuous variable if it were measured

more precisely, we are limited to a binary variable in this case, due to the measure-

ment strategy. Other variables are naturally binary, such as whether someone voted

in the last presidential election, or whether a woman experienced a pregnancy prior

to age 18. In either case, employing a linear regression model is not the optimum

strategy with a binary response. Hence, in this chapter we consider some alternative,

nonlinear regression models that are especially suited to dichotomous dependent

variables. I begin by considering the problems encountered in using linear regression

for such situations. I then introduce the two most popular alternatives, logistic and

probit regression, along with the theoretical rationale for these techniques. Details of

interpretation, estimation, and inference are covered, with analogies to counterparts

in linear regression. I then introduce two other, less known variants on binary

response models, the scobit and complementary log-log models. Although not as

easily interpreted as the logit model, these techniques overcome some of the limita-

tions of logit and probit. Because of its interpretational advantages, primary empha-

sis is given in this chapter to the logistic regression model. The chapter closes with

a discussion of assessing both discriminatory power and empirical consistency for

this model.

247

Regression with Social Data: Modeling Continuous and Limited Response Variables,

By Alfred DeMaris

c07.qxd 8/27/2004 2:54 PM Page 247

248 REGRESSION WITH A BINARY RESPONSE

LINEAR PROBABILITY MODEL

Suppose that we have a binary response, Y

, coded 1 if the ith case is in the category

of interest, and 0 otherwise. (The coding of a binary response is actually arbitrary,

but dummy coding is especially convenient, as will become apparent.) Recall that

the linear regression model for the conditional mean of Y, given x,is

E(Y

) ⫽ β

⫹ β

⫹

...

⫹ β

However, the mean of a dummy coded variable is the proportion of people in the cat-

egory of interest, or equivalently, the probability of being in the interest category,

denoted π. Letting π

be the probability of being in the interest category given the ith

covariate pattern, the linear regression model for the conditional mean of a binary

response is

⫽ β

⫹ β

⫹

...

⫹ β

. (7.1)

Because the probability is being modeled as a linear function of the parameters, this

is referred to as the linear probability model (LPM) (Aldrich and Nelson, 1984;

Long, 1997). The regression coeﬃcients are interpreted in terms of the probability

of being in the interest category on Y. Hence, β

represents the change in the proba-

bility for each unit increase in X

, net of the other covariates, and so on.

Example

Employing questions from the NSFH on partners’ violence toward each other, I

categorized 4095 married and cohabiting couples surveyed between 1987 and

1994 according to whether or not either partner had been violent toward the other

during that period. These data are in the violence dataset. A total of 555 couples,

or 13.55%, had experienced intimate violence. Of interest here is the extent to

which couple violence is a function of several couple characteristics, including

whether they were cohabiting, as opposed to legally married (cohabiting); the

duration of the relationship, in years, as of the initial survey (relationship dura-

tion); whether either partner in the couple was a minority (minority couple); the

female’s age at the start of the union ( female’s age at union); the degree to which

the male was socially or emotionally isolated from his or his partner’s immediate

kin (male’s isolation); the degree of economic disadvantage exhibited by the cou-

ple’s neighborhood of residence at the time of the initial survey (economic disad-

vantage); and whether either partner had a problem with alcohol or drugs

(alcohol/drug problem). The continuous variables relationship duration, female’s

age at union, male’s isolation, and economic disadvantage are all centered.

Although the couple’s violence proﬁle is, in actuality, a three-category variable,

here I simply distinguish the violent from the nonviolent. In Chapter 8 I consider

the three-level response in greater detail. To keep things relatively simple, issues

of sample selectivity and other critical explanatory variables are omitted in what

c07.qxd 8/27/2004 2:54 PM Page 248

follows. For the larger project on which this example is based, the reader should

consult DeMaris et al. (2003).

The OLS column in Table 7.1 presents the results of regressing violence on cou-

ple characteristics using the LPM. The F test suggests that the model is signiﬁcant

as a whole, and the R

indicates that it accounts for about 5% of the variation in vio-

lence. Several factors have signiﬁcant eﬀects in expected directions. Consistent with

past research (Stets, 1991), cohabitors are more likely than marrieds to be violent.

The coeﬃcient of .161 means that, net of other eﬀects, cohabitors’ probability of

LINEAR PROBABILITY MODEL 249

Table 7.1 OLS, Logit, and Probit Estimates for the Regression of Violence on Couple

Characteristics

Model

Predictor Estimate OLS Logit Probit

Intercept b .117*** ⫺2.151*** ⫺1.247***

.007 .066 .034

t or z 17.750 ⫺32.854 ⫺37.058

Relationship duration b ⫺.004*** ⫺.046*** ⫺.023***

.0004 .005 .002

t or z ⫺9.460 ⫺9.299 ⫺9.352

Cohabiting b .161*** .810*** .485***

.037 .241 .145

t or z 4.360 3.363 3.340

Minority couple b .023 .221* .122*

.012 .108 .059

t or z 1.850 2.042 2.082

Female’s age at union b ⫺.003*** ⫺.027*** ⫺.014***

.001 .007 .004

t or z ⫺3.520 ⫺3.588 ⫺3.538

Male’s isolation b .002* .020* .011*

.001 .008 .004

t or z 2.440 2.541 2.492

Economic disadvantage b .003** .023* .012*

.001 .009 .005

t or z 2.590 2.483 2.329

Alcohol/drug problem b .158*** 1.029*** .589***

.022 .156 .092

t or z 7.060 6.611 6.429

F or model χ

(7 df ) 29.130*** 195.394*** 191.860***

.048

.060 .059

.047 .049

GSC

.085 .084

.127

∆

.051

* p ⬍ .05. ** p ⬍ .01. *** p ⬍ .001.

c07.qxd 8/27/2004 2:54 PM Page 249

being violent is estimated to be .161 higher than for married people. Each additional

year that the couple has been together is estimated to reduce the probability of vio-

lence by .004. The other eﬀects are interpreted similarly.

Problems with the LPM

Although OLS is adequate for a quick exploration of the nature and signiﬁcance of

predictor eﬀects, there are several diﬃculties with the LPM. To begin, the assump-

tion of a model that is linear in the parameters is problematic. The conditional mean

is a probability and is therefore conﬁned to the [0,1] interval. The linear predictor,

on the other hand, has no such constraints. For the linear predictor, 冱

, to be

bounded by 0 and 1, it is necessary a priori to constrain the β’s to be very small, par-

ticularly when the X’s exhibit considerable variability [see Aldrich and Nelson

(1984) for speciﬁc examples of this]. Without such constraints, probabilities would

fall outside their logical range. This is evident in the OLS estimates in Table 7.1. If

we calculate the estimated probability of violence for a “low-risk” couple, that is, a

married nonminority couple without alcohol or drug problems that is 2 standard

deviations above the mean on relationship duration (SD ⫽ 12.823) and female’s age

at union (SD ⫽ 7.081), 2 standard deviations below the mean on male’s isolation

(SD ⫽ 6.296), and 1 standard deviation below the mean on economic disadvantage

(SD ⫽ 5.128), we have π

⫽ .117 ⫺ .004(2)(12.823) ⫺ .003(2)(7.081) ⫹ .002(2)

(⫺6.296) ⫹ .003(⫺5.128) ⫽⫺.069, which is clearly an impossible value.

On the other hand, for the sake of argument, let’s suppose that the linear model

in equation (7.1) is the true model for the probability of interest. First, as long as the

mean structure—in this case, the structural model for π

—is speciﬁed correctly, the

parameter estimates will be consistent (Cameron and Trivedi, 1998). Nonetheless,

recall that estimation via OLS assumes that the errors have zero mean and constant

variance across all covariate patterns. Now, the model for an individual observation

is Y

⫽ π

⫹ ε

. The errors can take on only two possible values: ε

⫽ 1 ⫺ π

for cases

in the interest category and ε

⫽⫺π

for cases in the other category. As long as the

mean of the errors at each covariate pattern is zero, the OLS estimates will be unbi-

ased. To see that this holds here, note that the probability function f(ε) for the errors

is as follows: ε⫽ 1 ⫺π with probability π (i.e., the probability that Y ⫽ 1) and

ε ⫽⫺π with probability 1 ⫺ π (the probability that Y ⫽ 0). Thus, the mean of the

errors is

E(ε) ⫽

冱

εf(ε) ⫽ (1 ⫺ π)π ⫹ (⫺π)(1⫺ π) ⫽ 0.

What about the constant-variance assumption? The variance of the errors is

V(ε) ⫽ E(ε ⫺ E(ε))

⫽

冱

[ε ⫺ E(ε)]

f(ε) ⫽

冱

f(ε)

⫽ (1 ⫺ π)

π ⫹ (⫺π)

(1 ⫺ π) ⫽ π (1 ⫺ π)[(1 ⫺ π) ⫹ π]

⫽ π(1 ⫺ π). (7.2)

250 REGRESSION WITH A BINARY RESPONSE

c07.qxd 8/27/2004 2:54 PM Page 250

In that π is a function of the explanatory variables, as shown by equation (7.1), the

implication of expression (7.2) is that the errors vary with the X’s and are therefore

inherently heteroscedastic. The consequences of this, of course, are that the OLS

estimators are no longer those with the smallest sampling variance and that OLS

estimates of the standard errors of coeﬃcient estimates are biased. We could correct

these particular problems by using WLS estimation (Aldrich and Nelson, 1984).

However, the preferred approach is to employ a nonlinear probability model, which,

as we will see, makes more theoretical sense.

NONLINEAR PROBABILITY MODELS

Theoretically, the idea that probabilities are linear in explanatory variables is not par-

ticularly reasonable. Suppose that we are interested in modeling the eﬀect of salary

oﬀers on the probability of changing jobs. For someone who is “on the fence” on the

subject (i.e. someone with a 50–50 chance of changing jobs), a small increment

in salary might be enough to entice one to switch to the new job. On the other hand,

for those with either very low or very high probabilities of changing jobs, small

diﬀerences in salary between the current and alternative positions probably will have

little eﬀect on their intentions. In other words, the eﬀects of explanatory variables on

the probability of an event should be stronger among those with probabilities in the

middle range, but weaker when probabilities are near 0 or 1. Linear models, in con-

trast, assume that a predictor has the same eﬀect on the response, regardless of the

initial level of the response.

Latent-Variable Motivation of Probit and Logistic Regression

This idea can be developed more formally by assuming that our binary response is a

proxy for a latent continuous variable, or latent scale, which follows a linear regres-

sion model. That is, assume that there is an unobserved, continuous variable, Y

, such

that

⫽

冱

⫹ ε

and assume further that the ε

are independent random variables having a symmetric

distribution with zero mean and constant variance across values of x. What we

observe, on the other hand, is a binary indicator, Y

, that takes on the value 1 when-

ever Y

is greater than some threshold value, c, and that takes on the value 0 other-

wise. For convenience, it is usually assumed that c is 0. Then the probability that Y

is 1 is

P(Y

⫽ 1) ⫽ P(Y

⬎ 0) ⫽ P

冢

冱

⫹ ε

⬎ 0

冣

⫽ P

冢

⬎⫺

冱

冣

(7.3)

NONLINEAR PROBABILITY MODELS 251

c07.qxd 8/27/2004 2:54 PM Page 251

⫽ P

冢

⬍

冱

冣

(7.4)

⫽ F

冢

冱

冣

(7.5)

The terms (7.3) and (7.4) are equal because the distribution of ε is assumed to be

symmetric. (For example, the probability that a standard normal variable is greater

than ⫺z is the same as the probability that it is less than z.)

Expressions (7.4) and (7.5) imply that the probability of Y being 1 can be modeled

as a distribution function of ε, denoted F(ε). The two distributions most often employed

are the standard normal distribution and the standard logistic distribution. Figure 7.1

depicts the standard normal (probit) and standard logistic (logit) densities. They are both

bell-shaped curves, but the logistic density has greater spread. The variances of these

densities are 1 for the standard normal and π

/3 (approximately equal to 3.29) for the

standard logistic. The formulas for these functions are as follows. The standard normal

density function (which uses the special symbol, φ) for a variable X,is

φ(x) ⫽

ᎏ

兹

苶

ᎏ

exp

冢

⫺

ᎏ

冣

, (7.6)

252 REGRESSION WITH A BINARY RESPONSE

Figure 7.1 Standard normal (probit) and standard logistic densities.

c07.qxd 8/27/2004 2:54 PM Page 252

whereas the standard logistic density (which uses the symbol λ) is

λ(x) ⫽

ᎏ

[1 ⫹

(

)

x)]

ᎏ

. (7.7)

Recall that the distribution function for a variable X,F(x), is the probability that

X ⱕ x. This is just the area under the curve to the left of x. Hence, the standard nor-

mal distribution function, represented by the special symbol, Φ,is

Φ(x) ⫽ 冕

⫺∞

ᎏ

兹

苶

ᎏ

exp

冢

⫺

ᎏ

冣

du,

where the integration symbol (冕

⫺∞

) indicates the area under the stated function

between negative inﬁnity and x. This integral has no closed-form solution and must

be obtained by actually summing areas under the standard normal curve. However,

tables of areas under the standard normal curve can easily be used to ﬁnd F( x) given

any value of x. The standard logistic distribution function, denoted with the symbol

Λ,is

Λ(x) ⫽

ᎏ

1 ⫹

(

(x)

ᎏ

which, unlike the standard normal, does have a closed form.

Figure 7.2 depicts the standard normal and standard logistic distribution func-

tions. It is evident that both are S-shaped curves that range from 0 to 1, as is logical

for a probability. The curves are quite similar, except that the logistic curve approaches

both 0 and 1 more gradually than does the normal. Indeed, it is not necessary to pose

a latent variable as underlying the model. We could simply choose a distribution

function as our model for probability due to its property of remaining within the [0,1]

interval for any value of x. Notice also that for both curves, the rate at which the

probability changes with x is minimal when the probability is either very small or

very large. As mentioned above, this is also a theoretically desirable property of a

probability function.

At this point, to simplify matters, let z

⫽ 冱β

. Then the probit model for

P(Y

⫽ 1) ⫽ π

, following expression (7.5), is

⫽ Φ(z

) ⫽ 冕

⫺∞

ᎏ

兹

苶

ᎏ

exp

冢

⫺

ᎏ

冣

du, (7.8)

and the logistic regression model for π

⫽ Λ(z

) ⫽

ᎏ

1 ⫹

(

)

ᎏ

. (7.9)

It should be clear that these are both nonlinear functions of the parameters (i.e., the

betas). Nevertheless, both models can be linearized, using the appropriate link func-

tion. Recall from Chapter 1 that in generalized linear models, which encompass both

probit and logit, the link function is a transformation of the mean of Y (in this case,

π) that equals the linear predictor, 冱β

. The linear form of the probit model in

equation (7.8) is, therefore,

NONLINEAR PROBABILITY MODELS 253

c07.qxd 8/27/2004 2:54 PM Page 253

⫺1

(π

) ⫽

冱

, (7.10)

where Φ

⫺1

(π

), called the probit, or normit, link, is the z

such that F(z

) ⫽ π

. [For

example, Φ

⫺1

(.025) ⫽⫺1.96, since Φ(⫺1.96) ⫽ .025.] The linear form of the logis-

tic regression model in equation (7.9) uses the logit link (as derived in Chapter 1)

ᎏ

1 ⫺

ᎏ

⫽

冱

. (7.11)

Equations (7.10) and (7.11) now resemble ordinary regression models in that the

response variables can take on any value in the real numbers, just as can the linear

predictor on the right-hand side. The coeﬃcients, rather than indicating the eﬀects

of explanatory variables on Y

, indicate the eﬀects on the probit or logit link. Below

I consider more intuitive interpretations of the β’s, especially in the logit model.

Estimation

Both probit and logit models are constructed by assuming that a particular density

underlies the data. Hence, these models are typically estimated using maximum

254 REGRESSION WITH A BINARY RESPONSE

Figure 7.2 Standard normal (probit) and standard logistic distributions.

c07.qxd 8/27/2004 2:54 PM Page 254