Dougherty С. Introduction to Econometrics, 3Ed

Подождите немного. Документ загружается.

COVARIANCE, VARIANCE, AND CORRELATION

Figure 1.1.

Hourly earnings and schooling, 20 NLSY respondents

The sample covariance, Cov(X, Y), is a statistic that enables you to summarize this association with a

single number. In general, given n observations on two variables X and Y, the sample covariance

between X and Y is given by

∑

−−=

−−++−−=

YYXX

YYXXYYXX

))((

)])((...))([(

),(Cov

(1.1)

where, as usual, a bar over the symbol for a variable denotes its sample mean.

Note: In Section 1.4 we will also define the population covariance. To distinguish between the

two, we will use Cov(X, Y) to refer to the sample covariance and

to refer to the population

covariance between

and

. This convention is parallel to the one we will use for variance: Var(

)

referring to the sample variance, and

referring to the population variance.

Further note:

Some texts define sample covariance, and sample variance, dividing by

–1 instead

, for reasons that will be explained in Section 1.5.

The calculation of the sample covariance for

and

is shown in Table 1.2. We start by

calculating the sample means for schooling and earnings, which we will denote

and

13.250 and

is 14.225. We then calculate the deviations of

and

from these means for each

individual in the sample (fourth and fifth columns of the table). Next we calculate the product of the

deviations for each individual (sixth column). Finally we calculate the mean of these products,

15.294, and this is the sample covariance.

You will note that in this case the covariance is positive. This is as you would expect. A positive

association, as in this example, will be summarized by a positive sample covariance, and a negative

association by a negative one.

0 2 4 6 8 10 12 14 16 18 20

hest

rade completed

Hourly earnings ($)

COVARIANCE, VARIANCE, AND CORRELATION

ABLE

1.2

Observation S Y

(S –

S )

(Y –

Y )

(S –

S )(Y – Y )

1 15 17.24 1.75 3.016 5.277

2 16 15.00 2.75 0.775 2.133

3 8 14.91 –5.25 0.685 –3.599

464.5–7.25 –9.725 70.503

5 15 18.00 1.75 3.776 6.607

6126.29–1.25 –7.935 9.918

7 12 19.23 –1.25 5.006 –6.257

8 18 18.69 4.75 4.466 21.211

9127.21–1.25 –7.015 8.768

10 20 42.06 6.75 27.836 187.890

11 17 15.38 3.75 1.156 4.333

12 12 12.70 –1.25 –1.525 1.906

13 12 26.00 –1.25 11.776 –14.719

14 9 7.50 –4.25 –6.725 28.579

15 15 5.00 1.75 –9.225 –16.143

16 12 21.63 –1.25 7.406 –9.257

17 16 12.10 2.75 –2.125 –5.842

18 12 5.55 –1.25 –8.675 10.843

19 12 7.50 –1.25 –6.725 8.406

20 14 8.00 0.75 –6.225 –4.668

Total 265 284.49 305.888

Average 13.250 14.225 15.294

Figure 1.2.

0 2 4 6 8 10 12 14 16 18 20

hest

rade completed

Hourly earnings ($)

COVARIANCE, VARIANCE, AND CORRELATION

It is worthwhile investigating the reason for this. Figure 1.2 is the same as Figure 1.1, but the

scatter of observations has been quartered by vertical and horizontal lines drawn through the points

and

, respectively. The intersection of these lines is therefore the point ( S ,

), the point giving

mean schooling and mean hourly earnings for the sample. To use a physical analogy, this is the center

of gravity of the points representing the observations.

Any point lying in quadrant A is for an individual with above-average schooling and above-

average earnings. For such an observation, both (S –

S ) and (Y –

) are positive, and (S – S )(Y –

) must therefore be positive, so the observation makes a positive contribution to the covariance

expression. Example: Individual 10, who majored in biology in college and then went to medical

school, has 20 years of schooling and her earnings are the equivalent of $42.06 per hour. (S –

S ) is

6.75, (Y –

) is 27.84, and the product is 187.89.

Next consider quadrant B. Here the individuals have above-average schooling but below-average

earnings. (S –

S ) is positive, but (Y –

) is negative, so (S – S )(Y –

) is negative and the

contribution to the covariance is negative. Example: Individual 20 completed two years of four-year

college majoring in media studies, but then dropped out, and earns only $8.00 per hour working in the

office of an automobile repair shop.

In quadrant C, both schooling and earnings are below average, so (S –

S ) and (Y –

) are both

negative, and (S –

S )(Y –

) is positive. Example: Individual 4, who was born in Mexico and had

only six years of schooling, is a manual worker in a market garden and has very low earnings.

Finally, individuals in quadrant D have above average earnings despite having below-average

schooling, so (S –

S ) is negative, (Y –

) is positive, and (S – S )(Y –

) makes a negative

contribution to the covariance. Example: Individual 3 has slightly above-average earnings as a

construction laborer, despite only completing elementary school.

Since the sample covariance is the average value of (S –

S )(Y –

) for the 20 observations, it

will be positive if positive contributions from quadrants A and C dominate and negative if the negative

ones from quadrants B and D dominate. In other words, the sample covariance will be positive if, as

in this example, the scatter is upward-sloping, and negative if the scatter is downward-sloping.

1.2 Some Basic Covariance Rules

There are some rules that follow in a perfectly straightforward way from the definition of covariance,

and since they are going to be used many times in future chapters it is worthwhile establishing them

immediately:

Covariance Rule 1 If Y = V + W, Cov(X, Y) = Cov(X, V) + Cov(X, W)

Covariance Rule 2 If Y = bZ, where b is a constant and Z is a variable,

Cov(X, Y) = bCov(X, Z)

Covariance Rule 3 If Y = b, where b is a constant, Cov(X, Y) = 0

COVARIANCE, VARIANCE, AND CORRELATION

Proof of Covariance Rule 1

Since Y = V + W, Y

= V

+ W

and WVY

. Hence

),(Cov),(Cov

))((

])[])([(

))((

),(Cov

WXVX

WWXX

VVXX

WWVVXX

WVWVXX

YYXX

iii

−−+−−=

−+−−=

+−+−=

−−=

∑∑

∑

(1.2)

Proof of Covariance Rule 2

If Y = bZ, Y

= bZ

and ZbY

. Hence

),(Cov))((

))((

),(Cov

ZXbZZXX

ZbbZXX

YYXX

=−−=

−−=−−=

∑

∑∑

(1.3)

Proof of Covariance Rule 3

This is trivial. If Y = b,

= b and Y

–

= 0 for all observations. Hence

0)0)((

))((

),(Cov

=−=

−−=−−=

∑

∑∑

bbXX

YYXX

(1.4)

Further Developments

With these basic rules, you can simplify much more complicated covariance expressions. For

example, if a variable Y is equal to the sum of three variables U, V, and W,

Cov(X, Y) = Cov(X, [U + V + W]) = Cov(X, U ) + Cov(X, [V + W]) (1.5)

COVARIANCE, VARIANCE, AND CORRELATION

using Rule 1 and breaking up Y into two parts, U and V+W. Hence

Cov(X, Y) = Cov(X, U ) + Cov(X, V ) + Cov(X, W) (1.6)

using Rule 1 again.

Another example: If Y = b

+ b

Z, where b

and b

are constants and Z is a variable,

Cov(X, Y) = Cov(X, [b

+ b

Z])

= Cov(X, b

) + Cov(X, b

Z) using Rule 1

= 0 + Cov(X, b

Z) using Rule 3

= b

Cov(X, Z) using Rule 2 (1.7)

1.3 Alternative Expression for Sample Covariance

The sample covariance between X and Y has been defined as

∑

−−=

YYXX

))((

),(Cov

(1.8)

An alternative, and equivalent, expression is

Cov(X, Y) =

YXYX

−













∑

(1.9)

You may find this to be more convenient if you are unfortunate enough to have to calculate a

covariance by hand. In practice you will normally perform calculations of this kind using a statistical

package on a computer.

It is easy to prove that the two expressions are equivalent.

)

...

(

)(

))((

),(Cov

1111

YXYXYXYX

YYXX

nnnn

iiii

+−−+

+−−=

−−=

∑

(1.10)

Adding each column, and using the fact that

XnX

∑

and

YnY

∑

COVARIANCE, VARIANCE, AND CORRELATION

YXYX

YXnYX

YXnYXnYXnYX

YXnYXXYYX

−

























−=













+−−=













+−−=

∑

∑∑∑

===

111

),(Cov

(1.11)

Exercises

1.1

In a large bureaucracy the annual salary of each individual,

, is determined by the formula

= 10,000 + 500

+ 200

where

is the number of years of schooling of the individual and

is the length of time, in years,

of employment.

is the individual’s age. Calculate Cov(

), Cov(

), and Cov(

) for the

sample of five individuals shown below and verify that

Cov(

) = 500Cov(

) + 200Cov(

Explain analytically why this should be the case.

Individual

Age

(years)

Years of

Schooling

Length of

Service Salary

1 18 11 1 15,700

2 29 14 6 18,200

3 33 12 8 17,600

4 35 16 10 20,000

5 45 12 5 17,000

1.2*

In a certain country the tax paid by a firm,

, is determined by the rule

= –1.2 + 0.2

– 0.1

where

is profits and

is investment, the third term being the effect of an investment incentive.

is sales. All variables are measured in $ million at annual rates. Calculate Cov(

), Cov(

and Cov(

) for the sample of four firms shown below and verify that

Cov(

) = 0.2Cov(

) – 0.1Cov(

Explain analytically why this should be the case.

COVARIANCE, VARIANCE, AND CORRELATION

Firm Sales Profits Investment Tax

1 100 20 10 1.8

2509 40.2

38012 40.8

47015 61.2

1.4 Population Covariance

If X and Y are random variables, the expected value of the product of their deviations from their means

is defined to be the population covariance,

= E[(X –

)(Y –

)] (1.12)

where

and

are the population means of X and Y, respectively.

As you would expect, if the population covariance is unknown, the sample covariance will

provide an estimate of it, given a sample of observations. However, the estimate will be biased

downwards, for

E[Cov(X, Y)] =

−

(1.13)

The reason is that the sample deviations are measured from the sample means of X and Y and tend to

underestimate the deviations from the true means. Obviously we can construct an unbiased estimator

by multiplying the sample estimate by n/(n–1). A proof of (1.13) will not be given here, but you could

construct one yourself using Appendix R.3 as a guide (first read Section 1.5). The rules for population

covariance are exactly the same as those for sample covariance, but the proofs will be omitted because

they require integral calculus.

If X and Y are independent, their population covariance is 0, since then

E[(X –

)(Y –

)] = E(X –

)E(Y –

)

= 0

0 (1.14)

by virtue of the independence property noted in the Review and the fact that E(X) and E(Y) are equal

and

, respectively.

1.5 Sample Variance

In the Review the term variance was used to refer to the population variance. For purposes that will

become apparent in the discussion of regression analysis, it will be useful to introduce, with three

warnings, the notion of sample variance. For a sample of n observations, X

, ..., X

, the sample

variance will be defined as the average squared deviation in the sample:

COVARIANCE, VARIANCE, AND CORRELATION

∑

−=

)(

)(Var

(1.15)

The three warnings are:

1. The sample variance, thus defined, is a biased estimator of the population variance. Appendix

R.3 demonstrates that

, defined as

∑

−

)(

is an unbiased estimator of

. It follows that the expected value of Var(

) is [(

–1)/

]

and

that it is therefore biased downwards. Note that as

becomes large, (

–1)/

tends to 1, so the

bias becomes progressively attenuated. It can easily be shown that plim Var(

) is equal to

and hence that it is an example of a consistent estimator that is biased for small samples.

2. Because

is unbiased, some texts prefer to define it as the sample variance and either avoid

referring to Var(

) at all or find some other name for it. Unfortunately, there is no generally

agreed convention on this point. In each text, you must check the definition.

3. Because there is no agreed convention, there is no agreed notation, and a great many symbols

have been pressed into service. In this text the population variance of a variable

denoted

. If there is no ambiguity concerning the variable in question, the subscript may be

dropped. The sample variance will always be denoted Var(

Why does the sample variance underestimate the population variance? The reason is that it is

calculated as the average squared deviation from the sample mean rather than the true mean. Because

the sample mean is automatically in the center of the sample, the deviations from it tend to be smaller

than those from the population mean.

1.6 Variance Rules

There are some straightforward and very useful rules for variances, which are counterparts of those for

covariance discussed in Section 1.2. They apply equally to sample variance and population variance:

Variance Rule 1

, Var(

) = Var(

) + Var(

) + 2Cov(

Variance Rule 2

, where

is a constant, Var(

) =

Var(

Variance Rule 3

, where

is a constant, Var(

) = 0.

Variance Rule 4

, where

is a constant, Var(

) = Var(

First, note that the variance of a variable

can be thought of as the covariance of

with itself:

COVARIANCE, VARIANCE, AND CORRELATION

),(Cov))((

)(

)(Var

XXXXXX

=−−=−=

∑∑

. (1.16)

In view of this equivalence, we can make use of the covariance rules to establish the variance rules.

We are also able to obtain an alternative form for Var(X), making use of (1.9), the alternative form for

sample covariance:

)(Var XX

−













∑

(1.17)

Proof of Variance Rule 1

If Y = V + W,

)Cov(),Cov(])[,Cov(),Cov()Var( Y,WVYWVYYYY

+=+==

using Covariance Rule 1

)],([Cov)],([Cov WWVVWV

+++=

),(Cov),(Cov),(Cov),(Cov WWWVVWVV

+++=

using Covariance Rule 1 again

),(Cov2)(Var)(Var WVWV

++=

(1.18)

Proof of Variance Rule 2

If Y = bZ, where b is a constant, using Covariance Rule 2 twice,

)Var(),Cov(=)Cov(

),Cov(),Cov(),Cov()Var(

ZbZZbZ,aZb

YZbYbZYYY

===

(1.19)

Proof of Variance Rule 3

If Y = b, where b is a constant, using Covariance Rule 3,

Var(Y) = Cov(b, b) = 0 (1.20)

This is trivial. If Y is a constant, its average value is the same constant and (Y –

) is 0 for all

observations. Hence Var(Y) is 0.

Proof of Variance Rule 4

If Y = V + b, where V is a variable and b is a constant, using Variance Rule 1,

COVARIANCE, VARIANCE, AND CORRELATION

Var(Y) = Var(V + b) = Var(V) + Var(b) + 2Cov(V, b)

= Var(V) (1.21)

Population variance obeys the same rules, but again the proofs are omitted because they require

integral calculus.

Exercises

1.3

Using the data in Exercise 1.1, calculate Var(Y), Var(S), Var(T), and Cov(S, T) and verify that

Var(Y) = 250,000 Var(X) + 40,000 Var(T) + 200,000 Cov(S, T),

explaining analytically why this should be the case.

1.4*

Using the data in Exercise 1.2, calculate Var(T), Var(P), Var(I) and Cov(P, I), and verify that

Var(T) = 0.04Var(P) + 0.01Var(I) – 0.04Cov(P, I),

explaining analytically why this should be the case.

1.7 Population Variance of the Sample Mean

If two variables X and Y are independent (and hence their population covariance

is 0), the

population variance of their sum is equal to the sum of their population variances:

222

XYYXYX

σσ

σσσσ

++=

(1.22)

This result can be extended to obtain the general rule that the population variance of the sum of

any number of mutually independent variables is equal to the sum of their variances, and one is able to

show that, if a random variable X has variance

, the population variance of the sample mean,

will be equal to

/n, where n is the number of observations in the sample, provided that the

observations are generated independently.