Alfred DeMaris - Regression with Social Data, Modeling Continuous and Limited Response Variables
Подождите немного. Документ загружается.
this slope is known, given that we know a point on the line, P(x
0
,y
0
), the equation for
the line is simply (y y
0
) b
tan
(x x
0
) (see Section (I.P).
As we saw earlier (i.e., in Section I.P), the slope of a straight line indicates the
number of units that Y changes as X increases by 1 unit, along the line. However, for
a nonlinear function of X, as shown in the figure, the slope of the line passing
through point P on the curve is continually changing as P moves along the curve.
Similarly, along the curve, f(x), Y does not change at a uniform rate as X increases.
Rather, at some points along the curve Y is changing very rapidly with X (the slope
at P is very steep), while at other points Y is changing more slowly (the slope at P is
very shallow). Only along a straight line does Y change at a uniform rate with X.
Hence, for any function in general, the slope, b
tan
, indicates the instantaneous rate
of change of Y with increase in X at a particular point, x.
Generally, the change in a function f(x) as X increases from x
0
to x
1
is f(x
1
)
f(x
0
). The ratio of this change to the increase in X is then [f(x
1
) f(x
0
)]/(x
1
x
0
).
This represents the average rate of change in the function with change in X over the
interval from x
0
to x
1
. This ratio is also the slope of the line joining point P(x
0
,y
0
) and
point Q(x
1
,y
1
) in Figure A.2, called a secant line for the curve and denoted “S” in the
figure. We denote the slope of this line as b
sec
. This slope is a very crude approxi-
mation to b
tan
. However, if we move point Q closer to point P and then recalculate
MATHEMATICS TUTORIALS 465
Figure A.2 Derivative of a function of X as the slope of the line tangent to the curve at the point P(x
0
, y
0
).
bapp01.qxd 30.8.04 11:46 Page 465
b
sec
[f(x
1
) f(x
0
)]/(x
1
x
0
), b
sec
will become a better approximation to b
tan
. If we
continue in this fashion, moving Q increasingly closer to P and calculating b
sec
each
time, the line S will rotate toward a “limiting” position as Q converges to the point
P. That position will be T, the tangent line of interest. This notion motivates the
following definition.
Definition. The slope, b
tan
, of the line tangent to the curve at the point (x
0
,y
0
) is
b
tan
lim
x
1
→x
0
f(x
x
1
1
)
f
x
(
0
x
0
)
.
In English, this says that the slope of the line tangent to a function at a point is the
value approached by the ratio of the change in the function to the change in x at that
point as the change in x diminishes to zero (i.e., as x
1
converges to x
0
). This expres-
sion is so important that it is given some special notation (that is more consistent
with the “English interpretation” above). First, let ∆x x
1
x
0
, so that x
1
x
0
∆x.
Also let b
tan
f(x
0
). This slope is also called the first derivative of f with respect to
X at the point x
0
. In this altered notation, the first derivative is defined as
f(x
0
) lim
∆x→0
f(x
0
∆
∆
x
x
) f(x
0
)
.
Thus, the first derivative is the slope of the line tangent to the curve, f(x), at x
0
[or at
the point P(x
0
,y
0
)]. More intuitively, as noted previously, it is the instantaneous rate
of change in Y along the function f(x), with increase in X, at the point x
0
. Because of
this interpretation, the first derivative is ideal for quantifying the causal impact of X
on Y, at least at a particular value of X.
It is typical to present the formula for the first derivative in terms of x, rather than
x
0
, making the notation more general. Thus, the first derivative of any function f(x),
at the point x,is
f(x) lim
∆x→0
f(x ∆
∆
x
x
) f(x)
Example 1. Let’s see how this works. Suppose that we calculate the first derivative
of f(x) at the point x, where f(x) α βx. (Notice that this is the simple linear
regression function, as for the mean of Y at a given x.) We have
f(x) lim
∆x→0
lim
∆x→0
lim
∆x→0
β
∆
∆
x
x
lim
∆x→0
β β.
The last equality results because the expression we are computing the limiting value
for (i.e., β) is no longer a function of ∆x. In fact, it is a constant value regardless of
what ∆x converges to. Hence its limit, as ∆x converges to zero, remains at that value.
α βxβ∆xα βx
∆x
α β(x∆x)(α βx)
∆x
466 MATHEMATICS TUTORIALS
bapp01.qxd 30.8.04 11:46 Page 466
This example demonstrates that the slope of f(x) in the simple linear regression
model is β, a constant. That is, E(Y) changes at a constant rate with increase in X,at
any point x, along the regression line.
Example 2. Now suppose that f(x) x
2
. Again, the derivative is
f(x) lim
∆x→0
f(x ∆
∆
x
x
) f(x)
lim
∆x→0
(x∆
∆
x
x
)
2
x
2
lim
∆x→0
lim
∆x→0
∆x(2
∆
x
x
∆x)
lim
∆x→0
2x ∆x 2x 0 2x.
In this case, the first derivative, f(x) 2x, is a function of x. That is, the rate of
change of y with increase in x depends on the value of x. Thus, unlike a linear func-
tion of x, this nonlinear function of x (i.e., x
2
) has a nonconstant slope. For example,
when x 1, the slope is 2(1) 2, so y is increasing at twice the rate of x. When x 2,
on the other hand, the slope is 2(2) 4, so y is increasing at four times the rate of x,
and so on.
B. Rules for Finding Derivatives
Although we have defined the first derivative in terms of a limiting value above, it is
not necessary to compute this limit every time we want to find the first derivative.
Fortunately, there is a set of algebraic rules that provide derivatives for the functions
that we will encounter. First, let us introduce some additional notation. The first
derivative of f(x) with respect to x is also denoted
d
d
x
f(x). This notation will be
especially useful when more complicated functions are involved. The rules are:
(1)
d
d
x
(c) 0, where c is a constant (i.e., the derivative of a constant with respect
to x equals zero).
Example:
d
d
x
(5) 0.
(2)
d
d
x
e
x
e
x
.
(3)
d
d
x
ln x
1
x
.
(4)
d
d
x
cf(x) c
d
d
x
f(x), where c is a constant.
x
2
2x ∆x (∆x)
2
x
2
∆x
MATHEMATICS TUTORIALS 467
bapp01.qxd 30.8.04 11:46 Page 467
Example:
d
d
x
(3.75 ln x) 3.75
d
d
x
lnx
3.
x
75
.
(5)
d
d
x
x
r
rx
r1
, where r is any real-valued exponent.
Examples:
d
d
x
x
2
2x;
d
d
x
x
5
5x
4
;
d
d
x
x
2/3
2
3
x
1/3
;
d
d
x
4x
3
12x
2
.
(6)
d
d
x
[f(x) g(x)]
d
d
x
f(x)
d
d
x
g(x).
Example:
d
d
x
(ln x x
2
)
d
d
x
(ln x)
d
d
x
(x
2
)
1
x
2x.
(7)
d
d
x
[f(x)g(x)] f(x)g(x) g(x)f(x).
Example: Find
d
d
x
(兹x
苶
ln x).
Solution: Let f(x) 兹x
苶
x
1/2
and g(x) ln x. Then f(x)
1
2
x
1/2
1/2兹x
苶
, and
g(x) 1/x. Hence,
f(x)g(x) g(x)f(x) 兹x
苶
1
x
(ln x)
2兹
1
x
苶
兹
1
x
苶
2
l
兹
n x
x
苶
2
2
兹
ln
x
苶
x
.
(8)
d
d
x
冤
g
f(
(
x
x
)
)
冥
.
Example: Find
d
d
x
冢
兹
e
x
x
苶
冣
.
Solution: Let f(x) e
x
and g(x) 兹x
苶
. Then f(x) e
x
and g(x)
1
2
x
1/2
1/2兹x
苶
.
Hence,
兹x
苶
e
x
e
x
x
(1/2兹x
苶
)
e
x
2
2
x
x
3/2
1
.
One final rule that is particularly useful is based on the composition of two func-
tions, f(x) and g(x).
Definition. Given two functions, f(x) and g(x), the composition of f(x) with g(x) is
the function defined by f(g(x)).
This needs some explanation. The composition of f with g essentially takes g(x)
as the input to f(x). That is, “x” in f(x) becomes “g(x).” We then perform on g(x) the
operations specified by the function f(x). Another way to view this operation is to
call f(x) the “outside function” and g(x) the “inside function.” Then f(g(x)) is the out-
side function evaluated at the inside function of x. [Exercises (14) to (16) in Section
I.Q already involved compositions of functions.]
Example. Suppose that f(x) x
2
and g(x) ln x. Then the composition f(g(x)) is (ln
x)
2
. Notice that g(x) ln x becomes “x,” or the input, to the function f(x) x
2
.
g(x)f(x) f(x)g(x)
[g(x)]
2
g(x)f(x) f(x)g(x)
[g(x)]
2
468 MATHEMATICS TUTORIALS
bapp01.qxd 30.8.04 11:46 Page 468
As another example, suppose that f(x) e
x
and g(x) α βx. Then f(g(x)) e
α βx
.
As a third example, suppose that f(x) x
2
and g(x) y a bx. Then f(g(x))
(y a bx)
2
.
Chain Rule. If g(x) is differentiable at the point x and f(x) is differentiable at the
point g(x), the composition of f with g, f(g(x)), is differentiable at the point x.
Moreover,
(9)
d
d
x
[f(g(x))] f[g(x)]g(x)
冤
dg
d
(x)
f(g(x))
冥冤
d
d
x
g(x)
冥
.
That is, the derivative of f(g(x)) is the derivative of the outside function evaluated at
the inside function times the derivative of the inside function with respect to x.
Example 1. Find
d
d
x
ln x
2
.
Solution: Let f(x) ln x and g(x) x
2
. That is, ln x is the outside function and x
2
is
the inside function, so that f(g(x)) is ln(x
2
) ln x
2
. Then
d
d
x
ln x
2
冢
d
d
x
2
ln x
2
冣冢
d
d
x
x
2
冣
x
1
2
(2x)
2
x
.
Example 2. Find
d
d
x
(5 x
2
)
3
.
Solution: One way to solve this, of course, is to find the cube of (5 x
2
) and then
take the first derivative with respect to x of the resulting expression. This is the “long
way around.” Using rule IV.B(9), however, we have
d
d
x
(5 x
2
)
3
d(5
d
x
2
)
(5 x
2
)
3
d
d
x
(5 x
2
) 3(5 x
2
)
2
(2x) 6x(5 x
2
)
2
.
The reader can verify that the same answer is arrived at going the long way around.
Example 3. Find
d
d
x
e
5兹x
苶
.
Solution: Let f(x) e
x
and g(x) 5兹x
苶
. Then
d
d
x
e
5兹x
苶
d5
d
兹x
苶
e
5兹x
苶
d
d
x
5兹x
苶
e
5兹x
苶
冢
2兹
5
x
苶
冣
5
2
e
兹
5兹
x
苶
x
苶
.
Higher-Order Derivatives. The second-, third-, and higher-order derivatives of f(x)
with respect to x are just the derivatives of the next-lower-order derivative. That is,
f(x)
d
d
x
f(x) is the second derivative of f(x) with respect to x,
f(x)
d
d
x
f(x) is the third derivative of f(x) with respect to x, and so on.
MATHEMATICS TUTORIALS 469
bapp01.qxd 30.8.04 11:46 Page 469
Example. Find the second derivative of f(x) x
3
with respect to x.
Solution:f(x) 3x
2
,f(x)
d
d
x
(3x
2
) 6x.
In this book we are not concerned with derivatives any higher than second order.
C. Partial Derivatives
Partial derivatives are derivatives with respect to a particular variable, say x, for func-
tions in more than one variable. Suppose that y f(x,w,z). That is, f is now a func-
tion of three independent variables, x, w, and z. Suppose that we fix w and z at the
respective values of w
0
and z
0
. Then y f(x,w
0
,z
0
) is only a function of x. In other
words, holding w and z constant at the values of w
0
and z
0
, respectively, the values
of y change only as we plug different values of x into the function. These resulting
values of y (as x varies) then represent the “curve” of the function as x takes on
different values. This brings us to the following.
Definition. The first partial derivative of f(x,w
0
,z
0
) with respect to x, at x
0
, is the
slope of the tangent line to the curve f(x,w
0
,z
0
) at the point (x
0
,w
0
,z
0
).
The first partial derivative of f(x,w,z) with respect to x is denoted:
∂
∂
x
f(x,w,z)orf
x
(x,w,z).
The partial derivative (unless otherwise stated, we assume that this refers to the
first partial derivative) is calculated by treating all other independent variables in the
function as though they are constants. In that we are fixing them at specific values
(e.g., w
0
,z
0
), this makes intuitive sense.
Example 1. Find
∂
∂
x
(.5 1.2x 3w 兹2
苶
z).
Solution: In that w and z are being treated as constants, the only term involving x is
“1.2x.” Hence,
∂
∂
x
(.5 1.2x 3w 兹2
苶
z) 1.2.
Example 2. Find
∂
∂
x
(xwz
2
w兹x
苶
z
3
ln x).
Solution:
∂
∂
x
(xwz
2
w兹x
苶
z
3
ln x) wz
2
1
2
wx
1/2
z
x
3
wz
2
2兹
w
x
苶
z
x
3
.
Example 3. Find
∂
∂
X
3
(α β
1
X
1
β
2
X
2
β
3
X
3
β
4
X
2
X
3
).
Solution:
∂
∂
X
3
(α β
1
X
1
β
2
X
2
β
3
X
3
β
4
X
2
X
3
) β
3
β
4
X
2
.
Second Partial Derivatives. We can compute second- and higher-order partial
derivatives. These are just derivatives of the next-lower-order partial derivative.
470 MATHEMATICS TUTORIALS
bapp01.qxd 30.8.04 11:46 Page 470
Again, we will only be concerned with first- and second-order partial derivatives in
this work. Because the function in question consists of more than one independent
variable, second-order derivatives can be constructed either with respect to the same,
or a different, variable than is true for the first partial derivative. Hence, for y f(x,z)
we have
∂
∂
x
2
y
2
∂
∂
x
冢
∂
∂
y
x
冣
: the second partial derivative of y with respect to x,
also denoted f
xx
(x,z).
∂
∂
z
2
2
y
∂
∂
z
冢
∂
∂
y
z
冣
: the second partial derivative of y with respect to z,
also denoted f
zz
(x,z).
∂
∂
z
2
∂
y
x
∂
∂
z
冢
∂
∂
y
x
冣
: the mixed second partial with respect to x, then z,
also denoted f
xz
(x,z).
∂
∂
x
2
∂
y
z
∂
∂
x
冢
∂
∂
y
z
冣
: the mixed second partial with respect to z, then x,
also denoted f
zx
(x,z).
Second partial derivatives with respect to each variable or each pair of variables can
be constructed for functions of several variables as well as for functions of just two
variables. As with first partial derivatives, the second partial derivative with respect
to a given variable is constructed by treating all other variables as constants.
Example 1. Find
∂
∂
x
2
2
(xwz
2
w兹x
苶
z
3
ln x).
Solution: From Example 2 in Section IV.C we found that
∂
∂
x
(xwz
2
w兹x
苶
z
3
ln x) wz
2
1
2
wx
1/2
z
x
3
. Hence,
∂
∂
x
2
2
(xwz
2
w兹x
苶
z
3
ln x)
∂
∂
x
冢
wz
2
1
2
wx
1/2
z
3
x
1
冣
1
4
wx
3/2
z
3
x
2
4
x
3
w
/2
x
z
3
2
.
Example 2. Find
∂
∂
z
2
2
(xwz
2
w兹x
苶
z
3
ln x).
Solution: First, we note that
∂
∂
z
(xwz
2
w兹x
苶
z
3
ln x) 2xwz 3z
2
ln x. Then we
have that
∂
∂
z
2
2
(xwz
2
w兹x
苶
z
3
ln x)
∂
∂
z
(2xwz 3z
2
ln x) 2xw 6z ln x.
Example 3. Find
∂z
∂
∂
2
x
(xwz
2
w兹x
苶
z
3
ln x).
MATHEMATICS TUTORIALS 471
bapp01.qxd 30.8.04 11:46 Page 471
Solution: We saw that
∂
∂
x
(xwz
2
w兹x
苶
z
3
ln x) wz
2
1
2
wx
1/2
z
x
3
, Hence,
∂z
∂
∂
2
x
(xwz
2
w兹x
苶
z
3
ln x)
∂
∂
z
冢
wz
2
1
2
wx
1/2
z
3
x
1
冣
2wz 3z
2
x
1
.
Example 4. Find
∂x
∂
2
∂z
(xwz
2
w兹x
苶
z
3
ln x).
Solution:
∂x
∂
2
∂z
(xwz
2
w兹x
苶
z
3
ln x)
∂
∂
x
冤
∂
∂
z
(xwz
2
w兹x
苶
z
3
ln x)
冥
∂
∂
x
(2xwz 3z
2
ln x) 2wz 3z
2
x
1
.
It is not just coincidence that the mixed second partial derivatives in Examples 3
and 4 are equal. By theorem, the mixed second partial derivatives are equal for a
wide class of functions (Munem and Foulis, 1984).
D. Exercises
(1) Find
d
d
x
(x
3
).
(2) Find
d
d
x
(x
4/5
).
(3) Find
d
d
x
(3x
4
2x
3
x
2
1).
(4) Find f(x) for f(x) ln x x
3
and evaluate f(3). [Hint: First find f(x) in
general, then plug in “3” for x and evaluate the result.]
(5) Find f(2) for f(x) e
x
x
2
.
(6) Find f(x) if f(x) x
3
e
x
.
(7) Find f(x) if f(x) x ln x.
(8) Find f(x) if f(x) 兹x
苶
ln x
2
.
(9) Find f(x) if f(x)
兹
ln x
x
苶
.
(10) Find f(x) if f(x) x
4
.
(11) Find f(x) if f(x) ln x.
(12) Find f(x) if f(x) e
5x
.
(13) Find f(x) if f(x) (ln x)
2
.
(14) Find f(x) if f(x) e
x
2
.
(15) Find
d
d
x
冢
1
e
x
e
x
冣
.
(16) Find
d
d
x
冢
1
e
α
e
α
β
x
βx
冣
.
472 MATHEMATICS TUTORIALS
bapp01.qxd 30.8.04 11:46 Page 472
(17) Find f(x) if f(x) ln x
2
.
(18) Find f(x) if f(x) (x 2)
4
.
(19) Find f(x) if f(x)
冢
1
2
x ln x
冣
3
.
(20) Find
∂
∂
x
(x
2
3xy y
2
).
(21) Find
∂
∂
x
冢
1
e
α
e
α
β
x
βx
γ
w
γw
λ
z
λz
冣
.
V. MATRIX ALGEBRA
Matrix algebra is a compact and powerful form of algebra that makes possible com-
putations that would be extremely cumbersome to perform in ordinary, called scalar,
algebra. In this tutorial, I define the major elements and tools of matrix algebra and
demonstrate various applications of its use in statistics. Many students find matrix
notation and matrix manipulations to be especially arcane. In this tutorial I will try,
as much as possible, to demystify these topics.
A. Why Matrix Algebra?
Sometimes you just can’t “get there from here” very well without matrix algebra.
Let’s consider an example. Suppose that we wish to find the expected value and vari-
ance of the ordinary least squares (OLS) estimator of β
k
in the linear regression
model. That is, we need to find E(b
k
) and V(b
k
), applying rules for finding the expec-
tation and variance of sample statistics. With just one regressor in the model, this
isn’t too daunting in scalar algebra. The formula for b is just
b
冱
(
冱
X
(X
X
苶
)(Y
X
苶
)
2
Y
苶
)
.
However, with two regressors in the model, the formulas for b
1
and b
2
are
,
.
The reader can well imagine how cumbersome these expressions get with even more
regressors in the model. In contrast, the formula in matrix notation for the vector, or
冱
(X
1
X
苶
1
)
2
冱
(X
2
X
苶
2
)(Y Y
苶
)
冱
(X
1
X
苶
1
)(X
2
X
苶
2
)
冱
(X
1
X
苶
1
)(YY
苶
)
b
2
冱
(X
1
X
苶
1
)
2
冱
(X
2
X
苶
2
)
2
冤
冱
(X
1
X
苶
1
)(X
2
X
苶
2
)
冥
2
冱
(X
2
X
苶
2
)
2
冱
(X
1
X
苶
1
)(Y Y
苶
)
冱
(X
1
X
苶
1
)(X
2
X
苶
2
)
冱
(X
2
X
苶
2
)(YY
苶
)
b
1
冱
(X
1
X
苶
1
)
2
冱
(X
2
X
苶
2
)
2
冤
冱
(X
1
X
苶
1
)(X
2
X
苶
2
)
冥
2
MATHEMATICS TUTORIALS 473
bapp01.qxd 30.8.04 11:46 Page 473
set, of regression estimates, b, for any number of regressors in the model, is
b (XX)
1
Xy, which is considerably more parsimonious. Using this expression,
finding the expectation and variance of b is quite straightforward, as is demonstrated
in Section V.J below.
B. Notation and Definitions
Notation. The following notational conventions will be used throughout this sec-
tion: Capital letters (e.g., A, B, X, Z) denote matrices. Lowercase letters in boldface
(e.g., a, b, x, z) represent vectors. Lowercase letters at the end of the alphabet (e.g.,
u, v, w, x, y, z) represent variables. Lowercase letters at the beginning of the alpha-
bet (e.g., a, b, c, d) represent constants.
Definitions. A vector is a collection of values, or scalars, arranged in a column. For
example,
x
冤冥
is a vector consisting of the three scalars (numbers) 1, 2, and 3. A vector can also be
written as a row of numbers. When it is, it is referred to as a row vector and given a
transpose (x) superscript. In fact, the transpose of a column vector is the same vec-
tor written as a row. Thus,
x[].
In contrast to some works, this book will always use a transpose sign to indicate a
row vector.
A matrix is simply a collection of column (row) vectors:
X
冤冥
.
Matrices are distinguished according to the number of rows and columns in the
matrix. These constitute the dimensions, or order, of a matrix and are often shown
as subscripts to the matrix symbol. In that the matrix, X, above, has three rows and
three columns, it would be denoted X
33
. In general, X
rc
is a matrix with r rows and
c columns. Individual elements inside a matrix are denoted, in the abstract, with i
and j subscripts to indicate their row and column location within the matrix. Thus,
x
ij
is the element in the ith row and jth column of the matrix X. For example, in the
matrix immediately above, x
32
6.
A vector is a matrix with either r or c equal to 1. Vectors can also be distinguished
by dimensional subscripts. For example,
x
31
冤冥
,
1
2
3
7
8
9
4
5
6
1
2
3
123
1
2
3
474 MATHEMATICS TUTORIALS
bapp01.qxd 30.8.04 11:46 Page 474