Stone M., Goldbart P. Mathematics for Physics: A Guided Tour for Graduate Students

Подождите немного. Документ загружается.

Linear algebra review 755

{0} is the subspace containing only the zero vector. In this case U +V can be identified

with U ⊕ V .

If U is a subspace of W then we can seek a complementary space V such that W =

U ⊕ V , or, equivalently, W = U + V with U ∩ V ={0}. Such complementary spaces

are not unique. Consider R

, for example, with U being the vectors in the xy-plane. If e

is any vector that does not lie in this plane then the one-dimensional space spanned by

e is a complementary space for U .

A.4.2 Quotient spaces

We have seen that if U is a subspace of W there are many complementary subspaces V

such that W = U ⊕ V . We can, however, define a unique space that we might denote

by W −U and refer to as the difference of the two spaces. It is more common, however,

to see this space written as W /U and referred to as the quotient of W modulo U . This

quotient space is the vector space of equivalence classes of vectors, where we do not

distinguish between two vectors in W if their difference lies in U. In other words

x = y (mod U) ⇔ x − y ∈ U . (A.48)

The collection of elements in W that are equivalent to x (mod U ) composes a coset,

written x +U , a set whose elements are x +u where u is any vector in U. These cosets

are the elements of W /U .

When we have a linear map A : U → V , the quotient space V /Im A is

often called

the co-kernel of A.

Given a positive-definite inner product, we can define a unique orthogonal comple-

ment of U ⊂ W . We define U

⊥

to be the set

⊥

={x ∈ W : x, y=0, ∀y ∈ U }. (A.49)

It is easy to see that this is a linear subspace and that U ⊕U

⊥

= W . For finite-dimensional

spaces

dim W /U = dim U

⊥

= dim W − dim U

and (U

⊥

)

⊥

= U . For infinite-dimensional spaces we only have (U

⊥

)

⊥

⊇ U . (Be

careful, however. If the inner product is not positive definite, U and U

⊥

may have

non-zero vectors in common.)

Although they have the same dimensions, do not confuse W /U with U

⊥

, and in

particular do not use the phrase orthogonal complement without specifying an inner

product.

A practical example of a quotient space occurs in digital imaging. A colour camera

reduces the infinite-dimensional space L of coloured light incident on each pixel to

three numbers, R, G and B, these obtained by pairing the spectral intensity with the

756 Appendix A

frequency response (an element of L

∗

) of the red, green and blue detectors at that point.

The space of distinguishable colours is therefore only three dimensional. Many different

incident spectra will give the same output RGB signal, and are therefore equivalent as

far as the camera is concerned. In the colour industry these equivalent colours are called

metamers. Equivalent colours differ by spectral intensities that lie in the space B of

metameric black. There is no inner product here, so it is meaningless to think of the

space of distinguishable colours as being B

⊥

. It is, however, precisely what we mean

by L/B.

A.4.3 Projection-operator decompositions

An operator P : V → V that obeys P

= P is called a projection operator. It projects

a vector x ∈ V to Px ∈ Im P along Ker P – in the sense of casting a shadow onto Im P

with the light coming from the direction Ker P. In other words all vectors lying in Ker P

are killed, whilst any vector already in Im P is left alone by P. (If x ∈ Im P then x = Py

for some y ∈ V , and Px = P

y = Py = x.) The only vector common to both Ker P and

Im P is 0, and so

V = Ker P ⊕ Im P. (A.50)

A set of projection operators P

that are “orthogonal”

= δ

, (A.51)

and sum to the identity operator



= I, (A.52)

is called a resolution of the identity. The resulting equation

x =



x (A.53)

decomposes x uniquely into a sum of terms P

x ∈ Im P

and so decomposes V into a

direct sum of subspaces V

≡ Im P

V =

. (A.54)

Exercise A.8: Let P

be a projection operator. Show that P

= I −P

is also a projection

operator and P

= 0. Show also that Im P

= Ker P

and Ker P

= Im P

Linear algebra review 757

A.5 Inhomogeneous linear equations

Suppose we wish to solve the system of linear equations

+ a

+···+a

= b

+ a

+···+a

= b

+ a

+···+a

= b

or, in matrix notation,

Ay = b, (A.55)

where A is the m-by-n matrix with entries a

. Faced with such a problem, we should

start by asking ourselves the questions:

(i) Does a solution exist?

(ii) If a solution does exist, is it unique?

These issues are best addressed by considering the matrix A as a linear operator

A : V → W , where V is n-dimensional and W is m-dimensional. The natural language

is then that of the range and null-spaces of A. There is no solution to the equation Ay = b

when Im A is not the whole of W and b does not lie in Im A. Similarly, the solution will

not be unique if there are distinct vectors x

, x

such that Ax

= Ax

. This means that

A(x

− x

) = 0,or(x

− x

) ∈ Ker A. These situations are linked, as we have seen, by

the range–null-space theorem:

dim (Ker A) + dim (Im A) = dim V . (A.56)

Thus, if m > n there are bound to be some vectors b for which no solution exists. When

m < n the solution cannot be unique.

A.5.1 Rank and index

Suppose V ≡ W (so m = n and the matrix is square) and we choose an inner product,

x, y,onV . Then x ∈ Ker A implies that, for all y

0 =y, Ax=A

†

y, x, (A.57)

or that x is perpendicular to the range of A

†

. Conversely, let x be perpendicular to the

range of A

†

. Then

x, A

†

y=0, ∀y ∈ V , (A.58)

758 Appendix A

which means that

Ax, y=0, ∀y ∈ V , (A.59)

and, by the non-degeneracy of the inner product, this means that Ax = 0. The net result

is that

Ker A = (Im A

†

)

⊥

. (A.60)

Similarly

Ker A

†

= (Im A)

⊥

. (A.61)

Now

dim (Ker A) + dim (Im A) = dim V ,

dim (Ker A

†

) + dim (Im A

†

) = dim V , (A.62)

but

dim (Ker A) = dim (Im A

†

)

⊥

= dim V − dim (Im A

†

)

= dim (Ker A

†

Thus, for finite-dimensional square matrices, we have

dim (Ker A) = dim (Ker A

†

In particular, the row and column rank of a square matrix coincide.

Example: Consider the matrix

A =

⎛

⎝

123

111

234

⎞

⎠

Clearly, the number of linearly independent rows is two, since the third row is the sum

of the other two. The number of linearly independent columns is also two – although

less obviously so – because

−

⎛

⎝

⎞

⎠

+ 2

⎛

⎝

⎞

⎠

⎛

⎝

⎞

⎠

Linear algebra review 759

Warning: The equality dim (Ker A) = dim (Ker A

†

) need not hold in infinite-

dimensional spaces. Consider the space with basis e

, e

, ...indexed by the positive

integers. Define Ae

= e

, Ae

= e

, and so on. This operator has dim (Ker A) = 0. The

adjoint with respect to the natural inner product has A

†

= 0, A

†

= e

, A

†

= e

Thus Ker A

†

={e

}, and dim (Ker A

†

) = 1. The difference dim (Ker A) −dim (Ker A

†

)

is called the index of the operator. The index of an operator is often related to topological

properties of the space on which it acts, and in this way appears in physics as the origin

of anomalies in quantum field theory.

A.5.2 Fredholm alternative

The results of the previous section can be summarized as saying that the Fredholm

alternative holds for finite square matrices. The Fredholm Alternative is the set of

statements

I. Either

(i) Ax = b has a unique solution,

(ii) Ax = 0 has a solution.

II. If Ax = 0 has n linearly independent solutions, then so does A

†

x = 0.

III. If alternative (ii) holds, then Ax = b has no solution unless b is orthogonal to all

solutions of A

†

x = 0.

It should be obvious that this is a recasting of the statements that

dim (Ker A) = dim (Ker A

†

and

(Ker A

†

)

⊥

= Im A. (A.63)

Notice that finite-dimensionality is essential here. Neither of these statements is

guaranteed to be true in infinite-dimensional spaces.

A.6 Determinants

A.6.1 Skew-symmetric n-linear forms

You will be familiar with the elementary definition of the determinant of an n-by-n matrix

A having entries a

det A ≡

... a

def

= 

...i

...a

. (A.64)

760 Appendix A

Here, 

...i

is the Levi-Civita symbol, which is skew-symmetric in all its indices and



12...n

= 1. From this definition we see that the determinant changes sign if any pair of

its rows are interchanged, and that it is linear in each row. In other words

λa

+ µb

λa

+ µb

... λa

+ µb

... c

= λ

... a

... c

+ µ

... b

... c

If we consider each row as being the components of a vector in an n-dimensional vector

space V , we may regard the determinant as being a skew-symmetric n-linear form,

i.e. a map

ω :

n factors

C DA B

V × V × ...V → F (A.65)

which is linear in each slot,

ω(λa + µb, c

, ..., c

) = λω(a, c

, ..., c

) + µω(b, c

, ..., c

), (A.66)

and changes sign when any two arguments are interchanged,

ω(..., a

, ..., a

, ...)=−ω(..., a

, ..., a

, ...). (A.67)

We will denote the space of skew-symmetric n-linear forms on V by the symbol

∗

). Let ω be an arbitrary skew-symmetric n-linear form in

∗

), and let

, e

, ..., e

} be a basis for V .Ifa

= a

(i = 1, ..., n) is a collection of n vectors,

we compute

ω(a

, a

, ..., a

) = a

...a

ω(e

, e

, ..., e

)

= a

...a



...,i

ω(e

, e

, ..., e

). (A.68)

In the first line we have exploited the linearity of ω in each slot, and in going from the

first to the second line we have used skew-symmetry to rearrange the basis vectors in

The index j on a

should really be a superscript since a

is the j-th contravariant component of the vector

. We are writing it as a subscript only for compatibility with other equations in this section.

Linear algebra review 761

their canonical order. We deduce that all skew-symmetric n-forms are proportional to

the determinant

ω(a

, a

, ..., a

) ∝

... a

and that the proportionality factor is the number ω(e

, e

, ..., e

). When the number of

its slots is equal to the dimension of the vector space, there is therefore essentially only

one skew-symmetric multilinear form and

∗

) is a one-dimensional vector space.

Now we use the notion of skew-symmetric n-linear forms to give a powerful definition

of the determinant of an endomorphism, i.e. a linear map A : V → V . Let ω be a non-zero

skew-symmetric n-linear form. The object

, x

, ..., x

)

def

= ω(Ax

, Ax

, ..., Ax

) (A.69)

is also a skew-symmetric n-linear form. Since there is only one such object up to

multiplicative constants, we must have

ω(Ax

, Ax

, ..., Ax

) ∝ ω(x

, x

, ..., x

). (A.70)

We define “det A” to be the constant of proportionality. Thus

ω(Ax

, Ax

, ..., Ax

) = det (A)ω(x

, x

, ..., x

). (A.71)

By writing this out in a basis where the linear map A is represented by the matrix A,we

easily see that

det A = det A. (A.72)

The new definition is therefore compatible with the old one. The advantage of this more

sophisticated definition is that it makes no appeal to a basis, and so shows that the

determinant of an endomorphism is a basis-independent concept. A byproduct is an easy

proof that det (AB) = det (A)det (B), a result that is not so easy to establish with the

elementary definition. We write

det (AB)ω(x

, x

, ..., x

) = ω(ABx

, ABx

, ..., ABx

)

= ω(A(Bx

), A(Bx

), ..., A(Bx

))

= det (A)ω(Bx

, Bx

, ..., Bx

)

= det (A)det (B)ω(x

, x

, ..., x

). (A.73)

Cancelling the common factor of ω(x

, x

, ..., x

) completes the proof.

762 Appendix A

Exercise A.9: Let ω be a skew-symmetric n-linear form on an n-dimensional vector

space. Assuming that ω does not vanish identically, show that a set of n vec-

tors x

, x

, ..., x

is linearly independent, and hence forms a basis, if, and only if,

ω(x

, x

, ..., x

) = 0.

Exercise A.10: Extend the pairing between V and its dual space V

∗

to a pairing between

the one-dimensional

∗

) and its dual space. Use this pairing, together with the result

of Exercise A.5, to show that

det A

= det A

∗

=[det A]

∗

=[det A]

= det A = det A,

where the “∗” denotes the conjugate operator (and not complex conjugation) and the

penultimate equality holds because transposition has no effect on a one-by-one matrix.

Conclude that det A = det A

. A determinant is therefore unaffected by the interchange

of its rows with its columns.

Exercise A.11: Cauchy–Binet formula. Let A be an m-by-n matrix and B be an n-by-m

matrix. The matrix product AB is therefore defined, and is an m-by-m matrix. Let S be

a subset of {1, ..., n} with m elements, and let A

be the m-by-m matrix whose columns

are the columns of A corresponding to indices in S. Similarly let B

be the m-by-m

matrix whose rows are the rows of B with indices in S. Show that

det AB =



det A

det B

where the sum is over all n!/m!(n − m)! subsets S.Ifm > n there are no such subsets.

Show that in this case det AB = 0.

Exercise A.12: Let

A =





be a partitioned matrix where a is m-by-m, b is m-by-n, c is n-by-m and d is n-by-n.By

making a Gaussian decomposition

A =







0 





show that, for invertible d, we have Schur’s determinant formula

det A = det(d) det(a − bd

−1

c).

I. Schur, J. für reine und angewandte Math., 147 (1917) 205.

Linear algebra review 763

A.6.2 The adjugate matrix

Given a square matrix

A =

⎛

⎜

⎝

... a

⎞

⎟

⎠

(A.74)

and an element a

, we define the corresponding minor M

to be the determinant of the

(n−1)-by-(n −1) matrix constructed by deleting from A the row and column containing

. The number

= (−1)

i+j

(A.75)

is then called the cofactor of the element a

. (It is traditional to use uppercase letters to

denote cofactors.) The basic result involving cofactors is that





= δ



det A. (A.76)

When i = i



, this is known as the Laplace development of the determinant about row i.

We get zero when i = i



because we are effectively developing a determinant with two

equal rows. We now define the adjugate matrix,

Adj A, to be the transposed matrix of

the cofactors:

(Adj A)

= A

. (A.77)

In terms of this we have

A(Adj A) = (det A)I. (A.78)

In other words

−1

det A

Adj A. (A.79)

Each entry in the adjugate matrix is a polynomial of degree n − 1 in the entries of the

original matrix. Thus, no division is required to form it, and the adjugate matrix exists

even if the inverse matrix does not.

Some authors rather confusingly call this the adjoint matrix.

764 Appendix A

Exercise A.13: It is possible to Laplace-develop a determinant about a set of rows. For

example, the development of a 4-by-4 determinant about its first two rows is given by:

−

Understand why this formula is correct and, using that insight, describe the general rule.

Exercise A.14: Sylvester’s lemma.

Let A and B be two n-by-n matrices. Show that

det A det B =



det A



det B



where A



and B



are constructed by selecting a fixed set of k < n columns of B (which

we can, without loss of generality, take to be the first k columns) and interchanging them

with k columns of A, preserving the order of the columns. The sum is over all n!/k!(n−k)!

ways of choosing columns of A. (Hint: show that, without loss of generality, we can take

the columns of A to be a set of basis vectors, and that, in this case, the lemma becomes

a restatement of your “general rule” from the previous problem.)

Cayley’s theorem

You will know that the possible eigenvalues of the n-by-n matrix A are given by the

roots of its characteristic equation

0 = det (A − λI) = (−1)



− tr (A)λ

n−1

+···+(−1)

det (A)

, (A.80)

and have probably met with Cayley’s theorem that asserts that every matrix obeys its

own characteristic equation.

− tr (A)A

n−1

+···+(−1)

det (A)I = 0. (A.81)

The proof of Cayley’s theorem involves the adjugate matrix. We write

det (A − λI) = (−1)



+ α

n−1

+···+α

(A.82)

and observe that

det (A − λI)I = (A −λI)Adj (A − λI). (A.83)

J. J. Sylvester, Phil. Mag., 1 (1851) 295.