Davidson K.R., Donsig A.P. Real Analysis with Real Applications

Подождите немного. Документ загружается.

7.4 Orthonormal Sets 193

NOTE: This inner product and L

norm have the factor

2π

unlike the original def-

inition of hf, gi and kfk

for f ∈ C[a, b] given in the last two sections. We will

always use these normalizations for C[−π, π] and never for general spaces C[a, b].

7.4.5. LEMMA. The functions {1,

√

2cosnθ,

√

2sinnθ : n ≥ 1} form an or-

thonormal set in C[−π, π] with this inner product.

PROOF. Starting with the cosines for n ≥ m ≥ 1, we have

√

2cosnθ,

√

2cosmθi =

−π

cosnθ cosmθ dt

2π

−π

cos(m + n)θ + cos(m − n)θ dt,

where we’ve used the identity 2cosA cosB = cos(A + B) + cos(A − B). If

n > m, then both m + n and m − n are not zero and the integral is

√

2cosnθ,

√

2cosmθi =

2π

sin(m + n)θ

m + n

sin(m − n)θ

m − n

−π

= 0.

If n = m, then

√

2cosnθ,

√

2cosnθi =

2π

sin2nθ

+ θ

−π

= 1.

The other cases are very similar and are left to the reader. ¥

A trigonometric polynomial is a ﬁnite sum

f(θ) = A

k=1

coskθ + B

sinkθ.

In this case, the coefﬁcients may be recovered using these orthogonality relations.

Indeed,

hf, 1i = A

h1, 1i +

k=1

hcoskθ, 1i + B

hsinkθ, 1i = A

hf, cosnθi = A

h1, cosnθi +

k=1

hcoskθ, cosnθi + B

hsinkθ, cosnθi

= A

hcosnθ, cosnθi =

and

hf, sinnθi = A

h1, sinnθi +

k=1

hcoskθ, sinnθi + B

hsinkθ, sinnθi

= B

hsinnθ, sinnθi =

194 Normed Vector Spaces

We make the following deﬁnition.

7.4.6. DEFINITION. Denote the Fourier series of f ∈ C[−π, π] by

f ∼ A

∞

n=1

cosnθ + B

sinnθ,

where A

2π

−π

f(t) dt, and, for n ≥ 1,

−π

f(t) cosnt dt and B

−π

f(t) sinnt dt.

The sequences (A

)

n≥0

and (B

)

n≥1

are the Fourier coefﬁcients of f .

If f is an arbitrary continuous function, it is not immediately evident that the

Fourier series of f will equal f . This is a serious problem, which required a lot of

work and helped force mathematicians to adopt the careful deﬁnitions and concern

with proofs that drive this book. The answer must involve convergence of a series

of functions. The general ideas that we need are developed in Chapter 8.

7.4.7. LEMMA. Let {e

, . . . , e

} be an orthonormal set in an inner product

space V . If M is the subspace spanned by {e

, . . . , e

}, then every vector x ∈ M

can be written uniquely as

i=1

, where α

= hx, e

i. In other words, the set

, . . . , e

} is linearly independent.

Moreover, for each y in V with hy, e

i = β

and each x =

j=1

in M,

hx, yi =

i=1

In particular, kxk

j=1

PROOF. If we can write a vector x ∈ M as

i=1

and as

i=1

, then

hx, e

i =

j=1

, e

j=1

, e

i = α

and

hx, e

i =

j=1

, e

j=1

, e

i = β

7.4 Orthonormal Sets 195

Thus, α

= β

for all i. In other words, the coefﬁcients of x are uniquely de-

termined. In particular, if x = 0, then α

= 0 for 1 ≤ i ≤ n; whence the set

, . . . , e

} is linearly independent.

Let y be a vector in V with hy, e

i = β

and let x =

j=1

be a vector in M.

Compute

hx, yi =

j=1

, y

j=1

, yi =

j=1

Taking y = x gives kxk

j=1

. ¥

This lemma sufﬁces to understand ﬁnite-dimensional inner product spaces. In

particular, we see that every inner product of ﬁnite dimension n space behaves

exactly as R

with the dot product, once we coordinatize it using an orthonormal

basis.

7.4.8. COROLLARY. If V is an inner product space of ﬁnite dimension n, then

it has an orthonormal basis {e

: 1 ≤ i ≤ n} and the inner product is given by

i=1

j=1

i=1

and the norm is

i=1

1/2

PROOF. By deﬁnition of dimension, V has a basis consisting of n linearly inde-

pendent vectors. Apply the Gram–Schmidt process (7.4.3) to this basis to obtain

an orthonormal basis spanning V (Exercise 7.4.B). Now Lemma 7.4.7 provides the

formulas for inner product and norm. ¥

Exercises for Section 7.4

A. Show that every inner product space satisﬁes the parallelogram law:

kx + yk

+ kx − yk

= 2kxk

+ 2kyk

for all x, y ∈ V.

B. The Gram–Schmidt process. We use the notation of (7.4.3).

(a) Show by induction that span{y

, . . . , y

} = span{x

, . . . , x

} for 1 ≤ k ≤ n.

(b) Show that y

k+1

is orthogonal each y

for 1 ≤ i ≤ k.

k+1

= 0 if and only if it belongs to span{y

, . . . , y

(d) Hence conclude that the Gram–Schmidt process produces an orthonormal basis for

span{x

, . . . , x

C. Show that every ﬁnite-dimensional inner product space has an orthonormal basis.

196 Normed Vector Spaces

D. Show that if {x

: k ≥ 1} is a countable set of vectors in an inner product space, then

the Gram–Schmidt process still produces an orthonormal set {f

: i ∈ S} for S ⊂ N

with properties (1) and (2) of 7.4.3.

E. Find an orthonormal basis for the n × n matrices using the inner product of Exer-

cise 7.3.I.

F. Complete the proof of Lemma 7.4.5.

G. Show that f

= sin(nπx) for n ≥ 1 forms an orthonormal set in C[0, 1] with respect

to the L

[0, 1] norm.

H. (a) Find the Fourier series for cos

(θ).

(b) Use trig identities to verify that cos

(θ) can be expressed as the trig polynomial

you found in (a).

I. If f ∈ C[−π, π] has the Fourier series f ∼ A

∞

n=1

cosnθ + B

sinnθ, show

that A

∞

n=1

+ |B

≤

2π

−π

|f(x)|

dx.

HINT: Consider the ﬁnite sums.

NOTE: We will show in Example 13.6.5 that this is an equality.

J. Let f(x) = x for −π ≤ x ≤ π. Compute the inner product hf, sinnxi for n ≥ 1.

Hence show that

∞

n=1

≤

HINT: Integrate by parts. (See Example 13.6.5.)

7.5. Orthogonal Expansions in Inner Product Spaces

Given our success in the last section in understanding ﬁnite-dimensional inner

product spaces by using orthonormal sets, it is natural to look at orthonormal sets

in other inner product spaces. The ﬁrst major theorem of this section, the Projec-

tion Theorem, deals with ﬁnite orthonormal sets in general inner product spaces.

The rest of the section is devoted to extending the Projection Theorem to inﬁnite

orthonormal sets.

7.5.1. DEFINITION. A projection is a linear map P such that P

= P . In

addition, say that P is an orthogonal projection if kerP = {v ∈ V : P v = 0} is

orthogonal to RanP = P V .

Observe that kerP = Ran(I −P ) because P x = 0 if and only if (I −P )x = x.

This shows that kerP ⊂ Ran(I − P ). Conversely, x = (I − P )y implies that

P x = (P −P

)y = 0. So when P is an orthogonal projection, the vectors P x and

(I − P )x are orthogonal. Therefore, we obtain the Pythagorean identity

kxk

= kP xk

+ k(I − P )xk

7.5 Orthogonal Expansions in Inner Product Spaces 197

7.5.2. PROJECTION THEOREM.

Let {e

, . . . , e

} be an orthonormal set in an inner product space V and let M be

the subspace spanned by {e

, . . . , e

}. Deﬁne P : V → M by P y =

j=1

hy, e

for each y ∈ V . Then P is the orthogonal projection onto M and

(7.5.3) kyk

≥

j=1

hy, e

Moreover, for all v ∈ M ,

(7.5.4) ky − vk

= ky − P yk

+ kP y − vk

In particular, P y is the closest vector in M to y.

Figure 7.2 illustrates the theorem when M is a plane in R

P y

FIGURE 7.2. The projection of a point.

PROOF. We leave it to the reader to verify that P is linear by using the bilinearity of

the inner product. It is evident by its deﬁnition that P maps V into M. Suppose that

a typical vector in M is expressed as x =

j=1

. By Lemma 7.4.7, hx, e

i = α

P x =

j=1

hx, e

j=1

= x.

For any y ∈ V , we have P y ∈ M and so P

y = P (P y) = P y. Therefore,

P is a projection of V onto M. As P y belongs to M, Lemma 7.4.7 shows that

kP yk

j=1

, where β

= hy, e

198 Normed Vector Spaces

Now with x ∈ M and y ∈ V as in the previous lemma, we compute

kx − yk

= hx − y, x − yi = hx, xi − 2hx, yi + hy, yi

j=1

− 2

j=1

+ kyk

j=1

− 2

j=1

−

j=1

+ kyk

j=1

(α

− β

)

− kP yk

+ kyk

= kx − P yk

− kP yk

+ kyk

In the third step, we added and subtracted the term

j=1

in order to recognize the

rest as a sum of squares. If x = P y, then β

= α

and so the preceding equation

becomes

kP y − yk

= −kP yk

+ kyk

Substituting this into the preceding equation gives (7.5.4). To verify (7.5.3), ob-

serve kyk

≥ kP yk

j=1

hy, e

To see that P is an orthogonal projection, suppose that x = P x =

j=1

belongs to the range M and y belongs to kerP , so that P y = 0. Then from the

deﬁnition of P , we see that β

= hy, e

i = 0 for 1 ≤ i ≤ n. Thus

hx, yi =

j=1

= 0.

So kerP is orthogonal to RanP . ¥

How much of the Projection Theorem extends to inﬁnite-dimensional spaces

such as C[a, b]? We can generalize the inequality in (7.5.3) to any countable or-

thonormal set in an inner product space.

7.5.5. BESSEL’S INEQUALITY.

Let S ⊆ N and let {e

: n ∈ S} be an orthonormal set in an inner product space

V . For x ∈ V ,

n∈S

hx, e

≤ kxk

PROOF. Let us write α

= hx, e

i. If S is a ﬁnite set, then the Projection Theorem

applies. In particular, if P x denotes the projection onto the span of the e

, then

n∈S

hx, e

= kP xk

≤ kxk

7.5 Orthogonal Expansions in Inner Product Spaces 199

So suppose that S = N is inﬁnite. Using limits and the preceding argument for the

ﬁnite set {e

: 1 ≤ n ≤ N} gives

∞

n=1

hx, e

= lim

N→∞

n=1

hx, e

≤ lim

N→∞

kxk

= kxk

To extend other parts of the Projection Theorem, we need to deal with inﬁnite

series of vectors and their convergence. This can be a delicate issue, however, the

problem has an accessible solution if the inner product space is complete.

7.5.6. DEFINITION. A complete inner product space is called a Hilbert space.

We give a Hilbert space in the next example, but it is important to know that not

all “natural” inner product spaces are complete. For example, the space C[−π, π]

examined in the previous section is not complete (the argument is outlined in Ex-

ercise 7.5.H). An abstract way to complete C[−π, π] in the L

norm to obtain the

Hilbert space L

(−π, π) is discussed in Section 9.6. Alternatively, L

(−π, π) may

be constructed by developing a more powerful theory of integration, known as the

Lebesgue integral, which is a central topic in a course on measure theory.

7.5.7. EXAMPLE. The space `

consists of all sequences x = (x

)

∞

n=1

such

that kxk

∞

n=1

1/2

is ﬁnite. The inner product on `

is given by

hx, yi =

∞

n=1

In order for this inner product to be well deﬁned, we need to know that this series

always converges. In fact, it always converges absolutely (see Exercise 7.5.B).

7.5.8. THEOREM. The space `

is complete.

PROOF. We must show that if a sequence x

= (x

k,n

)

∞

n=1

is Cauchy, then it con-

verges to a vector x in `

. We know that for every ε > 0, there is a number K so

large that kx

− x

k < ε for all k, l ≥ K. In particular,

k,n

− x

l,n

| ≤ kx

− x

k < ε for all k, l ≥ K.

So for each coordinate n, the sequence (x

k,n

)

∞

k=1

is a Cauchy sequence of real

numbers. By the completeness of the real numbers (Theorem 2.7.4), there is a real

number y

, so that

= lim

k→∞

k,n

exists for each n ≥ 1.

Let y = (y

)

∞

n=1

. We need to show two things: ﬁrst, that y is in `

and, second,

that x

converges in `

to y.

200 Normed Vector Spaces

It also follows from the triangle inequality that

k − kx

≤ kx

− x

k < ε for all k, l ≥ K.

Hence the sequence (kx

∞

k=1

is Cauchy. Let L = lim

k→∞

Fix an integer N. Then compute

n=1

= lim

k→∞

n=1

k,n

≤ lim

k→∞

= L

Now take a limit as N tends to inﬁnity to obtain

kyk

= lim

N→∞

n=1

≤ L

This shows that y belongs to `

A similar argument shows that x

converges to y. Indeed, ﬁx ε > 0 and choose

K as before using the Cauchy criterion. Then ﬁx N and compute

n=1

− x

k,n

= lim

l→∞

n=1

l,n

− x

k,n

≤ lim

l→∞

− x

≤ ε

Now that the right hand side is independent of N, we may let N tend to inﬁnity to

obtain

ky − x

= lim

N→∞

n=1

− x

k,n

≤ ε

for all k ≥ K. Since ε > 0 is arbitrary, this establishes convergence. ¥

In a Hilbert space, the closed span of a set of vectors S, denoted

spanS, is the

closure of the linear subspace spanned by S. This is still a subspace. Since it is a

closed subset of a complete space, it is also complete. Thus closed subspaces of

Hilbert spaces are themselves Hilbert spaces in the given inner product.

We can now give a sharpening of Bessel’s inequality. This theorem tells us

precisely when Bessel’s inequality is an equality. Solely to avoid the technicalities

of uncountable bases (see Appendix 2.8), we assume that our Hilbert spaces are

separable, meaning that every orthonormal set is countable. In other words, we

assume that an orthonormal set can be indexed either by a ﬁnite set or by N.

7.5.9. PARSEVAL’S THEOREM.

Let S ⊂ N and E = {e

: n ∈ S} be an orthonormal set in a Hilbert space H.

Then the subspace M = spanE consists of all vectors x =

n∈S

, where the

coefﬁcient sequence (α

)

∞

n=1

belongs to `

. Further, if x is a vector in H, then x

belongs to M if and only if

n∈S

hx, e

= kxk

7.5 Orthogonal Expansions in Inner Product Spaces 201

PROOF. When S is a ﬁnite set, this theorem follows from the Projection Theorem

(Theorem 7.5.2). So suppose that S = N.

Suppose that (α

)

∞

n=1

∈ `

. Deﬁne x

n=1

. We will show that this

is a Cauchy sequence. Indeed, if ε > 0, then the convergence of

n≥1

|α

shows

that there is an integer K so that

∞

n=K+1

|α

< ε

. Thus if l ≥ k ≥ K,

− x

n=k+1

|α

< ε

As H is complete, this sequence converges to a vector x.

Since M is closed and each x

lies in M, it follows that x belongs to M.

Moreover, using Corollary 7.3.7,

hx, e

i = lim

k→∞

, e

i = α

for all n ≥ 1.

So we can write without confusion x =

∞

n=1

. Thus M contains all of the `

linear combinations of the basis vectors.

Now let x be an arbitrary vector in H and set α

= hx, e

i. By Bessel’s

inequality (7.5.5), the sequence (α

)

∞

n=1

belongs to `

and

n≥1

|α

≤ kxk

. Let

y =

∞

n=1

. Compute

kx − yk

= kxk

− 2hx, yi + kyk

= kxk

− 2

∞

n=1

hx, α

i +

∞

n=1

|α

= kxk

− 2

∞

n=1

|α

∞

n=1

|α

= kxk

−

∞

n=1

|α

So we see that if Bessel’s inequality is an equality, then x = y and thus it belongs

to M.

Conversely, if x belongs to M, we must show that the series

∞

n=1

actually

converges to x itself. Since x belongs to M, it is the limit of vectors in the algebraic

span of the basis vectors. So given any ε > 0, there is an integer N and a vector z

in span{e

: 1 ≤ n ≤ N } such that kx − zk < ε. By the Projection Theorem, the

vector x

n=1

is closer to x:

kx − x

k ≤ kx − zk < ε.

202 Normed Vector Spaces

Since this holds for all ε > 0, we deduce that a subsequence of the sequence

)

∞

n=1

converges to x. But this whole sequence converges(as shown in the second

paragraph), so that x =

∞

n=1

. ¥

7.5.10. COROLLARY. Let E = {e

: n ∈ S} be an orthonormal set in a

Hilbert space H. Then there is a continuous linear orthogonal projection P

of H

onto M =

spanE given by P

x =

n∈S

hx, e

PROOF. The preceding proof established that y = P

x =

n∈S

hx, e

is de-

ﬁned and that

kxk

= kyk

+ kx − yk

= kP

+ kx − P

Since the coefﬁcients are determined in a linear fashion,

hαx + βy, e

i = αhx, e

i + βhy, e

it follows that P

(αx+βy) = αP

x+βP

y for all x, y ∈ H and scalars α, β ∈ R.

So P

is linear.

Also since kP

xk ≤ kxk, we obtain that

x − P

yk = kP

(x − y)k ≤ kx − yk.

So P

is Lipschitz and thus (uniformly) continuous. Parseval’s Theorem also es-

tablished that P

x = x if and only if x ∈ M. In particular, the range of P

precisely M.

Finally, P

y = 0 if and only if hy, e

i = 0 for all n ∈ S. Thus if x = P

x =

n∈S

and P

y = 0, it follows that

hx, yi =

n∈S

, yi = 0.

Hence P

is an orthogonal projection. ¥

Recall that an orthonormal basis for a Hilbert space was deﬁned as a maximal

orthonormal set. Every Hilbert space has an orthonormal basis, but a proof of this

fact requires assumptions from set theory, including the Axiom of Choice.

7.5.11. COROLLARY. If E = {e

: i ≥ 1} is an orthonormal basis for a

Hilbert H, every vector x ∈ H may be uniquely expressed as x =

∞

i=1

, where

= hx, e

PROOF. We need to show that the closed span of a basis is H. If M =

span{E} is

a proper subspace, there is a vector x ∈ H that is not in M. Let y = x − P

x and

e = y/kyk. By the previous corollary, y 6= 0. So e is a unit vector such that

e =

(x − P

kyk

= 0.