Appel W. Mathematics for Physics and Physicists

Подождите немного. Документ загружается.

Random vectors 537

Name m ℓ Name m ℓ

Niobe 8.5 5 Frigga 9.6 6

Feronia 10.3 7 Diana 9.1 5

Klytia 10.3 6 Eurynome 9.3 8

Galatea 10.1 7 Sappho 9.3 6

Eurydike 10.0 8 Terpsichore 9.7 11

Freia 9.0 5 Alkmene 9.4 7

The correlation for the ﬁrst eight asteroids is r ≈ 0.785, w hich is quite large.

Worse, the correlation for the last ﬁve is r ≈ 0.932. Of course, this is merely

an accident, due to the small sample used. There is in fact no correlation

between the length of the name and the magnitude of asteroids. With the

twelve given, the correlation becomes ≈ 0.4 and with ﬁfty asteroids

it is only

r ≈ −0.097.

Remark 20.42 The covariance of (X , Y ) may be seen as a kind of “scalar product” of X −E(X )

with Y −E(Y ) and the standard deviation as the associated “norm.” The correlation is then

the cosine of the angle between the two functions in L

space:

r =

(X |Y )

kX k

kY k

So there is no surprise in the fact that this number is between −1 and 1. Moreover, |r| = 1 if

and only if the functions X −E(X ) and Y −E(Y ) are linearly dependent, as we found above.

20.6.b Independen t random variables

DEFINITION 20.43 (Independence) Two random variables X and Y are in-

dependent random variables if the joint distribution function can be ex-

pressed as the direct product (or “tensor” product) of their marginal distribu-

tion functions F

X Y

= F

⊗ F

, that is, if

X Y

(x, y) = F

(x) F

( y) for all x, y ∈ R.

It is usual to abbreviate “independent random variables” by i.r.v.

There is an equivalent form of the deﬁnition which involves the σ-algebra

generated by a random variable:

DEFINITION 20.44 (σ-algebra generated by an r.v.) Let X be a random vari-

able. The σ-algebra generated by X, denoted T (X ), is the set of all subs ets

of the form X

−1

(A) for a Borel subset A.

PROPOSITION 20.45 Two random variables X and Y are independent if a nd only

if the σ-algebras generated by X and Y are independent, that is, if, for all A ∈ T (X )

and B ∈ T (Y ), we have P(A ∩ B) = P(A) ·P(B).

Numbered 51 to 10 0 in the Ephemerids of Minor Planets (Leningrad, 1979).

538 Random variables

If the joint probability density exists, we then have f

X Y

= f

⊗ f

, tha t

is,

X Y

(x, y) = f

(x) f

( y) for all x, y ∈ R.

We have the following important result:

THEOREM 20.46 Two independent r.v. X and Y are uncorrelated.

Proof. We can assume that the variances σ

and σ

exist; otherwise the correlation

is not deﬁned. Then we have

Cov(X , Y ) =

(x −m

)( y −m

) f

X Y

(x, y) dx dy

(x −m

) f

(x) dx

( y −m

) f

( y) dy = 0,

since each factor vanishes.

The converse is not true in general, however.

Example 20.47 Let Ω = [0, 2π] with the uniform probability measure P = dθ/2π. Let

X (θ) = cos θ and Y = si n θ. The r.v. X and Y are not independent (intuitively, because there

is a relation between them, namely X

+ Y

= 1), but on the other hand the covariance is

equal to

Cov(X , Y ) =

Ω

cos θ sin θ

dθ

2π

= 0,

so they are uncorrelated.

20.6.c Random vectors

It is now possible to generalize the preceding results to n random variables.

DEFINITION 20.48 An n-dimensional random vector is an n-tuple of ran-

dom variables, or in other words a measurable map

X = (X

, . . . , X

) : Ω −→ R

DEFINITION 20.49 The distribution function (or joint distribution func-

tion) of a random vector X is the function deﬁned on R

( x) = P



¶ x

for i = 1, . . . , n



, where x = (x

, . . . , x

PROPOSITION 20.50 Let F

be the distribution function of X. Then for all

x ∈ R

we have

(−∞, x

, . . . , x

) = ··· = F

, . . . , −∞, . . . , x

) = ···

= F

, . . . , −∞) = 0

and F

X Y

(+∞, . . ., +∞) = 1.

Morevoer, F

is increasing a nd right-continuous with respect to each variable.

Image measures 539

DEFINITION 20.51 The probability dens ity (or joint probability density) of

a random vector X is the function (or distrib ut ion)

, . . . , x

)

def

∂

∂ x

. . .∂ x

, . . . , x

We then have

, . . . , x

) =

−∞

. . .

−∞

f (s

, . . . , s

) ds

···ds

20.7

Image measures

20.7.a Case of a single random variable

 Exercise 20.5 The probability density of the velocity of an atom of a pe rfect gas is known

(it is a maxwellian

). What is the probability density of the kinetic energy mv

/2 of this atom?

To treat this kind of problem, we must consider the situation where, given

a random variable X , a second random variable Y = ϕ(X ) is deﬁned. This is

called a change of random variable. The task is to determine the distribution

function of Y , and its probability density, knowing those of X .

When the change of random variable is bijective, there is a unique y = ϕ(x)

associated to any x ∈ R, and it is then easy to see that if f denotes the

probability density of X and g denotes that of Y , then

g( y) =

f (x)



′

(x)



x=ϕ

−1

( y)

provided ϕ

′

has no zero.

Proof. Indeed, we have G( y) = P{X ¶ ϕ

−1

( y)} = F ◦ϕ

−1

( y) if ϕ is increasing, and

G( y) = P{X ¾ ϕ

−1

(x)} = 1 − P{X < ϕ

−1

( y)} = 1 − F ◦ϕ

−1

( y) if ϕ is decreasing.

Then

g( y) = G

′

( y) =



F ◦ϕ

−1



= ±F

′

◦ϕ

−1

·(ϕ

−1

)

′

= ±

f (x)

′

(x)

as was to be shown.

If the function ϕ is not a bijection, we have the following more general

result (compare with the change of variable formula for a Dirac distribution,

Exercise 7. 4.d, page 198):

In other words, a gaussian with a physicist’s name.

540 Random variables

THEOREM 20.52 Let ϕ : X 7→ Y be a change of random variable and y ∈ R

such that the equation

y = ϕ(x)

has at most cou nta bly many solutions, and let S

denote the set of these solutions.

Assume, moreover, that ϕ

′

(x) 6= 0 for x ∈ S

. Then the probability density g of Y is

related to the probability density f of X by

g( y) =

x∈S

f (x)



′

(x)



20.7.b Case of a random vector

Theorem 20.52 generalizes to a function ϕ : R

→ R

as follows:

THEOREM 20.53 Let ϕ : X 7→ Y be a change of random vector, and let y ∈ R

be such that the equation y = ϕ( x) has at most countably many solutions x

, k ∈ I.

Assume, moreover, that the differential dϕ( x

) is invertible for all i ∈ I. Then the

probability density g of Y is related to the probability density f ofX by

g( y) =

k∈I

f ( x

)

|det dϕ(x

k∈I

f ( x

)



D(x

, . . . , x

)

D( y

, . . . , y

)



where the last factor is the inverse of the jacobian matrix of the change of variable.

20.8

Expectation and char acteristic function

20.8.a Expectation of a function of random variables

Let X be a random variable and Y = ϕ(X ). The expectation of Y , if it exists,

can be computed using the probability density f

of X by the formula

E(Y ) = E



ϕ(X )



Ω



X (ω)



dP(ω) =

ϕ(x) f

(x) dx.

More generally, if Y = ϕ(X

, . . . , X

) is a random variable deﬁned by

change of variable from a random vector, the expecta tion of Y (if it exists)

may b e computed using the joint probability density f

···X

, . . . , x

) of

, . . . , X

) by the formula

E(Y ) =

ϕ(x

, . . . , x

) f

···X

, . . . , x

) dx

. . .dx

Expectation and characteristic function 541

For instance, if X and Y are random variables , the expectation E(X Y ) of

the product X Y can be computed using the joint probability density f

X Y

E{X Y } =

Ω

X (ω) Y (ω) dP(ω) =

x y f

X Y

(x, y) dx dy.

THEOREM 20.54 The expectation has the following properties:

i) it is linear, that is, we have

E{X

}

whenever the right-hand side is deﬁned;

ii) if X is a lmost surely equal to a constant a, E{X } = a; in particular, we have



E{X }



= E{X };

iii) we have



E{X Y }



¶ E{X

}E{Y

by the Cauchy-Schwarz inequality, with equality if and only if X and Y are

linearly dependent;

iv) if X and Y are independent, then E{X Y } = E{X }E{Y }.

20.8.b Moments, variance

Using the formulae of the previous section, we see t hat the moments and cen-

tered moments of a random variable X are given by expectations of functions

of X , namely,

ˆµ

= E{X

} and µ

= E



X −E{X }



Moreover, the k-th moment (centered or not) exists if and only if the function

x 7→ x

f (x) is Lebes gue-integrable, where f is the probability density of X ,

that is, if



f (x) dx < +∞.

As a special case, the variance is given by

= E



(X −E{X })

} = E{X

}−E{X }

20.8.c C haracteristic function

DEFINITION 20.55 The characteristic function ϕ

of a random variable X

is the function deﬁned for ν ∈ R by

(ν) = E



iνX



542 Ran dom variables

The following result is then immediate.

THEOREM 20.56 The characteristic function ϕ

of a random variable X is the

Fourier transform

of its probability density (in the sense of distributions):

(ν) =

Ω

iνX (ω)

dP(ω) =

∞

−∞

iνx

f (x) dx.

Notice that this quantity is always deﬁ ned, for any random variable X , because

iνX

| = 1 and Ω is of total measure equal to 1 (or equivalently, because the

density probability f , when it exists as a function, is Leb esgue-integrable).

According to the general theory of the Fourier transform, we have there-

fore:

PROPOSITION 20.57 Let X be a random variable and let ϕ

be its characteristic

function. Then

i) ϕ

(0) = 1 and



(ν)



¶ 1 for a ll ν ∈ R;

ii) if the k-th order moment of X exists, ϕ

can be differentiated k times and we

have

= E{X

} = (−i)

(k)

(0);

iii) since X is real-valued, ϕ

is hermitian, i.e., we have ϕ

(−ν) = ϕ

(ν).

Proof. Note that ϕ

(0) =

+∞

−∞

(x) dx = 1 or, to phrase it differently, we have

(0) = E{1} = 1. The last two properties are direct translations of known results of

Fourier transforms.

THEOREM 20.58 The characteristic function of a random variable X determines

uniquely the distribution function of X .

In other words, two random variables have the same character istic function if

and only if they have the sa me distrib ut ion function. (This theorem follows

from an inversion formula which is the analogue of the Fourier inversion

formula and is due to P. Lévy).

Before going to the next section, note also the following important fact:

PROPOSITION 20.59 The characteristic function of the sum of two independent

random variables is the product of the characteristic functions of the summands:

X +Y

(ν) = ϕ

(ν) ·ϕ

(ν).

Be careful with the various conventions for the Fourier transform! It is usual in probability

to omit the factor 2 π which often oc curs in analysis.

Sum and product of random variables 543

Proof. Let f

X Y

be the joint probability density of (X , Y ). Since X and Y are

independent, we have f

X Y

(x, y) = f

(x) f

( y) and hence

X +Y

(ν) = E



iν(X +Y )



iν(x+ y)

X Y

(x, y) dx dy

iν x

(x) dx

iν y

( y) dy = ϕ

(ν) ·ϕ

(ν)

by the expect ation formula for a change of random variables.

20.8.d Ge nerating function

DEFINITION 20.60 The generating function G

of a random var iable X is

the function of a real variable s deﬁned a s the expectation of s

(s) = E(s

)

when it ex ists.

THEOREM 20.61 Let X : Ω → N be an integer-valued discrete r.v., and let p

P(X = n) for n ∈ N. Then G

is deﬁned at least for s ∈ [−1, 1] and is of C

∞

class

on ]−1, 1[. Moreover, we have G

(1) = 1 and



(s)



¶ 1 for a ll s ∈ [−1, 1].

In addition, for s ∈ [−1, 1], we h ave the series expansion

(s) =

∞

n=0

and hence P(X = n) =

(n)

(0)

In particular

E(X ) = G

′

(1) and Var(X ) = G

′′

(1) + G

′

(1) −



′

(1)



20.9

Sum and product of random variables

20.9.a Sum of random variables

Let us assume that every day I eat an amount of chocolate which is between

0 and 100 g, and let us model this situation by a random variable X

for the

amount eaten on the ﬁrst day, another random variable X

for th e amount

eaten on the second day, and so on, each random variable being assumed

to have a uniform distribution on [0, 100]. We would like to known the

distribution of the total amount of chocolate eaten in two days.

544 Ran dom variables

Let us start with a slightly simplistic reasoning, where the amount of

chocolate can only take integral values k between 1 and 100 with probabil-

ity P(X

= k) = 1/100. There is a very small probability that I will eat only

2 g of chocolate, since this requires that I eat 1 g the ﬁrst day and 1 g the sec-

ond, with a total probability equal to 1/10 000 (if we assume that the amount

eaten the ﬁrst day is independent of the amount eaten the second). The prob-

ability that I ea t 5 g is larger, since this may happen if I ea t 1 g and then 4 g,

or 2 g and then 3 g, or 3 g and then 2 g, or (ﬁnally) 4 g a nd then 1 g; th is gives

a total probability equal to 4/10 000.

The reader will now have no trouble showing that the probability that I

eat n grams of chocolate is given by the formula

P(X

+ X

= n) = P(X

= 1) P(X

= n − 1)

+ P(X

= 2) P(X

= n − 2) + ···+ P(X

= n −1) P(X

= 1).

With a ﬁner subdivision of the interval [0, 100], a similar formula would hold,

and pas sing to the continuous limit, we expect to ﬁnd that

total

(x) =

100

f (t) f (x − t) dt = f ∗ f (x),

where f

total

is the density of X

+ X

and f that of X

or X

(since the support

of f is [0, 100]). In other words, the probability dens ity of the sum of two

independent random variables is the convolution product of the probability

densities of the arguments.

This result can be established rigorously in two different manners, using a

change of variable or the characteristic function. The reader may ﬁnd either

of those two proofs more enlightening than th e other.

(Using the characteristic function)

From Proposition 20.59 we know that the characteristic function of the sum of

two independent random variab les is the product of the characteristic functions

of the summands. Using the inverse Fourier transform, we obtain:

THEOREM 20.62 (Sum of r.v.) Let X

and X

be tw o independent random var i-

ables, with probability densities f

and f

, respectively (in the sense of distributions).

Then the probability density of the sum X

+ X

is given by

= f

∗ f

or equivalently

(x) =

+∞

−∞

(t) f

(x − t) dt,

which is only correct if the random variables involved are continuous, the probability

densities then being integrable functions.

Sum and product of random variables 545

If we denote by

the transpose of f

, then the probability density of the difference

− X

is given by

−X

= f

∗

or f

−X

(x) =

+∞

−∞

(t) f

(x + t) dt.

Note that

= 1, as it should be.

(With change of variables)

Let now f

, x

) d enote the joint probability density of t he pair of random

variables (X

, X

), and make the ch ange of variab les Y

= X

and Y

+ X

. The jacobian of this transformation is given by

| J | =



D( y

, y

)

D(x

, x

)



1 0

1 1



= 1,

and the joint probability density of (Y

, Y

) is g( y

, y

) = f

( y

, y

− y

); the

probability density of Y = X

+ X

= Y

is the marginal density, so we get:

THEOREM 20.63 (Sum of r.v.) Let Y = X

+ X

and let f

, x

) denote the

joint probability density of the pa ir (X

, X

). Then the probability density of Y is

given by

g( y) =

+∞

−∞

(t, y − t) dt.

If the random variables X

and X

are independent, we have f

= f

⊗ f

and the

probability density of Y is the convolution g = f

∗ f

g( y) =

+∞

−∞

(t) f

( y − t) dt.

Example 20.64 The sum of two independent gaussian random variables X

and X

, with means

and m

and variance σ

and σ

, respectively is another gaussian variable.

Adpating the result of Exercise 10.2 on page 295, which computes the convolution of two

centered g au ssians, one can show that X

+ X

is gau ssian with mean m

+ m

and variance

+ σ

The following result is also very useful:

THEOREM 20.65 (Bienaym

e identity) Let X

, . . . , X

be independent random

variables, each of which has ﬁnite variance Var(X

) = σ

. Then the variance of the

sum e xists and is the sum of the variances of the X

: we have

Var(X

+ ···+ X

) = Var(X

) + ···+ Var(X

Proof. Without loss of generality, we may assume that each random variable is cen-

tered (otherwise, replace X

by X

−E(X

) without changing its variance).

546 Random variables

Let ϕ

denote the characteristic function of the random variable X

. We have

′

(0) = 0 for k = 1, . . . ,n, and by Theorem 20.59 on page 542 we c an write

+···+X

(ν) = ϕ

(ν) ·ϕ

(ν) ···ϕ

(ν).

On the other hand, the variance of X

is given by Var(X

) = −ϕ

′′

(0). It th erefore suf-

ﬁces to differentiate the above function twice and the n evaluate the resulting expression

at 0. Since the reader will easily check that all ﬁrst derivatives give zero contributions,

there only remains the formula

′′

+···+X

(0) = ϕ

′′

(0) ·ϕ

(0) ···ϕ

(0) + ···+ ϕ

(0) ···ϕ

n−1

(0) ·ϕ

′′

(0).

Since in addition we have ϕ

(0) = 1 for all k, the proof is complete.

20.9.b Pr oduct of random variables

While we are on this topic, we might as well consider the topic of product

of random variables. For this, put Y

= X

and Y

= X

. This change of

variable has jacobian

| J | =



D( y

, y

)

D(x

, x

)



1 0



= |x

and h ence we ob tain:

PROPOSITION 20.66 (Product of r.v.) The probability density of Y = X

given by

g( y) =

+∞

−∞





|t|

dt,

in terms of the joint probability density f

of (X

, X

). If the random variables X

and X

are independent, w e have

g( y) =

+∞

−∞

(t) f





|t|

dt.

Similarly, the reader will have no difﬁculty ch ecking t he following:

PROPOSITION 20.67 (Ratio of r.v.) Th e probability density of Y = X

g( y) =

+∞

−∞

(t, t y) |t| dt

and, if the random variables X

and X

are independent,

g( y) =

+∞

−∞

(t) f

(t y) |t| dt.