Appel W. Mathematics for Physics and Physicists

Подождите немного. Документ загружается.

Expectation and variance 527

Example 20.13 Consider again the weighted die above. The probability density of the r.v. X

representing the result of throwing this die is

f (x) =

δ(x −1) +

δ(x −2) +

δ(x −3) +

δ(x −4) +

δ(x −5) +

δ(x −6).

Remark 20.14 In many cases, a random variable has a distribution function which can be

expressed as the sum of a continuous distribution function and a step f u nct ion (which is itself

a distribution f u nction). The probability density is then the sum of an ordinary function and

at most countably many Dirac distributions.

20.3

Expectation and variance

20.3.a Case of a discrete r.v.

DEFINITION 20.15 Let X be a discrete random var iable. Assume that

X (Ω) = {p

; k ∈ I}∪ D,

where I is a countable set and P{X ∈ D} = 0, and let p

= P(X = x

) for

k ∈ I. The e xpectation of X is the quantity

E(X ) =

k∈I

P(X = x

) =

k∈I

if this series converges absolutely.

Thus, t he expectation is the “expected” average of the random variable. We

believe that in some ways it represents the typical behavior of X . To make this

more precise, we try to characterize how the values of X can be far from the

expectation. For instance, during an exam graded on a 200-point scale, the

class average may be equal to 100 with grades all contained between 90 and

110, or with grades ranging from 0 to 200; although two such distributions

may yield the same expecta tion, th ey are of course very different.

To measure th e deviation from the mean, the ﬁ rst idea might be to consider

the average of the differences from the expected value, that is, to consider



X −E(X )





−E(x)



;

however, t his is always zero, by the deﬁnition of the expectation. Anoth er idea

is therefore needed, and since we want to add all the deviations, irrespective

of sig n, we can try to use





X −E(X )







−E(x)



528 Random variables

However, absolute values are always algebraically delicate to h andle. It is better,

therefore, to get rid of the signs by taking the square of X − E(X ), possibly

taking the square root of the result obta ined by taking the average of this.

DEFINITION 20.16 The variance of t he dis crete r.v. X is the quantity

Var (X ) = σ

(X ) = E





X −E(X )







−E(X )



= E(X

) −



E(X )



which may be either a non-negative real number or +∞. The standard devi-

ation of X is given by σ(X ) =

Var (X ).

Consider the example of giving global grades to students having pass ed

four e xams, say in Mathematics, Physics, Chemistry, and Medieval French

Poetry. All four courses should play a similar role in determining the ranking

of the students, even though they may carry different coefﬁcients. For this

purpose t he graders must ensure that the standard deviation of th eir own

grades is equal to some value ﬁxed beforehand with their colleagues (say, 4). It

is then of no consequence that the average in Mathematics is different from

the average in Medieval French Poetry. On the other hand, if the standard

deviation in Mathematics is equal to 6 while that in Medieval French Poet ry

is 2, then the Medieval French Poetry exam has little effect in distinguishing

the students (they all have more or less the same grade), whereas Mathematics

is crucial, even if — this being the entrance exam to the French Academy —

the coefﬁcient for Mathematics is a factor of 2 smaller.

20.3.b Case of a continuous r.v.

We will take the following deﬁnition a s generalizing the discrete case.

DEFINITION 20.17 Let X be a random variable with probability density f .

The expectation E(X) of the r.v. X is the number

E(X ) =

+∞

−∞

t f (t) dt

when the integral exists. Th e variance Var(X) and the standard deviation

σ(X) are given by

Var (X ) = σ

(X ) =

+∞

−∞



t −E(X )



f (t) dt =

+∞

−∞

f (t) dt −



E(x)



when those integrals exist.

In order to preserve the unit of a p hysi cal quantity, for instance.

Expectation and variance 529

Remark 20.18 The reader may check, as an exercise, that if an r.v. X is discrete, and a

distribution is u sed to represent its probability density, then the above deﬁnition provides the

same deﬁnition of the expectation of X as Deﬁnition 20.15.

Remark 20.19 Putting f (x) =



ψ(x)



, the reader can also recognize in the deﬁnition of

expectation and standard de viation the quantities denoted 〈x〉 and ∆x in quantum mechanics

(see Sect ion 13.3 on page 359).

Remark 20.20 The exp ectation and the variance are special cases of what are called moments of

random variables.

For a continuous random variab le, we write equivalently

x f (x) dx =

Ω

X (ω) dP(ω).

• The ﬁrst formula uses the probability density;

we can consider that we

integrate the function x 7→ x with respect to the measure dµ = f (x) dx.

• The second formule uses an integral over the sample space Ω, which

is equipped with a σ-algebra and a probability measure, which sufﬁces

to deﬁned an integral in the same manner that the Lebesgue integral is

constructed.

The function considered is this time the random variable

X , and it is integrated with respect to the measure dP(ω).

Mathematicians usually avoid using the density f (which requires the use

of the th eory of d istributions as s oon as F is discontinuous) and appeal in-

stead to another measure denoted dF = f (x) dx and to the so-called “Stieltjes

integrals” (see sidebar 8 on page 552). However, since density probabilities

are probably more intuitive to physicists, we will use those instead, it being

understood that it may be a distr ibution, and must be considered accordingly,

even if it is written “under an integral sign.”

 Exercise 20.2 A random variable X is distributed according to a Cauchy distribution if its

probability density is of the type

f (x) =

+ (x −m)

for some real number m and some positive real number a. Compute the expectation and

standard deviation of a random variable with this distribution (answer in the table, page 616).

And, as already noticed, may be generalized to the case of discontinuous F by considering

f as a distribution and writing 〈f , x 7→ x〉 instead of the integral.

This is indeed what is called the Lebesgue integral with respect to an arbitrary measure on

an arbitrary measure space.

530 Random vari ables

20.4

An example: the Poisson distribution

20.4.a Particles in a conﬁned gas

Consider a box with constant volume V , containing N identical, classical

(i.e., not quantum) particles, which we assume not to interact. Moreover,

assume that the system has reached thermodynamic equilibrium, and is in the

situation of thermodynamic limit (in particular, N is supposed to be very large).

Fix now a small volume δ V ≪ V . What is the probability that one

given par ticle is located in this small volume? If the s ystem is ergodic, this

probability is p = δ V /V .

What is now the probability that there are two particles in this volume?

That there are n particles? More generally, we ask for the probab ility distribu-

tion of the number of part icles contained in δ V .

To start with, notice that since the particles are classical and do not interact,

there is no correlation between their positions. Thus, the probability for

one particle to be in the given volume δ V is independent of the number

of particles already located th ere. Hence, the probability distr ibution for the

number of particles in δ V is the binomial distribution P = B(N , p) wit h

values given by

P(n) =





(1 − p)

N−n

since n particles must be in δ V (with probability p) and N −n must be out-

side (with probability (1−p)). The expected value of the binomial distribution

n =

n=0

n P(n) = N p.

If N goes to inﬁnity, we will see in the next chapter that the binomial d is-

tribution can be approached by a gaussian distribution; however, this is only

correct if N p ≫ 1. For ﬁxed N (which is the case in physics ), if we consider

a very small volume δ V , it is possible to make n = N p = N δ V /V very

small compared to 1. Then we can write





≈

and (1 − p)

N−n

= exp

(N−n) log(1−p)

≈ e

−N p

= e

−n

which gives P(n) ≈

−n

An example: the Poisson distribution 531

DEFINITION 20.21 The Poisson distribution with parameter λ is the discrete

probability measure such th at

(n) = e

−λ

for all n ∈ N.

A random variable X is a Poisson random variable if P{X = n} = P

(n) for

all n ∈ N.

The Poisson distribution is useful to describe rare events.

The maximal

value of n 7→ P

(n) is attained for n ≈ λ. Moreover, the reader will easily

check the following results:

THEOREM 20.22 The e xpectation of a Poisson random with parameter λ is E(X ) =

λ, and its variance is Var(X ) = λ.

20.4.b Radioactive decay

A radiation source randomly emits α-particles. Assume that this radiation

is weak enough that it does not perturb the source (i.e., the power which is

radiated is statistically constant). Moreover, assume that th e probability that

a particle is emitted between t and t + ∆t depends only on ∆t, and not on

t or on the number of α-particles already emitted before time t (this is called

a Markov hypothesis). It seems reasonable to postulate that this probability

is proportional to ∆t w hen ∆t is very small; thus, we take it to be equal to

p∆t.

Let Q(t) be the probability that the source emits no particle during a

length of time of duration t. In the ab sence of time correlation, we have

therefore

Q(t + dt) = (1 − p dt) Q(t),

which, together with the condition Q(0) = 1, implies tha t Q(t) = e

−p t

What is the probability that n par ticles are emitted d uring an interval of

length t? To compute t his, partition the interval [0, t] into a large number

N of small intervals of length ∆t = t/N . D uring each of these intervals

of length ∆t, th e probability of emitting an α-particle is p∆t. Hence the

probability of emitting n particles is given by a binomial distribution, w hich

may be a pproximated by the Poisson distribution with parameter λ = pt:

P(n, t) =





(p∆t)

(1 − p∆t)

N−n

≈

(pt)

−p t

As might b e expected, this probability is maximal when n ≈ p t; the

average number of particles emitted during an interval of length t is given by

the expecta tion of th e Poisson distribution, and is also equal to pt.

The ﬁrst recorded use of the Poisson distribution is found in a statistical study of the

annual number of deaths caused by horse kicks in the Prussian cavalry in 1 898.

532 Random variables

20.5

Moments of a random variable

DEFINITION 20.23 The k-th moment (with k ∈ N) of a random variable X

with distribution function F is the quantity

ˆµ

def

+∞

−∞

f (x) dx =

Ω

(ω) dP(ω)

if it ex ists.

For k ¾ 1, the k-th moment does not necessarily exist. The 0-th moment

always exists, but it is always equal to 1 and is not very interesting. The

ﬁrst moment is simply the expectation of X . One can deﬁne variants of the

moment for a centered random variable, that is, one which expectation equal

to z ero.

DEFINITION 20.24 A centered random variable is a random variable with

expectation equal to 0. If X is a random variable such that the expectation of

X exists, and m = E(X ), then (X −m) is a centered random variable.

DEFINITION 20.25 The k-th centered moment of an r.v. X with expectation

m is given by

def

+∞

−∞

(x −m)

f (x) dx =

Ω



X (ω) −m)

dP(ω).

Remark 20.26 For k ¾ 1, the centered k-th moment exists if and only if the original (non-

centered) k-th moment exists.

Remark 20.27 The centered se cond moment is none other than the variance:

= Var(X ) = σ

(X ).

 Exercise 20.3 Show that µ

= ˆµ

−µ

Show that the kinetic energy of a distribution of mass is the sum of the kinetic energy

in the barycentric referential and the kinetic energy of the center of gravity (also called the

barycenter), carrying the total mass of the system.

The following deﬁnition is sometimes used.

DEFINITION 20.28 A reduced random variable is a random variab le for

which the standard deviation exists and satis ﬁes σ(X ) = 1.

If X is an r.v. with expectation m and standard deviation σ, the random

variable (X −m)/σ is always centered and reduced.

Moments of a random variable 533

DEFINITION 20.29 The median of a random variable X with distribution

function F is the number

1/2

= inf

x ∈ R ; F (x) ¾

If F is continuous, then x

1/2

is a solution of the equation F (x) =

. If,

moreover, F is strictly increasing, th is is the unique solution.

Example 20.30 The median occurs frequently in economics and statistics. The median salary

of a population, for instance, is the salary such that half of the population earn less, and half

earns more. It is therefore different from the average salary.

Take for instance the case of six individuals, named Angelo, Bianca, Corin, Dolabella,

Emilia, and Fang, with salaries as follows:

Angelo 5000 Bianca 9000 Corin 9000

Dolabella 9000 Emilia 5000 Fang 11000

The average salary is $8000. However, $8000 is not the median salary, since only two

persons earn less, and four earn more. In this example, the median salary is $9000.

If three other persons are added to the sample, Grumio, Hamlet and Imogen, earning

$6000, $7 0 0 0 and $8500, respectively, then the median salary becomes $8500.

Example 20.31 (Half-life and lifetime) Consider a radioactive particle that may disintegrate at

any moment. Start the experiment at time t = 0 (the particle has not yet disinteg rated) and

assume that the probability that it does disintegrate during an inﬁnitesimal time interval dt i s

equal to α dt (independently of t). As in Sectio n 20.4.b, one can show that t h e probability that

the parti cle be intact at time t is given by

F (t) = e

−αt

The median of the random variable “time before disintegration” is calle d the half-life (the

probability of disintegration during this length of time is

). To compute it , we must solve the

equation

F (t

1/2

) = exp{−α t

1/2

} =

and therefore we have t

1/2

= (log 2)/α. Note that the expectation of thi s r.v., which is called

the (average) lifetime of the particle, is equal to 1/α. Indeed, the probability density for

disintegration at time t is given by

f (t) = −F

′

(t) = α e

−αt

and the expectation is therefore

τ =

t f (t) dt =

tα e

−αt

dt =

1/2

log 2

> t

1/2

It is such that F (τ) = 1/e.

534 Random variables

20.6

Random vectors

20.6.a Pair of random variables

Sometimes it is necessary to consider a random vector, or more generally an

n-tuple of random variables. We start with the case of a pair (X , Y ) of random

variables, all deﬁned on the same probability space (Ω, Σ, P).

DEFINITION 20.32 The distribution function or joint distribution func-

tion of a pair (X , Y ) of random variables is the function

X Y

(x, y)

def

= P{X ¶ x and Y ¶ y},

deﬁned for x, y ∈ R, that is, the probability that both X is less tha n or equal

to x, and Y is less than or equal to y.

PROPOSITION 20.33 Let F

X Y

be the distribution fu nction of the pair (X , Y ).

Then, for a ny x, y ∈ R, w e have

X Y

(−∞, y) = F

X Y

(x, −∞) = 0 and F

X Y

(+∞, +∞) = 1.

Moreover, F

X Y

in increasing and right-continuous with respect to each variable.

The notion of probability density is also easy to extend:

DEFINITION 20.34 (Joint probabilit y density) The probability density (also

called joint probability density) of the pair (X , Y ) is the function (or some-

times the distrib ut ion)

X Y

(x, y)

def

∂

X Y

∂ x ∂ y

(x, y).

We then have

X Y

(x, y) =

−∞

X Y

(s, t) dt ds.

DEFINITION 20.35 (Marginal distr ibution) Let (X , Y ) be a pair of random

variables. The marginal distribution of X is the distribution of t he random

variable X .

Knowing the (joint) distribution function of the pair (X , Y ), it is very easy

to ﬁnd the distribution function of X , and hence its marginal distribution.

Since F

(x) is the probability that X ¶ x, with no condition whatsoever

imposed on the variable Y , we have

(x) = F (x, +∞),

Random vectors 535

and s imilarly the marginal distribution of Y is given by

( y) = F (+∞, y).

It follows that the marginal probability densities are

(x) =

+∞

−∞

X Y

(x, y) dy and f

( y) =

+∞

−∞

X Y

(x, y) dx.

We now int roduce the moments of a pair:

DEFINITION 20.36 Let (X , Y ) be a pair of r.v. and let k and ℓ be integers.

The ( k , ℓ)-th moment or moment of order ( k , ℓ) of (X , T ) is th e quantity

ˆµ

k,ℓ

def

ℓ

X Y

(x, y) dx dy,

if it ex ists. If we denote m

def

= ˆµ

1,0

and m

def

= ˆµ

0,1

the expecta tions of X and

Y , the corresponding centered moment is given by

k,ℓ

def

(x −m

)

( y −m

)

ℓ

X Y

(x, y) dx dy.

DEFINITION 20.37 The variances of (X , Y ) are the centered moments of

order (2, 0) and (0, 2), denoted Var(X ) = σ

= µ

and Var(Y ) = σ

= µ

The covariance of (X , Y ), denoted Cov(X , Y ), is the centered moment of

order (1, 1), that is,

Cov(X , Y )

def

= µ

We then have the relation

Cov(X , Y ) = E



X −E(X )



Y −E(Y )



Remark 20.38 Note also t he obvious formula Cov(X , X ) = σ

= Var

(X ).

 Exercise 20.4 Let X and Y be two random variables with moments of order 1 and 2. Show

that

Var(X + Y ) = Var(X ) + Var(Y ) + 2 Cov(X , Y ).

Note that the centered moment of order (1, 1) is related with the non-

centered moment by means of the formula µ

= ˆµ

−µ

Finally, we have the correlation of random variables:

DEFINITION 20.39 The correlation of the pair (X , Y ) is the quantity

def

Cov(X , Y )

Var(X ) Var(Y )

if the moments involved e xist.

536 Random variables

This has the following properties (following from the Cauchy-Schwarz in-

equality):

THEOREM 20.40 Let (X , Y ) be a pair of random variables with correlation r,

assumed to exist. Then we have

i) |r| ¶ 1;

ii) |r| = 1 if and only if the random variables X and Y are related by a linear

transformation, more precisely if

Y −m

= r

X −m

almost surely.

DEFINITION 20.41 (Uncorrelated r.v.) Two random variables X and Y are

uncorrelated if their correlation exists and is equal to zero.

The correlation is useful in particular when trying to quantify a link be-

tween two statistical series of numbers. Suppose that for N students, we know

their siz e T

and their weight P

, 1 ¶ n ¶ N , and we want to show that there

exists a statistical link between the s ize a nd the weight. Compute the average

weight P and the average size T . The standard deviations of those random

variables are given by

(T ) =

n=1

− T

and σ

(P ) =

n=1

− P

The correlation between size and weight is then equal to

r =

σ(T ) σ(P )

n=1

− T )(P

− P ).

If it is close to 1, there is a strong positive correlation between size and weight,

that is, the weight increases in general w hen the size increases. I f the correla-

tion is close to −1, the correlation is negative (the taller you are, the ligh ter).

Otherwise, there is little or no correlation. If |r| = 1, there is even an afﬁne

formula relating the two stat istical series.

The correlation must be treated with some care. A value r = 0.8 may

indicate an interesting link, or not, depending on the circumstances. For

instance, the size of the dataset may be crucial, as the following example

(taken from Jean Meeus [66]) indicates.

The table below gives the absolute magnitude

m of some asteroids and the

number of letters ℓ in t heir name:

A logarithmic measure of brightness at ﬁxed distance.