A Modern Introduction to Probability and Statistics, Understanding Why and How - Dekking, Kraaikamp, Lopuhaa, Meester (Современное введение в теорию вероятностей и статистику - Как? и Почему? )
Подождите немного. Документ загружается.
13.6 Exercises 191
13.5 Solutions to the quick exercises
13.1 The answers you have found should be in the neighborhood of the fol-
lowing exact values:
n 1 4 16 400
P
|
¯
X
n
− µ| < 0.5
0.27 0.52 0.85 1.00
13.2 Because Y has an Exp(1) distribution µ = 1 and Var(Y )=σ
2
=1;we
find for k ≥ 1:
P(|Y − µ| <kσ)=P(|Y − 1| <k)
=P(1−k<Y <k+1)=P(Y<k+1)=1−e
−k−1
.
Using this formula and (13.1) we obtain the following numbers:
k 1234
Lower bound from Chebyshev 0 0.750 0.889 0.938
P(|Y − 1| <k) 0.865 0.950 0.982 0.993
13.3 The value of
¯
Y
n
for this bar equals its area 0.26 · 0.5=0.13. The bar
represents 13% of the values, or 0.13 ·500 = 65 realizations.
13.6 Exercises
13.1 Verify the “µ±afewσ” rule as you did in Quick exercise 13.2 for the fol-
lowing distributions: U(−1, 1), U (−a, a), N(0, 1), N (µ, σ
2
), Par(3), Geo(1/2).
Construct a table as in the answer to the quick exercise and enter a line for
each distribution.
13.2 An accountant wants to simplify his bookkeeping by rounding amounts
to the nearest integer, for example, rounding
99.53 and 100.46 both to
100. What is the cumulative effect of this if there are, say, 100 amounts? To
study this we model the rounding errors by 100 independent U (−0.5, 0.5) ran-
dom variables X
1
, X
2
, ..., X
100
.
a. Compute the expectation and the variance of the X
i
.
b. Use Chebyshev’s inequality to compute an upper bound for the probability
P(|X
1
+ X
2
+ ···+ X
100
| > 10) that the cumulative rounding error X
1
+
X
2
+ ···+ X
100
exceeds 10.
192 13 The law of large numbers
13.3 Consider the situation of the previous exercise. A manager wants to
know what happens to the mean absolute error
1
n
n
i=1
|X
i
| as n becomes
large. What can you say about this, applying the law of large numbers?
13.4 Of the voters in Florida, a proportion p will vote for candidate G,
and a proportion 1 −p will vote for candidate B. In an election poll a number
of voters are asked for whom they will vote. Let X
i
be the indicator random
variable for the event “the ith person interviewed will vote for G.” A model
for the election poll is that the people to be interviewed are selected in such
a way that the indicator random variables X
1
, X
2
,. . . are independent and
have a Ber(p) distribution.
a. Suppose we use
¯
X
n
to predict p. According to Chebyshev’s inequality, how
large should n be (how many people should be interviewed) such that the
probability that
¯
X
n
is within 0.2ofthe“true”p is at least 0.9?
Hint: solve this first for p =1/2, and use that p(1 − p) ≤ 1/4 for all
0 ≤ p ≤ 1.
b. Answer the same question, but now
¯
X
n
should be within 0.1ofp.
c. Answer the question from part a, but now the probability should be at
least 0.95.
d. If p>1/2 candidate G wins; if
¯
X
n
> 1/2 you predict that G will win.
Find an n (as small as you can) such that the probability that you predict
correctly is at least 0.9, if in fact p =0.6.
13.5 You are trying to determine the melting point of a new material, of
which you have a large number of samples. For each sample that you measure
you find a value close to the actual melting point c but corrupted with a
measurement error. We model this with random variables:
M
i
= c + U
i
where M
i
is the measured value in degree Kelvin, and U
i
is the occurring
random error. It is known that E[U
i
]=0andVar(U
i
) = 3, for each i,andthat
we may consider the random variables M
1
, M
2
, . . . independent. According
to Chebyshev’s inequality, how many samples do you need to measure to be
90% sure that the average of the measurements is within half a degree of c?
13.6 The casino La bella Fortuna is for sale and you think you might want
to buy it, but you want to know how much money you are going to make. All
the present owner can tell you is that the roulette game Red or Black is played
about 1000 times a night, 365 days a year. Each time it is played you have
probability 19/37 of winning the player’s bet of
1 and probability 18/37 of
having to pay the player
1.
Explain in detail why the law of large numbers can be used to determine the
income of the casino, and determine how much it is.
13.6 Exercises 193
13.7 Let X
1
, X
2
,...bea sequence of independent andidenticallydistributed
random variables with distributions function F . Define F
n
as follows: for any a
F
n
(a)=
number of X
i
in (−∞,a]
n
.
Consider a fixed and introduce the appropriate indicator random variables (as
in Section 13.4). Compute their expectation and variance and show that the
law of large numbers tells us that
lim
n→∞
P(|F
n
(a) − F (a)| >ε)=0.
13.8 In Section 13.4 we described how the probability density function
could be recovered from a sequence X
1
, X
2
, X
3
, .... We consider the
Gam(2, 1) probability density discussed in the main text and a histogram bar
around the point a =2.Thenf(a)=f (2) = 2e
−2
=0.27 and the estimate
for f (2) is
¯
Y
n
/2h,where
¯
Y
n
as in (13.3).
a. Express the standard deviation of
¯
Y
n
/2h in terms of n and h.
b. Choose h =0.25. How large should n be (according to Chebyshev’s in-
equality) so that the estimate is within 20% of the “true value”, with
probability 80%?
13.9 Let X
1
, X
2
, . . . be an independent sequence of U(−1, 1) random
variables and let T
n
=
1
n
n
i=1
X
2
i
. It is claimed that for some a and any
ε>0
lim
n→∞
P(|T
n
− a| >ε)=0.
a. Explain how this could be true.
b. Determine a.
13.10 Let M
n
be the maximum of n independent U (0, 1) random variables.
a. Derive the exact expression for P(|M
n
− 1| >ε).
Hint: see Section 8.4.
b. Show that lim
n→∞
P(|M
n
− 1| >ε) = 0. Can this be derived from Cheby-
shev’s inequality or the law of large numbers?
13.11 For some t>1, let X be a random variable taking the values 0 and t,
with probabilities
P(X =0)=1−
1
t
and P(X = t)=
1
t
.
Then E[X]=1andVar(X)=t −1. Consider the probability P(|X − 1| >a).
a. Verify the following: if t =10anda =8thenP(|X −1| >a)=1/10 and
Chebyshev’s inequality gives an upper bound for this probability of 9/64.
The difference is 9/64 − 1/10 ≈ 0.04. We will say that for t =10the
Chebyshev gap for X at a =8is0.04.
194 13 The law of large numbers
b. Compute the Chebyshev gap for t =10ata =5andata = 10.
c. Can you find a gap smaller than 0.01, smaller than 0.001, smaller than
0.0001?
d. Do you think one could improve Chebyshev’s inequality, i.e., find an upper
bound closer to the true probabilities?
13.12 (A more general law of large numbers). Let X
1
,X
2
,... be a
sequence of independent random variables, with E[X
i
]=µ
i
and Var(X
i
)=
σ
2
i
,fori =1, 2,.... Suppose that 0 <σ
2
i
≤ M , for all i.Leta be an arbitrary
positive number.
a. Apply Chebyshev’s inequality to show that
P
¯
X
n
−
1
n
n
i=1
µ
i
>a
≤
Var(X
1
)+···+Var(X
n
)
n
2
a
2
.
b. Conclude from a that
lim
n→∞
P
¯
X
n
−
1
n
n
i=1
µ
i
>a
=0.
Check that the law of large numbers is a special case of this result.
14
The central limit theorem
The central limit theorem is a refinement of the law of large numbers.
For a large number of independent identically distributed random variables
X
1
,...,X
n
, with finite variance, the average
¯
X
n
approximately has a normal
distribution, no matter what the distribution of the X
i
is. In the first section
we discuss the proper normalization of
¯
X
n
to obtain a normal distribution
in the limit. In the second section we will use the central limit theorem to
approximate probabilities of averages and sums of random variables.
14.1 Standardizing averages
In the previous chapter we saw that the law of large numbers guarantees
the convergence to µ of the average
¯
X
n
of n independent random variables
X
1
,...,X
n
, all having the same expectation µ and variance σ
2
.Thisconver-
gence was illustrated by Figure 13.1. Closer examination of this figure suggests
another phenomenon: for the two distributions considered (i.e., the Gam (2, 1)
distribution and a bimodal distribution), the probability density function of
¯
X
n
seems to become symmetrical and bell shaped around the expected value µ
as n becomes larger and larger. However, the bell collapses into a single spike
at µ. Nevertheless, by a proper normalization it is possible to stabilize the
bell shape, as we will see.
In order to let the distribution of
¯
X
n
settle down it seems to be a good idea
to stabilize the expectation and variance. Since E
¯
X
n
= µ for all n, only the
variance needs some special attention. In Figure 14.1 we depict the probability
density function of the centered average
¯
X
n
−µ of Gam (2, 1) random variables,
multiplied by three different powers of n. In the left column we display the
density of n
1
4
(
¯
X
n
−µ), in the middle column the density of n
1
2
(
¯
X
n
−µ), and
in the right column the density of n(
¯
X
n
− µ). These figures suggest that
√
n
is the right factor to stabilize the bell shape.
196 14 The central limit theorem
0.0
0.2
0.4
n =1
...............................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
..
.
..
.
..
....
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
n =1
...............................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
..
.
..
.
..
....
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
n =1
...............................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
..
.
..
.
..
....
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
0.0
0.2
0.4
n =2
..............................................................................
...
..
.
..
.
.
..
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
..
.
...
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
n =2
........................................................
...
...
.
..
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
...
..
...
.
..
.
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
n =2
..............................................
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
...
.....
....
.....
...
.
..
...
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
0.0
0.2
0.4
n =4
.......................................................................
......
.
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
n =4
.........................................................
...
..
.
..
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
...
.....
.
..
.
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
n =4
..............................................
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
...
......
..
......
...
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
0.0
0.2
0.4
n =16
...................................................................................
......
...
...
.
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
...
.
...
...
...
......
........
..
n =16
..............................................
.
.
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
...
...
...
...
..
.
.
.
.
..
.
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
n =16
..............................................
.
.......
........
.........
.........
..........
..............
......................
......................................
......................
...
............
...........
..........
.........
........
.......
........
.......
......
......
......
......
......
......
......
.....
......
....
...
..
......
....
.....
.....
−3 −2 −10 1 2 3
0.0
0.2
0.4
n = 100
.................................................................................................................
....
...
..
.
..
.
..
.
..
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
...
...
........
.....................................................
−3 −2 −10 1 2 3
n = 100
..............................................
.
.
..
.
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.........
...
.
..
.
..
.
..
.
..
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
−3 −2 −10 1 2 3
n = 100
..............................................
.....................................................................................................................................................
..........................................................................................
..........................................................
Fig. 14.1. Multiplying the difference
¯
X
n
−µ of n Gam (2, 1) random variables. Left
column: n
1
4
(
¯
X
n
− µ); middle column:
√
n(
¯
X
n
− µ); right column: n(
¯
X
n
− µ).
14.1 Standardizing averages 197
Indeed, according to the rule for the variance of an average (see page 182),
we have Var
¯
X
n
= σ
2
/n, and therefore for any number C:
Var
C(
¯
X
n
− µ)
=Var
C
¯
X
n
= C
2
Var
¯
X
n
= C
2
σ
2
n
.
To stabilize the variance we therefore must choose C =
√
n.Infact,bychoos-
ing C =
√
n/σ,onestandardizes the averages, i.e., the resulting random vari-
able Z
n
, defined by
Z
n
=
√
n
¯
X
n
− µ
σ
,n=1, 2,...,
has expected value 0 and variance 1. What more can we say about the distri-
bution of the random variables Z
n
?
In case X
1
,X
2
,... are independent N(µ, σ
2
) distributed random variables,
we know from Section 11.2 and the rule on expectation and variance under
change of units (see page 98), that Z
n
has an N(0, 1) distribution for all n.For
the gamma and bimodal random variables from Section 13.1 we depicted the
probability density function of Z
n
in Figure 14.2. For both examples we see
that the probability density functions of the Z
n
seem to converge to the prob-
ability density function of the N(0, 1) distribution, indicated by the dotted
line. The following amazing result states that this behavior generally occurs
no matter what distribution we start with.
The central limit theorem. Let X
1
,X
2
,... be any sequence
of independent identically distributed random variables with finite
positive variance. Let µ be the expected value and σ
2
the variance
of each of the X
i
.Forn ≥ 1, let Z
n
be defined by
Z
n
=
√
n
¯
X
n
− µ
σ
;
then for any number a
lim
n→∞
F
Z
n
(a)=Φ(a),
where Φ is the distribution function of the N (0, 1) distribution. In
words: the distribution function of Z
n
converges to the distribution
function Φ of the standard normal distribution.
Note that
Z
n
=
¯
X
n
− E
¯
X
n
Var
¯
X
n
,
which is a more direct way to see that Z
n
is the average
¯
X
n
standardized.
198 14 The central limit theorem
0.0
0.2
0.4
0.6
0.8
1.0
n =1
......................................................................................................................................
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
...
..
..........
....
.
...
.
...
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
.
.
.
..
.
.
..
.
.
..
.
...
.
.
..
.
.
...
.
..
..
..
.
...
.
.
...
.
...
.
...
...
.
...
..
..
..
...
...
..
...
...
..
...
....
...
....
..
....
...
....
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
n =1
........................................................................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
.
.
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
...
..
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
..
.
.
..
.
.....
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
...
.
.
..
...
...
...
.....
......
......
........
......
.......
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
0.0
0.2
0.4
0.6
0.8
1.0
n =2
..........................................................................................................................................
....
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
..
..
........
..
..
..
..
.
.
..
.
.
.
.
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
..
..
.
.
...
.
...
.
...
.
...
...
.
...
..
...
....
.
..
..
....
....
....
....
.....
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
n =2
.........................................................................................................................................
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
..
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
....
..
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
..
..
..
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
......
..
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
...
.
.
..
..
..
..
...
....
...
....
.....
........
.........
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0.0
0.2
0.4
0.6
0.8
1.0
n =4
..........................................................................................................
.........
....
...
..
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
...
..
.......
...
..
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
..
..
.
...
.
.
...
...
.
...
..
...
...
..
...
....
.
...
...
.....
....
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
n =4
..................................................................................................
.
..
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
..
......
.
.
..
.
.
.
..
.
.
.
.
..
.
..
..
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
...
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.......
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
..
.
.
.
........
.
.
...
.
........
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
...
.
...
.
..
...
..
....
....
.....
.....
...
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0.0
0.2
0.4
0.6
0.8
1.0
n =16
.....................................................................
...............
.......
.....
....
....
...
...
.
...
.
...
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
..
...
.........
.
..
...
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
...
.
.
...
.
...
.
...
...
..
...
...
..
....
..
..
....
....
.....
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
n =16
......................................................................
.
..
.....
......
...
....
...
...
..
..
...
..
.
..
..
..
.
.
..
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
..
..
..
..
......
......
..
.
.
..
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
..
.
.
.
...
..
.
..
..
..
...
..
..
....
...
....
.....
.....
.....
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
−3 −2 −10123
0.0
0.2
0.4
0.6
0.8
1.0
n = 100
.....................................................................
.......
.......
.....
....
....
...
...
..
...
.
...
.
..
..
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
..
............
...
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
...
.
..
..
..
..
..
..
...
...
..
....
...
....
.....
......
...
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
−3 −2 −10 1 2 3
n = 100
.....................................................................
.......
......
......
.....
....
...
...
...
...
..
...
...
.
..
...
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
...
..
...
.............
...
..
...
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
.
.
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
.
...
..
...
.
...
..
...
...
.
...
...
.....
....
......
......
.......
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Fig. 14.2. Densities of standardized averages Z
n
. Left column: from a gamma den-
sity; right column: from a bimodal density. Dotted line: N (0, 1) probability density.
14.2 Applications of the central limit theorem 199
One can also write Z
n
as a standardized sum
Z
n
=
X
1
+ ···+ X
n
− nµ
σ
√
n
. (14.1)
In the next section we will see that this last representation of Z
n
is very
helpful when one wants to approximate probabilities of sums of independent
identically distributed random variables.
Since
¯
X
n
=
σ
√
n
Z
n
+ µ,
it follows that
¯
X
n
approximately has an N(µ, σ
2
/n) distribution; see the
change-of-units rule for normal random variables on page 106. This explains
the symmetrical bell shape of the probability densities in Figure 13.1.
Remark 14.1 (Some history). Originally, the central limit theorem was
proved in 1733 by De Moivre for independent Ber (
1
2
) distributed random
variables. Lagrange extended De Moivre’s result to Ber (p) random variables
and later formulated the central limit theorem as stated above. Around
1901 a first rigorous proof of this result was given by Lyapunov. Several
versions of the central limit theorem exist with weaker conditions than those
presented here. For example, for applications it is interesting that it is not
necessary that all X
i
have the same distribution; see Ross [26], Section 8.3,
or Feller [8], Section 8.4, and Billingsley [3], Section 27.
14.2 Applications of the central limit theorem
The central limit theorem provides a tool to approximate the probability
distribution of the average or the sum of independent identically distributed
random variables. This plays an important role in applications, for instance,
see Sections 23.4, 24.1, 26.2, and 27.2. Here we will illustrate the use of the
central limit theorem to approximate probabilities of averages and sums of
random variables in three examples. The first example deals with an average;
the other two concern sums of random variables.
Did we have bad luck?
In the example in Section 13.3 averages of independent Gam (2, 1) distributed
random variables were simulated for n =1,...,500. In Figure 13.2 the realiza-
tion of
¯
X
n
for n = 400 is 1.99, which is almost exactly equal to the expected
value 2. For n = 500 the simulation was 2.06, a little bit farther away. Did
we have bad luck, or is a value 2.06 or higher not unusual? To answer this
question we want to compute P
¯
X
n
≥ 2.06
. We will find an approximation
of this probability using the central limit theorem.
200 14 The central limit theorem
Note that
P
¯
X
n
≥ 2.06
=P
¯
X
n
− µ ≥ 2.06 − µ
=P
√
n
¯
X
n
− µ
σ
≥
√
n
2.06 − µ
σ
=P
Z
n
≥
√
n
2.06 − µ
σ
.
Since the X
i
are Gam(2, 1) random variables, µ =E[X
i
]=2andσ
2
=
Var(X
i
) = 2. We find for n = 500 that
P
¯
X
500
≥ 2.06
=P
Z
500
≥
√
500
2.06 − 2
√
2
=P(Z
500
≥ 0.95)
=1− P(Z
500
< 0.95) .
It now follows from the central limit theorem that
P
¯
X
500
≥ 2.06
≈ 1 − Φ(0.95) = 0.1711.
This is close to the exact answer 0.1710881, which was obtained using the
probability density of
¯
X
n
as given in Section 13.1.
Thus we see that there is about a 17% probability that the average
¯
X
500
is at
least 0.06 above 2. Since 17% is quite large, we conclude that the value 2.06
is not unusual. In other words, we did not have bad luck; n = 500 is simply
not large enough to be that close. Would 2.06 be unusual if n = 5000?
Quick exercise 14.1 Show that P
¯
X
5000
≥ 2.06
≈ 0.0013, using the central
limit theorem.
Rounding amounts to the nearest integer
In Exercise 13.2 an accountant wanted to simplify his bookkeeping by round-
ing amounts to the nearest integer, and you were asked to use Chebyshev’s
inequality to compute an upper bound for the probability
p =P(|X
1
+ X
2
+ ···+ X
100
| > 10)
that the cumulative rounding error X
1
+ X
2
+ ···+ X
100
exceeds 10. This
upper bound equals 1/12. In order to know the exact value of p one has to
determine the distribution of the sum X
1
+···+X
100
. This is difficult, but the
central limit theorem is a handy tool to get an approximation of p. Clearly,
p =P(X
1
+ ···+ X
100
< −10) + P(X
1
+ ···+ X
100
> 10) .
Standardizing as in (14.1), for the second probability we write, with n = 100