Wilkinson D.J. Stochastic Modelling for Systems Biology
Подождите немного. Документ загружается.
54
PROBABILITY
MODELS
we only require independence
of
subsets
of
size r =
2.
That is, pair-wise indepen-
dence
does not imply mutual independence.
Definition 3.6 A partition
of
a sample space is simply the decomposition
of
the sam-
ple
space into a collection
of
mutually exclusive events with positive probability. That
is,
{ B1,
...
,
Bn}
forms a partition
of
S
if
n
• S =
B1
U
B2
U ·
..
U
Bn
=
UBi,
i=l
•
Bi
n
Bi
=
0,
Vi
#
j,
• P
(Bi)
> 0,
Vi.
Theorem 3.1 (Theorem
of
total probability) Suppose that we have a partition { B
1
,
...
,
Bn}
of
a sample space, S. Suppose further that we have an event A. Then
)
n
p (A) = L p (AIBi) p
(Bi).
i=l
Proof
A can be written as the disjoint union
A=
(An
B
1
)
U · · · U
(An
Bn),
and so the probability
of
A is given by
0
P
(A)=
P
((An
B1) u
..
· U
(An
Bn))
= P
(An
BI)
+ · · · + P
(An
Bn)
= P (AIB1) P (B1) + · · · + P (AIBn) P (Bn)
n
= Lp
(AIBi)
P
(Bi).
i=l
(by Axiom III)
(by
Proposition 3.4)
Theorem 3.2 (Bayes Theorem) For all events A, B such that P
(B)
> 0 we have
P
(AlB)
= P
(BIA)
P (A)
P(B)
.
This is a very important result
as
it tells us how to "turn conditional probabilities
around"-
that is, it tells us how to work out P
(AlB)
from P (BIA), and this is
often very useful.
Proof
By Definition 3.4 we have
0
P(AIB)
=
P(AnB)
P(B)
P (A)
P(BIA)
P(B)
(by Proposition 3.4)
PROBABILITY
55
.Example
A clinic offers you a free test for a very rare, but hideous disease. The test they offer
is very reliable.
If
you have the disease it has a 98% chance
of
giving a positive
result, and
if
you do not have the disease,
it
has only a
1%
chance
of
giving a positive
result. Despite having no a priori reason to suppose that you have the disease, you
nevertheless decide to take the test and find that you test positive - what is the
probability that you have the disease?
Let
P be the event "test positive" and D be the event "you have the disease."
We
know that
P (PID) = 0.98 and that P (PIDc) = 0.01.
We
want to know P (DIP), so we use Bayes Theorem.
p (DIP) = p (PID) p (D)
P(P)
P (PID)P (D)
P (PI D) P (D) + P (PIDc) p
(De)
0.98P
(D)
= 0.98 P
(D)+
0.01(1-
P
(D)).
(using Theorem 3.1)
So we see that the probability you have the disease given the test result depends
on the probability that you had the disease in the first place. This is a rare disease,
affecting only one in ten thousand people, so that
P (D) = 0.0001. Substituting this
in gives
P
(DI
P)
=
0.9~
x 0.0001
~
O
0.98
X 0.0001 +
O.Ql
X 0.9999 - .Ol.
So, your probability
of
having the disease has increased from 1 in 10,000 to 1 in 100,
but still is not that much to get worried about! Note the crucial difference between
P (PID) and P (DIP).
Another important thing to notice about the above example is the use of the theo-
rem
of
total probability in order to expand the bottom line
of
Bayes Theorem. In fact,
this is done so often that Bayes Theorem is often stated in this form.
Corollary 3.1 Suppose that we have a partition {
E1,
...
, En}
of
a sample
spaceS.
Suppose further that we have an event A, with P (A) >
0.
Then,
for
each Ej, the
probability
of
Ej
given A is
P(E·IA)
=
P(AIEi)P(Ej)
J p (A)
P (AIEi) P
(Ei)
P (AIE1) P (E1) + · · · + P (AIEn) P (En)
P(AIEi)P(Ej)
n
Lp
(A!Ei) P (Ei)
i=l
56
PROBABILITY
MODELS
In
particular,
if
the partition is simply { B,
Be},
then
this
simplifies to
P
(AIB)P(B)
p (BjA) = p (AlB) P (B) + P (AjBe) P (Be).
3.2 Discrete probability models
3.2.1 Introduction, mass functions, and distribution functions
We have seen how to relate events to sets and how
to
calculate probabilities for events
by
working with the sets that represent them. So far, however,
we
have not developed
any special techniques for thinking about random quantities. Discrete probability
models provide a
frl!lllework for thinking about discrete random quantities, and con-
tinuous probability models (to
be
considered
in
thenext
section) form a framework
for thinking about continuous random quantities.
Example
Consider the sample space for tossing a fair coin twice:
S =
{HH,HT,TH,TT}.
These outcomes are equally likely. There are several random quantities we could
associate with this experiment. For example, we could count the number
of
heads or
the number
of
tails.
Definition
3.7 A random quantity is a real valued function which acts on elements
of
the sample space
(outcomes)~
That
is,
to each outcome, the random variable assigns
a real number.
Random quantities (sometimes known as random variables) are always denoted by
upper case letters.
·
In
our example,
if
we let X
be
the number
of
heads,
we
have
X(HH)
=2,
X(HT)
= 1,
X(TH)
=
1,
X(TT)
=
0.
The observed value
of
a random quantity is the number corresponding to the actual
outcome. That is,
if
the outcome
of
an experiment
iss
E
S,
then
X(s)
E
lR
is the
observed value. This observed value is always denoted with a lower case letter -
here
x. Thus X = x means that the observed value
of
the random quantity X is the
number
x. The set
of
possible observed values for X is
Sx
=
{X(s)js
E S}.
For
the above example
we
have
Sx
=
{0,
1,
2}.
Clearly here the values are not all equally likely.
.,
.;
'
~-
DISCRETE
PROBABILITY
MODELS
57
Example
Roll one die and call the random number which is uppermost
Y.
The sample space
for the
random quantity Y is
Sy
=
{1,2,3,4,5,6}
and these outcomes are all equally likely. Now roll two dice and call their sum Z.
The sample space for Z is
Sz = {2,3,4,5,6,7,8,9,10,11,12}
and these outcomes are not equally likely. However, we know the probabilities
of
the events corresponding to each
of
these outcomes, and we could display them in a
table as follows.
Outcome
2 4
6
7
9 10
11
12
Probability
1/36
2136
3/36 4/36 5/36
6136
5/36
4136
3/36
2136
1/36
This is essentially a tabulation
of
the probability mass function for the random quan-
tity
z.
Definition 3.8 (probability mass function)
For
any discrete random variable
X,
we define the probability mass function (PMF)
to
be the function which gives the
·
•:
probability
of
each x E S x. Clearly we have
P(X=x)=
P({s}).
{sESJX(s)=x}
That is, the probability
of
getting a particular number is the sum
of
the probabilities
of
all those outcomes which have that number associated with them.
AlsoP
(X
= x)
2:
0 for each x E S
x,
and P
(X
= x) = 0 otherwise.
Definition 3.9
The
set
of
all pairs { (x, P
(X
= x))
lx
E
Sx}
is
known as the prob-
ability distribution
of
X.
Example
For the example above concerning the sum
of
two dice, the probability distribution
is
{
(2,
1/36),
(3,
2/36), (
4,
3/36),
(5,
4/36),
(6,
5/36),
(7,
6/36),
(8,5/36),(9,4/36),(10,3/36),(11,2/36),(12,1/36)}
and the probability mass function can be tabulated as follows.
X
2
4 5
6 7 9 10
11
12
p
(X
=
X)
1/36
2/36 3/36 4/36
5136
6/36 5/36 4/36 3/36
2136
1/36
For any discrete random quantity,
X,
we clearly have
L
P(X=x)=1
xESx
as every outcome has some number associated with it.
It
can often be useful to know
the probability that your random number is no greater than some particular value.
58
C!
a)
0
<0
0
x
i.i:'
...
ci
"!
0
0
ci
2
4
6
_____.
'
'
'
X
8
PROBABILITY MODELS
10
12
Figure 3.1 CDF
for
the
sum
of
a pair
of
fair
dice
Definition 3.10 (cumulative distribution function) The cumulative distribution func-
tion (CD
F), is defined by
Fx(x)
= P
(X::;
x)
=
P(X=y).
{yES
x
iy:Sx}
Example
For
the sum
of
two dice, the
CDF
can be tabulated for the outcomes as
X 2 3 4 6 7 8 9 J0
!!
!2
Fx(x)
1136
3/36
6/36
10/36 15/36
21136
26/36
30/36
33/36 35/36 36/36
but
it
is important to note that the
CDF
is defined
for
all real
numbers-
not
just
the
possible values. In our example we have
Fx(
-3)
= P
(X::;
-3)
=
0,
Fx(4.5) = P
(X::;
4.5) = P
(X::;
4)
= 6/36,
Fx(25)
= P
(X::;
25)
= 1.
We may plot the
CDF
for
our
example as shown in Figure 3.1.
It
is clear that for any random variable
X,
for all x E
JR,
Fx(x)
E
[0,
1]
and that
Fx(x)--+
0 as x-->
-oo
and
Fx(x)-->
1 as x--> +oo.
3.2.2 Expectation
and
variance
for
discrete random quantities
It
is useful to be able to summarise the distribution
of
random quantities. A location
measure
often used to summarise random quantities is the expectation
of
the random
quantity.
It
is the "centre
of
mass"
of
the probability distribution.
DISCRETE PROBABILITY MODELS
59
Definition 3.11 The expectation
of
a discrete random quantity
X,
written E (X) is
defined
by
E(X)=
:LxP(X=x).
xESx
The expectation is often denoted by
;.tx
or even just
p,.
Note that the expectation is
a known function
of
the probability distribution.
It
is not a random quantity, and it
is not the sample mean
of
a set
of
data (random or otherwise). In some sense (to be
made precise in
Proposition 3.14), it represents the value
of
the random variable that
you expect
to
get on average.
Example
For the sum
of
two dice,
X,
we have
1 2 3 1
E (X) = 2 X
36
+ 3 X
36
+ 4 X
36
+ · · · +
12
X
36
=
7.
By looking at the symmetry
of
the mass function, it is clear that in some sense 7 is
the
"central" value
of
the probability distribution.
As well
as
a method for summarising the location
of
a given probability distribu-
tion, it is also helpful
to
have a summary for the spread.
Definition 3.12 For a discrete random quantity
X,
the variance
of
X is defined by
Var(X)=
L
{(x-E(X))
2
P(X=x)}.
xESx
The variance is often denoted
o-~,
or
even just
o-
2
•
Again, this is a known function
of
the probability distribution. It is not random, and it is not the sample variance
of
a
set
of
data. The variance can be rewritten as
Var(X)
= L
xrP(X
=Xi)-
[E(X)V,
x,ESx
and this expression
is
usually a bit easier to work with.
We
also define the standard
deviation
of
a random quantity by
SD(X) =
y'Var
(X),
and this
is
usually denoted by
o-x
or just
o-.
The variance represents the average
squared distance
of
X from E (X). The units
of
variance are therefore the square
of
the units
of
X (and E (X)). Some people prefer to
work
with the standard deviation
because it has the same units
as
X and E (X).
Example
For the sum
of
two dice,
X,
we have
L
2 ( ) 2 1 2 2 2 3 2 1
329
X.
p X =
Xi
= 2 X - + 3 X - + 4 X - + ... +
12
X - = -
S
. '
36
36
36 36
6
XiE
X
60
PROBABILITY MODELS
and
so
Var
(X)
= 329 _
72
=
35,
6 6
{35
and
SD(X)
= y
6
.
3.2.3 Properties
of
expectation and variance
One
of
the reasons that expectation is widely used
as
a measure
of
location for prob-
ability distributions is the fact that it has many desirable mathematical properties
which make it elegant and convenient to work with. Indeed, many of the nice prop-
erties
of
expectation lead to corresponding nice properties for variance, which is one
of the reasons why variance is widely used
as
a measure
of
spread.
Suppose that X is a discrete random quantity, and that Y is another random quan-
tity that is a known function
of
X.
That is, Y
:=
g(X)
for some function g(-). What
is the expectation
of
Y?
Example
Throw a die, and let X be the number showing.
We
have
Sx
=
{1,2,3,4,5,6}
and each value is equally likely. Now suppose that we are actually interested in the
square
of
the number showing. Define a new random quantity Y = X
2
•
Then
Sy
=
{1,4,9,16,25,36}
and clearly each
of
these values is equally likely.
We
therefore have
1 1 1
91
E (Y) = 1 X 6 + 4 X 6 +
..
· + 36 X 6 =
6.
The above example illustrates the more general result, that for Y =
g(X),
we have
E(Y)=
2:
g(x)P(X=x).
xESx
Note that
in
general E
(g(X))
f=
g(E
(X)).
For the above example, E
(X
2
)
91/6
~
15.2, and E
(X)
2
= 3.5
2
= 12.25.
We can use this more general notion
of
expectation in order to redefine variance
purely in terms
of
expectation as follows:
Var
(X)=
E
([X-
E
(X}?)
= E
(X
2
)-
[E
(XW.
Having said that E
(g(X))
1=-
g(E
(X))
in general, it does in fact hold in the (very)
special, but important case where g ( ·) is a linear function.
Lemma 3.1 (expectation
of
a linear transformation)
If
we
have a random quan-
tity
X,
and a linear transformation, Y =
aX+
b,
where a and bare known
real
constants, then we have that
E(aX
+b)=
aE(X)
+b.
j l
t
: I
DIScllliTE
PROBABILITY MODELS
Proof.
E
(aX+
b)=
L
(ax+
b)
P
(X=
x)
xESx
= L
axP(X=x)+
L
bP(X=x)
xESx xESx
=a
L
xP(X=x)+b
L
P(X=x)
xESx
xESx
=
aE(X)
+b.
0
61
Lemma 3.2 (expectation
of
a snm)
For
two random quantities X
andY,
the ex-
pectation
of
their sum is given
by
E
(X+
Y)
= E
(X)+
E
(Y).
Note that this result is true irrespective
of
whether X and Y are independent. Let us
see why.
Proof. First,
Sx+Y
=
{x
+ Yi(x E
Sx)
n
(y
E
Sy
)},
and so
E(X
+
Y)
= L (x +
y)
P
((X=
x) n
(Y
= y))
=
_L _L
( x + y) P ( ( x = x) n
(Y
= y))
xESx
yESy
=
_L
_L
x P ((X = x) n
(Y
= y))
xESx
yESy
+
_L
_L
y P ((X = x) n
(Y
= y))
xESxyESy
L L
xP(X
=
x)P(Y
=
yiX
=
x)
xESx
yESy
+ L L
yP(Y=y)P(X=xiY=y)
yESy
xESx
L
xP(X=x)
L
P(Y=yiX=x)
xESx
yESy
+
_LyP(Y=y)
2:::
P(X=xiY=y)
yESy
xESx
L
xP(X=x)+
L
yP(Y=y)
xESx
yESy
= E
(X)+
E
(Y).
0
62
PROBABILITY
MODELS
We can now put together the result for the expectation
of
a sum, and for the ex-
pectation
of
a linear transformation in order to give the following important property,
commonly referred to as the
linearity
of
expectation.
Proposition
3.5
(expectation
of
a
linear
combination)
For
any
random
quantities
X
1,
...
,
Xn
and scalar constants
ao,
...
, an we have
E(ao
+ a1X1
+a2X2
+ · · · +
anXn)
=
ao
+
a1
E(X1)
+
a2E(X2)
+ · · ·
+anE(Xn).
Lemma
3.3 (expectation
of
an
independent
product)
If
X
andY
are
independent
random quantities, then
E
(XY)
= E
(X)E
(Y).
Proof.
Sxy
=
{xy!(x
E
Sx)
n (y E Sy)},
and so
'
E(XY)
= L
xyP
((X=
x)
n
(Y
= y))
xyESxy
= L L
xyP(X=x)P(Y=y)
xESx
yESy
= L
xP(X=x)
L
yP(Y=y)
xESx
yESy
=
E(X)E(Y)
0
Note that here it is vital that X and Y are independent or the result does not hold.
Lemma
3.4 (variance
of
a linear transformation)
If
X
is
a random quantity with
finite variance
Var
(X),
then
Var(aX
+b)=
a
2
Var(X).
Proof.
Var
(aX+
b)=
E
([aX+
b]
2
)-
[E
(aX+
b)]
2
= E (a
2
X
2
+
2abX
+ b
2
) -
[aE
(X)+
W
= a
2
[E
(X
2
)-
E(X)
2
]
=a
2
Var(X).
0
Lemma
3.5 (variance
of
an
independent
sum)
If
X and Y
are
independent
ran-
dom quantities, then
Var
(X
+
Y)
= Var
(X)
+ Var (Y) .
DISCRETE PROBABILITY MODELS
Proof
Var
(X+
Y) = E
([X+
Y]
2
) -
[E
(X+
Y)j2
= E
(X
2
+
2XY
+ Y
2
)-
(E(X)
+E(Y)J
2
=E(X
2
)
+2E(XY)+E(Y
2
)
-E(X)
2
-2E(X)E(Y)
63
-
E(Y)
2
= E
(X
2
)
+ 2 E
(X)
E (Y) + E
(Y
2
) -
E
(X)
2
- 2 E
(X)
E (Y)
-
E (Y)
2
0
= E
(X
2
)
- E (X)
2
+ E (Y
2
) - E (Y)
2
= Var (X) + Var (Y)
Again,
it
is
vital that X and Y are independent or the result does not hold. Notice
that this implies a slightly less attractive result for the standard deviation
of
the sum
of
two independent random quantities,
SD(X
+ Y) =
vSD(X)
2
+ SD(Y)
2
,
which is why it is often more convenient to work with variances. t
Putting together previous results we get the following proposition.
Proposition 3.6 (variance
of
a linear combination) For mutually independent X
1
,
X2,
...
,
Xn
we have
Var
(ao
+
a1X1
+ a2X2 + · · · +
anXn)
=
ai
Var
(XI)+
a~
Var (X2) + · · ·
+a~
Var
(Xn).
Before moving on to look at some interesting families
of
probability distributions,
it is worth emphasising the link between expectation and the theorem
of
total proba-
bility.
Proposition 3.7
IfF
is
an event and X is a discrete random quantity with outcome
space
Sx,
then by the theorem
of
total probability (Theorem 3.1) we have
P(F)=
L
P(FIX=x)P(X=x)
xESx
= E
(P
(FIX)),
where P
(FIX)
is the random quantity which takes the value P
(FIX=
x)
when X
takes the value
x.
t Note the similarity to Pythagoras' Theorem for the lengths
of
the sides
of
a triangle. Viewed in the
correct
way,
this result is seen to be a special case
of
Pythagoras' Theorem,
as
the standard devia-
tion
of
a random quantity can be viewed
as
a length, and independence leads to orthogonality
in
an
appropriately constructed inner-product space.