King M.R., Mody N.A. Numerical and Statistical Methods for Bioengineering: Applications in MATLAB

Подождите немного. Документ загружается.

propose the normal distribution, which is discus sed in his seminal book on proba-

bility, The Doctrine of Chances . This distribution is, however, named after Carl

Friedrich Gauss (1777–1855), who put forth its mathematical deﬁnition, and used

the assumption of a normal distribution of errors to rigorously prove the linear least-

squares method for estimation of model parameters. Before we discuss the normal

distribution any further, we must ﬁrst introduce the concept of the probability

density funct ion.

3.5.1 Continuous probability distributions

As discussed earlier, histograms pictorially convey the frequency distribution of a

population variable or a random variable. When the random variable is a continu-

ous variable, the class inter vals can be constructed of any width. As the number of

observations and the number of class intervals within the entire interval of study are

simultaneously increased, the resulting histogram begins to take on the shape of a

well-deﬁned curve. (What happens to the shape of a histogram if we have only a few

observations and we repeatedly decrease the size of the class intervals?) Figure 3.7

shows the relative frequency distribution or probability distribution after halving the

size of the class intervals in Figure 3.3. Note that this has effectively reduced the

individual class interval probabilities by a factor of 2. A smooth curve can theoret-

ically be obtained if the class intervals become inﬁnitely small and the number of

class intervals rises to inﬁnity. The total number of observations plotted thereby

proceeds to inﬁnity. Note that if the measured variable is of discrete nature, you

should not attempt to draw a curve connecting the midpoints of the top edges of the

bars, since continuous interpolation for a discrete variable is not deﬁned.

When the relative frequencies displayed on the y-axis are converted to a density scale

such that the entire area under the curve is 1, then the smooth curve is termed as a

probability density curve or simply a density curve (see Figure 3.8). The area under the

curve between any two values of the measured variable (random variable) x,sayx

and

, is equal to the probability of obtaining a value of x within the interval x

 x  x

Figure 3.7

Halving the size of the class intervals adds definition to the shape of the relative frequency histogram

(area = 12.1 minutes).

0 100 200 300 400

0.02

0.04

0.06

0.08

0.1

0.12

Moderate to vigorous physical activity

per weekday (minutes)

Probability distribution

167

3.5 Normal distribution

The mathematical formula that describes the shape of the density curve is called the

density function or simply the density, f(x). Thus, the probability density function

satisﬁes the following requirement: the area under the density curve is given by

∞

∞

fxðÞdx ¼ 1: (3:21)

The probability P(x) that the measurement x will lie in the interval [x

, x

] is given by

PxðÞ¼

fxðÞdx: (3:22)

Equation (3.22) implies that the probability at any point x along the curve is 0. This

may seem puzzli ng, since the density curve has non-zero density for a range of x

values. Note that a line erected at any value x along the curve is considered to be

inﬁnitesimally thin and therefore possesses no area. Therefore, even if the density

f(x) is non-zero at x, the probability or area under the curve at x is exactl y 0, and one

must specify a range of x values to obtain a probability value other than 0.

Using MATLAB

Statistics Toolbox offers the function pdf that uses a speciﬁc probability density

function (pdf) to generate a probability distribution. For example, the MAT LAB

function binopdf compu tes the discrete binomial probability distribution

(Equation (3.12)), i.e. the probability of obtaining k successes in n trials of a binomial

process characterized by a probability of success p in each trial. The syntax for

binopdf is

y = binopdf(x, n, p)

where n and p are the parameters of the binomial distribution and x is the vector of

values or outcomes whose probabilities are desired. Note that x must contain

positive integral values, otherwise a zero value is returned.

To plot the binomial probability distribution for the range of values speciﬁed in x,

use the bar function,

bar(x, y)

Figure 3.8

Density curve.

f(x)

Area under curve = 1

168

Probability and statistics

where x is the vector of outcomes and y co ntains the respective probabilities

(obtained from the pdf function).

The poisspdf function generates the Poisson probability distribution using the

probability mass function speciﬁed by Equation (3.19). Note that a function that

generates probabilities for the entire range of outcomes of a discrete random variable

is called a probability mass function. A density function is deﬁned only for continuous

random variables. Unlike a mass function, a density function must be integrated

over a range of values of the random variable to obtain the corresponding

probability.

The normpdf function calculates the normal density (see Equation (3.23)). The

syntax for normpdf is

y = normpdf(x, mu, sigma)

where mu and sigma are parameters of the normal probability distribution and x is

the vector of values at which the normal density is evaluated.

3.5.2 Normal probability density

The normal probability density is a continuous unimodal function symmetric about

μ, the mean value of the distribution, and is mathematically deﬁned as

fxðÞ¼

ﬃﬃﬃﬃﬃ

2π

ðxμÞ

=2σ

: (3:23)

The mean μ and variance σ

are the only numerical descriptive parameters of this

probability distribution. A normal curve is denoted by the symbol N(μ, σ). The

location of the normal distribution along the x-axis is determined by the parameter

μ, the mean of the distribution. The shape of the distribution – height and width – is

ﬁxed by σ

, the variance of the distribution. The normal distribution is plotted in

Figure 3.9 for three different sets of parameter values. Note how the shape of the

curve ﬂattens with increasing values of σ. The normal density curve extends from –∞

to ∞.

Figure 3.9

Three normal density curves, which include the standard normal probability distribution (μ =0,σ = 1).

−10 −5 0 5 10

0.2

0.4

0.6

0.8

f(x)

μ = 0

σ = 1

μ = −4

σ = 4

μ = 7

σ = 0.5

169

3.5 Normal distribution

We now show that the mean and variance of the normal distribution are indeed

the parameters μ and σ in Equation (3.23). We ﬁrst need to deﬁne the expectation of a

continuous random variable x.Anexpectation or an expected value , E(x), is the

average value obtained from performing many measurements or experiments.

The expected value or expectation of a random variable x that has a continuous probability

distribution f(x) is equal to the mean μ of the distribution and is defined as

ExðÞ¼μ ¼

∞

∞

xf xðÞdx: (3:24)

The expected value of a random variable x that follows a normal distribution N(μ, σ)

is calculated by substituting Equation (3.2 3) into Equation (3.24):

ExðÞ¼

∞

∞

ﬃﬃﬃﬃﬃ

2π

ðxμÞ

=2σ

∞

∞

x  μ þ μ

ﬃﬃﬃﬃﬃ

2π

ðxμÞ

=2σ

∞

∞

ðx  μÞ

ﬃﬃﬃﬃﬃ

2π

ðxμÞ

=2σ

dx þ μ

∞

∞

ﬃﬃﬃﬃﬃ

2π

ðxμÞ

=2σ

dx:

Substituting z = x − μ and

∞

∞

fxðÞdx ¼ 1(Equation (3.21)), we get

ExðÞ¼

∞

∞

ﬃﬃﬃﬃﬃ

2π

z

=2σ

dz þ μ  1:

The ﬁrst integral can be solved using geometric arguments. It consists of an odd

function ðz=

ﬃﬃﬃﬃﬃ

2π

σÞ multiplied by an even function ðe

z

=2σ

Þ, which yields an odd

function. The integral of an odd function over an even interval vanishes. Therefore,

for a normally distributed variable x,

ExðÞ¼μ:

Thus, μ in Equation (3.23) represents the mean of the normal distribution.

For any continuous probability distribution f (x), the variance is the expected value of x  μðÞ

and is

defined as

¼ Eð x  μðÞ

Þ¼

∞

∞

x  μðÞ

fxðÞdx: (3:25)

The variance of a normal distribution N(μ, σ) is calculated by substituting Equation

(3.23) into Equation (3.25):

¼ Eð x  μðÞ

Þ¼

∞

∞

x  μðÞ

ﬃﬃﬃﬃﬃ

2π

ðxμÞ

=2σ

dx:

Let z ¼ x  μðÞ. Then

ﬃﬃﬃﬃﬃ

2π

∞

∞

 e

z

=2σ

The function z

z

=2σ

is an even function. Therefore the interval of integration can

be halved:

170

Probability and statistics

ﬃﬃﬃﬃﬃ

2π

∞

 e

z

=2σ

dz:

Let y ¼ z

=2. Then dy ¼ zdz. The integral becomes

ﬃﬃﬃﬃﬃ

2π

∞

ﬃﬃﬃﬃﬃ

 e

y=σ

dy:

Integrating by parts

uv dy ¼ u

vdy

vdy





, we obtain

Eð x  μðÞ

Þ¼

ﬃﬃﬃﬃﬃ

2π

ﬃﬃﬃﬃﬃ

 e

y=σ

σ





∞



∞

ﬃﬃﬃﬃﬃ

σ



 e

y=σ



The ﬁrst term on the right-hand side is zero. In the second term we substitute z back

into the equation to get

Ex μðÞ



¼ 0 þ 2σ

∞

ﬃﬃﬃﬃﬃ

2π

z

=2σ

dz ¼ 2σ



¼ σ

Thus, the variance of a normal distribution is σ

, and the standard deviation is σ.

3.5.3 Expectations of sample-derived statistics

Now that you are familiar with some commonly used statistical measures such as

mean and variance, and the concept of expected value, we proceed to derive the

expected values of sample-derived statistics. Because the sample is a subset of a

population, the statistical values derived from the sample can only approximate the

true population parameters. Here, we derive the relation between the sample stat-

istical predictions and the true population characteristics. The relationships for the

expected values of statistics obtained from a sample do not assume that the random

variable obeys any particular distribution within the population.

By deﬁnition, the expectation E(x) of a measured variable x obtained from a

single experiment is equal to the average of measurements obtained from many,

many experiments, i.e. Ex

ðÞ¼μ. If we obtain n observations from n independently

performed experiments or n different individuals, then our sample size is n. We can

easily determine the mean



x of the n measurements, which we use as an estimate for

the population mean. If we obtained many samples each of size n, we could measure

the mean of each of the many samples. Each mean of a particular sample would be

slightly different from the means of other samples. If we were to plot the distribution

of the many sample means, we would obtain a frequency plot that would have a

mean value μ



and a variance s



. The expectation of the sampl e mean Eð



xÞ is the

average of many, many such sample means, i.e. μ



;so



xðÞ¼E

x¼1

ðÞ¼

x¼1

μ ¼ μ: (3:26)

Note that because E ðÞis a linear operator, we can interchange the positions of

ðÞ

and E ðÞ. Note also that Eð



xÞ or μ



(the mean of all sample means) is equal to the

population mean μ.

In statistical terminology,



x is an unbiased estimate for μ. A sample-derived statistic r is said to be an

unbiased estimate of the population parameter ρ if E(r)=ρ.

What is the variance of the distribution of the sample means about μ?Ifx is normally

distributed, the sum of i normal variables,

, and the mean of i normal variables,

171

3.5 Normal distribution



x ¼

=n, also follow a normal distribution. Therefore, the distribution of the

means of samples of size n will also be normally distributed. Figure 3.10 shows the

normal distribution of x as well as the distribution of the means of samples of size n.

The standard deviation of the distribution of sample means is always smaller than

the standard deviation of the distribution of the random variable, unless the sample

size n is 1.

The variance of the distribution of sample means is deﬁned as



¼ E



x  μðÞ



Now,



x  μðÞ



¼ E

i¼1

 μ

i¼1

 nμ

i¼1

 μðÞ

The squared term can be expanded using the following rule:

a þ b þ cðÞ

¼ a

þ b

þ c

þ 2ab þ 2ac þ 2bc:

E ð



x  μÞ



i¼1

 μðÞ

i¼1

j¼1; j6¼i

 μðÞx

 μ



The double summation in the second term will count each x

 μðÞx

 μ



term

twice. The order in which the expectations and the summ ations are carried out is

now switched:



x  μðÞ



i¼1

 μðÞ



i¼1

j¼1; j6¼i

 μðÞx

 μ



Figure 3.10

Normal distribution Nðμ; σ ¼ 2Þ of variable x and normal distribution Nðμ



¼ μ; s



¼ σ=

ﬃﬃﬃﬃﬃ

Þ of the means of

samples of size n = 10.

Normally distributed

sample means

= 10)

Normally

distributed

variable x

172

Probability and statistics

Let’s investigate the double summation term:

 μðÞx

 μ



∞

∞

∞

∞

 μðÞx

 μ



ðÞfx



If x

and x

are independent measurements (i.e. the chance of observing x

is not

inﬂuenced by the observation of x

), then

 μðÞx

 μ



∞

∞

 μðÞfx

ðÞdx

∞

∞

 μ



¼ 0 for i 6¼ j:

Therefore,



x  μðÞ



i¼1

 μðÞ



þ 0:

By deﬁnition, Ex

 μðÞ



¼ σ

,and



¼ E



x  μðÞ



: (3:27)

Equation (3.27) states that the variance of the distribution of sample means is equal

to the variance of the population divided by the sample size n. Since our goal is

always to minimize the variance of the distribution of the sample means and there-

fore obtain a precise estimate of the population mean, we should use as large a

sample size as practical.

The standard deviation of the distribution of sample means is



ﬃﬃﬃ

(3:28)

and is called the standard error of the mean or SEM.

The SEM is a very important statistic since it quantiﬁes how precise our estimate



x is for μ, or, in other words, the conﬁdence we have in our estimate for μ.

Many researchers report the error in their experimental results in terms of the

SEM. We will repeatedly use SEM to estimate conﬁdence intervals and perform

tests of signiﬁcance in Chapter 4 . It is important to note that Equation (3.28) is

valid only if the observations are truly independent of each other. Equation

(3.28) holds if the population from which the sampling is done is much larger

than the sample, so that during the random (unbiased) sampling process, the

population from which each observation or individual is chosen has the same

properties.

Let’s brieﬂy review what we have covered in this section so far. The mean of a sample

depends on the observations that comprise the sample and hence will differ for

different samples of the same size. The sample mean



x is the best (unbiased) estimate

of the population mean μ . Also, μ is the mean of the distribution of the sample

means. The variance of the distribution of sample means s



is σ

=n. The standard

deviation of the distribution of sample means, or the standard error of the mean,is

σ=

ﬃﬃﬃ

. If the sample means are normally distributed, then the distribution of sample

means is deﬁned as Nðμ; σ=

ﬃﬃﬃ

Þ.

We are also interested in determining the expected variance Es



of the sample

(not to be confused with the expected variance of the sample mean, s



, which we

173

3.5 Normal distribution

determined above). The sample variance s

is deﬁned by Equation (3.3). We also

make use of the following identity:

i¼1

 μðÞ

i¼1





xðÞ

þn



x  μðÞ

: (3:29)

Equation 3.29 can be easily proven by expanding the terms. For example,

i¼1

ðx





xÞ

i¼1

ðx

 2x



x þ



i¼1

 2



i¼1

þ n



i¼1

 n



Therefore

i¼1

ðx





xÞ

i¼1

ðx





Þ: (3:30)

You should expand the other two terms and pro ve the equality of Equation (3.29).

Substituting Equation (3.29) into Equation (3.3), we get

i¼1

ðx





xÞ

n  1

i¼1

 μðÞ

n



x  μðÞ

n  1

Then, the expected variance of the sample is given by



i¼1

 μðÞ



 nE



x  μðÞ



n  1

From Equat ions (3.25) and (3.27),



i¼1

 n

n  1

nσ

 σ

n  1

¼ σ

Therefore, the expectation of s

is equal to the population variance σ

. Note that

as deﬁned by Equation (3.3) is an unbiased estimate for σ

. As described in

Section 3.2, the term n – 1 in the denominator of Equation (3.3) comes from

using the same sample to calculate both the mean



x and the variance s

Actually, only n – 1 observations provide information about the value of s

One degree of freedom is lost when the data are used to estimate a population

parameter. The sample variance is expected to be less than the population

variance since the sample does not con tain all the observations that make up

the population. Hence the n – 1 term in the denominator adjusts for the inherent

underestimation of the variance that results when a sample is used to approx-

imate the population.

Keep in mind that the following relationships, derived in this secti on, for expected

values:



xðÞ¼μ; Es



¼ σ

; E



x  μðÞ



;

are valid for any form of distribution of the random variable.

174

Probability and statistics

3.5.4 Standard normal distribution and the z statistic

The population density curve is a fundamental tool for calculating the proportion of

a population that exhibits a characteristic of interest. As discussed in Section 3.5.1,

the area under a de nsity curve is exactly equal to 1. If the area under a population

density curve is computed as a function of the observed variable x, then determining

the area under the curve that lies between two values of x will yield the percentage of

the population that exhibits the characteristic within that range. The area under a

normal curve from −∞ to x is the cumulative probability, i.e.

PxðÞ¼

∞

ﬃﬃﬃﬃﬃ

2π

ðxμÞ

=2σ

: (3:31)

If an observation x, such as blood pressure, follo ws a normal distribution, then the

area under the normal curve deﬁning the population distribution from −∞ to x

estimates the fraction of the population that has blood pressure less than x.

Integration of Equation (3.23), the normal equation, does not lend itself to an

analytical solution. Nu merical solutions to Equation (3.31) have been computed

and tabulated for the purpose of calculating probabilities. Generating numerical

solutions and constructing tables for every normal curve is impr actical. Instead, we

simply convert any normal curve to a standardized normal curve.

A normal curve N(0, 1) is called the standard normal distribution and has a probability density given by

fxðÞ¼

ﬃﬃﬃﬃ

2π

z

; (3:32)

where z is a normally distributed variable with μ = 0 and σ =1.

Any variable x that follows a normal distribution can be rescaled as follows to

produce a standard normal distribution using Equation (3.33):

z ¼

x  μ

; (3:33)

this is known as the z statistic and is a measure of the number of standard deviations

between x and the mean μ. Note that z ¼1 represents a distance from the mean

equal to one standard deviation, z ¼2 represents a distance from the mean equal

to two standard deviations, and so on. The area enclosed by the standard normal

curve between z = −1 and z = +1 is 0.6827, as demonstrated in Figure 3.11. This

means that 68.27% of all observations are within ±1 standard deviation from the

mean. Similarly, the area enclosed by the standard normal curve between z = −2 and

z=+2 is 0.9544, or 95.44% of all observations are wi thin ±2 standard deviations

from the mean. Note that 99.74% of the enclosed area under the curve lies between

z=±3.

Using MATLAB

The normpdf function can be used in conjunction with the quad function to

determine the probability of obtaining z within a speciﬁc range of values. The

quad function performs numerical integration and has the syntax

i = quad(func, a, b)

175

3.5 Normal distribution

where func is a function handle, and a and b are the limits of the integration.

Numerical integration is the topic of focus in Chapter 6.

First, we create a function handle to normpdf:

f_z = @(x)normpdf(x, 0 , 1);

The normal distribution parameters are μ =0,σ =1,andx is any vector variable.

The probability of z assuming any value between −1 and +1 is obtained by typing

in the MATLAB Command Window the following:

prob = quad(f_z, -1, 1)

prob =

0.6827

Similarly, the probability of z assuming any value between −2 and +2 is

prob = quad(f_z, -2, 2)

prob =

0.9545

A very useful MATLAB function offered by Statist ical Toolbox is cdf, which

stands for cumulative distribution function. For a continuous distribution, the

cumulative distribution function Φ (cdf) is deﬁned as

Φ yðÞ¼

∞

fxðÞdx ¼ Px yðÞ; (3:34)

where fxðÞis the probabil ity density function (pdf) that speciﬁes the distribution for

the random varia ble x. We are particularly interested in the function normcdf,

which calculates the cumulative probability distribution for a normally distributed

random variable (Equation (3.31)). The syntax is

p = normcdf (x, mu, sigma)

where mu and sigma are the parameters of the normal distribution and x is the

vector of values at which the cumulative probabilities are desired.

To determine the probability of z lying between −1 and +1, we compute

p = normcdf(1, 0, 1) - normcdf(-1, 0, 1)

0.6827

What is the probability of observing z ≥ 2.5? The answer is given by

p=1– normcdf(2.5, 0, 1);

We can also ask the question in reverse – what value of z represents the 95th

percentile? In other words, what is the value of z such that 0.95 of the total area

under curve lies between −∞ and z? The value of z that separates the top 5% of the

population distribution from the remaining 95% is 1.645. A similar question is, “For

an enclosed area of 0.95 symmetrical positioned about the mean, what are the

corresponding values of z?” The answer: the area enclosed by the standardized

normal curve lying between z = ±1.96 contains 95% of the entire population. If

the population parameters are known, we can use Equation (3.33) to calculate the

corresponding value of x, i.e. x ¼ μ þ zσ.

176

Probability and statistics