Vidakovic B. Statistics for Bioengineering Sciences: With Matlab and WinBugs Support

Подождите немного. Документ загружается.

3.10 Exercises 105

G condition

0.64 0.36 D

0.21 0.79 D

I condition

0.49 0.51 D

0.52 0.48 D

B condition

0.55 0.45 O

0.58 0.42 O

0.40 0.60

O D

0.49 0.51 O D

Suppose a female subject, older than 21, of Pima Indian heritage living

near Phoenix, AZ, was selected at random. Using WinBUGS, approximate

the probabilities (a)

P(O|I), P(B|O

,G), and P(G|B, A

3.32. A Simpliﬁed Probabilistic Model of Visual Pathway. Our nervous

system consists of specialized cells called neurons that are connected to

one another in highly organized and speciﬁc ways. A neuron changes its

membrane voltage in response to an external stimulus or a signal from an

upstream neuron. For example, in the visual pathway, light excites photore-

ceptors in our eyes that are connected to more downstream neurons, and

the activities of this neuronal network eventually elicit visual perceptions.

The following diagram illustrates a simpliﬁed version of this connection (P

= photoreceptor cell; B = bipolar cell; R = retinal ganglion cell; L = lateral

geniculate nucleus ganglion cell, V = primary visual cortex simple cell):

Fig. 3.21 A simpliﬁed probabilistic model of visual pathway.

Let events S and S

denote, respectively, the presence and absence of a

light stimulus in a small time interval

∆t. Let events P,B,R, and L mean

that the corresponding cells produce a response (ﬁre) in the time interval

∆t. Suppose that the conditional probabilities of ﬁring for three parallel

branches i

=1,2, and 3 are the same:

P(S) =0.6

P(P

|S) =0.95 P(P

) =0.1

P(B

) =0.95 P(B

) =0.05

P(R

) =0.9 P(R

) =0.02

P(L

) =0.8 P(L

) =0.08

=1,2,3

In the schematic diagram in Fig. 3.21, three L ganglion cells, L

, L

, and

, connect to one simple cell in the primary visual cortex, V . Assume that

106 3 Probability, Conditional Probability, and Bayes’ Rule

input from at least one of the L cells is needed for the V simple cell to

respond with certainty. The V simple cell will not respond if the input there

from L cells is absent.

The following questions may need the support of WinBUGS.

(a) What is the probability of the V cell responding given that event S (light

stimulus present) has occurred?

(b) Assume that L

and R

have ﬁred. What is the probability of S?

MATLAB AND WINBUGS FILES USED IN THIS CHAPTER

http://springer.bme.gatech.edu/Ch3.Prob/

birthday.m, bridge.m, circuit.m, ComplexCircuit.m, demere.m, die.m,

inclusionexclusion.m, mistery.m, rollingdie1.m, rollingdie2.m,

sheriff.m, twoheaded.m,

alarm.odc, asia.odc, circuit1.odc, DeMere.odc,

misterioustransfers.odc, pima.odc, sprinkler.odc,

CHAPTER REFERENCES

Adel, A. L., Dorr, R. T., Liddil, J. D. (1993). The effect of anticancer drug sequence in experi-

mental combination chemotherapy. Cancer Invest., 11, 1, 15–24.

Barbeau, E. (1993). The problem of the car and goats. College Math. J., 24, 2, 149–154.

Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances. Philos.

Trans. R. Soc. Lond., 53, 370–418.

Bonferroni, C. E. (1937). Teoria statistica delle classi e calcolo delle probabilita. In Volume

in Onore di Ricarrdo dalla Volta, Universita di Firenza, 1–62.

Casscells, W., Schoenberger, A., and Grayboys, T. (1978). Interpretation by physicians of

clinical laboratory results. New Engl. J. Med., 299, 999–1000.

Gillman, L. (1992). The car and the goats, Am. Math. Mon., 99, 1, 3–7.

Kolmogorov, A. N. (1933). Grundbegriffe der Wahrscheinlichkeitsrechnung, Springer, Berlin.

Laplace P. S. (1774). Mémoire sur la probabilité des causes par les évenements. Mém. de

l’Ac. R. des Sciences de Paris, 6, 621–656.

Lauritzen, S. L. and Spiegelhalter, D. L. (1988). Local computations with probabilities on

graphical structures and their application to expert systems. J. R. Stat. Soc. B, 50,

157–194.

Chapter References 107

Piaget, J. and Inhelder B. (1976). The Origin of the Idea of Chance in Children. W. W. Norton

& Comp., NY, 276 pp.

Ross, S. (2009). A First Course in Probability, 8th Edition. Prentice Hall.

Selvin, S. (1975). A Problem in Probability. Am. Stat., 29, 1, 67.

Venn, J. (1880). On the employment of geometrical diagrams for the sensible representation

of logical propositions. Proc. Cambridge Philos. Soc., 4, 47–59.

Wong, S. K. N., O’Connell, M., Wisdom, J. A., and Dai, H. (2005). Carbon nanotubes as

multifunctional biological transporters and near-infrared agents for selective cancer

cell destruction. Proc. Natl. Acad. Sci., 102, 11600–11605.

Chapter 4

Sensitivity, Speciﬁcity, and Relatives

Poetry teaches us music, metaphor, condensation and speciﬁcity.

– Walter Mosley

WHAT IS COVERED IN THIS CHAPTER

• Deﬁnitions of Sensitivity, Speciﬁcity, Positive and Negative Predic-

tive Values, Likelihood ratio Positive and Negative, Measure of Agree-

ment.

• Combining Tests

• Performance of Tests: ROC Curves, Area Under ROC, Youden Index

• Two Examples: D-dimer and ADA

4.1 Introduction

This chapter introduces several notions fundamental for disease or device test-

ing. The sensitivity, speciﬁcity, and positive and negative predictive values of

a test are measures of the performance of a diagnostic test and are intimately

connected with probability calculations (estimations) and Bayes’ rule.

Although concepts such as “false positives,” “true negatives,” etc. are quite

intuitive, many students and even health professionals have difﬁculties in as-

sessing the associated probabilities. The following problem was posed by Cass-

cells et al. (1978) to 60 students and staff at an elite medical school: If a test

B. Vidakovic, Statistics for Bioengineering Sciences: With MATLAB and WinBUGS Support, 109

Springer Texts in Statistics, DOI 10.1007/978-1-4614-0394-4_4,

110 4 Sensitivity, Speciﬁcity, and Relatives

to detect a disease whose prevalence is 1/1000 has a false positive rate of 5%,

what is the chance that a person found to have a positive result actually has

the disease, assuming you know nothing about the person’s symptoms or signs?

Assuming that the probability of a positive result given the disease is 1,

the answer to this problem is approximately 2%. Casscells et al. found that

only 18% of participants gave this answer. The most frequent response was

95%, presumably on the supposition that, because the error rate of the test is

5%, it must get 95% of results correct.

Examples of misconceptions of test precision measures, especially involv-

ing the sensitivity and positive predictive value, are abundant.

When a multiplicity of tests are possible and a researcher is to select the

“best” test, the receiver operating characteristic (ROC) curve methodology is

used. This methodology is especially useful in setting a threshold that sepa-

rates positive and negative outcomes of a test.

4.2 Notation

Suppose that n subjects are selected randomly from a given population. The

population may not be very general; it could be a speciﬁc segment of subjects

(patients in a hospital who were checked in 10+ days ago, subjects with a his-

tory of heart attack, etc.). Now suppose that the true disease status (disease

present/absent) in all subjects is determined via a gold-standard assessment

and that we are interested in evaluating a particular test for the disease as-

suming that the gold-standard results are always true. For example, in testing

for breast cancer (BC), a mammogram is used as a test while the battery of

numerous patient symptoms, medical history, and biopsy results are used as

a gold standard.

A positive test would not necessarily mean the disease is present but rather

would mean that the test says the disease is present. For instance, a patient’s

mammogram appears to show breast cancer. A true-positive test result not

only means that the test says the disease is present, but that the disease really

is present. In this case, a positive mammogram of a patient for which the gold

standard indicates BC would be a true positive. In the same context, false

positives, true negatives, and false negatives are deﬁned in an obvious manner.

By classifying the patients with respect to the test results and the true

disease status, the following table can be constructed:

Disease (D) No disease (C) Total

Test positive (P) TP FP nP = TP + FP

Test negative (N) FN TN nN = FN + TN

Total nD = TP + FN nC = FP + TN n = nD + nC = nP + nN

4.2 Notation 111

where

TP True positive (test positive, disease present)

FP False positive (test positive, disease absent)

FN False negative (test negative, disease present)

TN True negative (test negative, disease absent)

nP Total number of positives (TP + FP)

nN Total number of negatives (TN + FN)

nD Total number with disease present (TP + FN)

nC Total number without disease present (TN + FP)

n Total sample size (TP + FP + FN + TN)

The test’s effectiveness is measured by the number of true positives and true

negatives relative to the total number of cases. It turns out that these two

numbers are used to deﬁne two complementary measures of test performance:

sensitivity and speciﬁcity.

We will illustrate the deﬁned numbers with the following example.

Example 4.1. BreastScreen Victoria. This 1994 study involved women who

participated in the BreastScreen Victoria initiative in Victoria, Australia,

where free biennial screenings for BC are provided to women aged 40 and

older. The data provided by Kavanagh at al. (2000) show that among 96,420

asymptomatic women, 5,401 had positive and 91,019 negative mammogram

results. The mammograms were read independently by two radiologists. In

the case of disagreement over whether to recall, a consensus was reached or

a third reader made the decision. The women were then recommended for

routine rescreen or referred for assessment. Assessment might include clini-

cal examination, further radiographs, ultrasound, or biopsy. After assessment,

women may have a cancer diagnosed, be recommended for routine rescreen-

ing, or be recommended for further assessment (early review). Out of 96,420

women, 665 were diagnosed with BC. Of those 665 diagnosed, the mammo-

gram was positive for 495 and negative for 160 women. The table summarizing

the data is below.

BC diagnosed BC not diagnosed Total

Mammogram positive 495 4906 5401

Mammogram negative 160 90859 91019

Total 665 95765 96420

Sensitivity is the ratio of the number of true positives and the number of

subjects with a disease, while speciﬁcity is the ratio of the number of true

negatives and the number of subjects without the disease. In the BC example

sensitivity is 495/665 = 75.57%, and speciﬁcity is 90859/95765 = 94.88%.

Both measures have to be reported since reporting only sensitivity or only

speciﬁcity reveals little information about the test. There are two extreme

112 4 Sensitivity, Speciﬁcity, and Relatives

cases. Imagine a test that classiﬁes all subjects as positive – trivially the sen-

sitivity is 100%. Since there are no negatives, the speciﬁcity is zero. Likewise,

a test that classiﬁes all subjects as negative has a speciﬁcity of 100% and zero

sensitivity.

The following table summarizes the key notions:

Sensitivity (Se) Se = TP/(TP + FN) = TP/nD

Speciﬁcity (Sp) Sp = TN/(FP + TN) = TN/nC

Prevalence (Pre) (TP + FN)/(TP + FP + FN + TN)= nD/n

Positive predictive value (PPV) PPV = TP/(TP + FP) = TP/nP

Negative predictive value (NPV) NPV = TN/(TN + FN) = TN/nN

Likelihood ratio positive (LRP) LRP = Se/(1-Sp)

Likelihood ratio negative (LRN) LRN = (1-Se)/Sp

Apparent prevalence (APre) APre = nP/n

Concordance, agreement (Ag) Ag =(TP + TN)/n

The population prevalence of a disease is deﬁned as the probability that a

randomly selected person from this population will have the disease. As the

table shows, the prevalence is estimated by (TP + FN)/(TP + FP + FN +TN) =

nD/n. For the Victoria BC data the prevalence is 665/96420

= 0.0069. This is

a valid estimator only if the table is a summary of a representative sample of

the population under analysis. In other words, the sample should have been

taken at random and the tabulation made subsequently. This is not the case

in many studies. The prevalence of some diseases in a general population is

often so small that insisting on a random sample would require huge sample

sizes in order to obtain a nonzero TP or FN table entries. When the table is

made from available cases and controls (convenience samples), the prevalence

for the population cannot be estimated from it.

Related quantity is the incidence of a disease in a population. It is deﬁned

as the probability that a randomly selected person from the subset of people

not affected by the disease will develop the disease in a ﬁxed time window

(week, month, year). While the prevalence relates to the magnitude, the inci-

dence provides information about the progression and dynamics of the disease.

One of the most important measures is the positive predictive value (PPV).

Based on the table it can be estimated as the proportion of true positives

among all positives, TP/nP. This is correct only if the population prevalence

is well estimated by nD/n, that is, if the table is representative of its popula-

tion. This is approximately the case for the Victoria BC data; the PPV is well

estimated by 495/5401

=0.0916.

If the table is constructed from a convenience sample, the prevalence (Pre)

would be external information, and the PPV is calculated as

PPV

Se ×Pre

Se ×Pre +(1 −Sp) ×(1 −Pre)

4.2 Notation 113

This is simply the Bayes rule and will be discussed more in the next section.

Why is the PPV so important? Imagine an almost perfect test for a particu-

lar disease, with a sensitivity of 100% and speciﬁcity of 99%. If the prevalence

of the disease in the population is 10%, then among ten positives there would

be approx. one false positive. However, if the population prevalence is 1/10000,

then for each true positive there would be approx. 100 false positives.

The likelihood ratio positive, or Bayes factor positive, represents the odds

that a positive test result would be found in a patient with, vs. without, a

disease. The likelihood ratio negative, or Bayes factor negative, represents the

odds that a negative test result would be found in a patient without, vs. with,

a disease. For example:

Posttest disease odds

LRP

pretest disease odds;

Posttest no-disease odds

= LRN ×pretest no-disease odds.

4.2.1 Conditional Probability Notation

The deﬁnitions introduced in the previous section depend on the relative fre-

quencies in the observed tables, and they are empirical. The theoretical coun-

terparts are expressed in terms of probabilities. The analogy to this is the

interplay of the probability of an event A,

P(A), which is theoretical, and rela-

tive frequency of the event n

/n, which is empirical.

Let T be the event that a subject tests positive and D, D

the hypothesis

that the subject does/does not have the disease. The sensitivity is the condi-

tional probability

P(T|D) =P(T ∩D)/P(D) [which is estimated by (TP/n)/(nD/n)

= TP/nD]. Analogously, the speciﬁcity is

P(T

) = P(T

∩D

)/P(D

), which

is estimated by (TN/n)/(nC/n) = TN/nC. We have argued that

P(D), the preva-

lence, cannot be estimated from the table unless the sample forming the table

is representative of the population. In the case of “convenience” samples, the

prevalence is evaluated separately or assumed known from other studies. If

the sample is randomly obtained from the population, the prevalence can be

estimated by nD/n. The probability of a positive test is in fact given by the rule

of total probability,

P(T) =P(T|D)P(D)+P(T|D

)P(D

). Note that P(T) depends

on the prevalence and is estimated by nP/n for a table from a random sample.

Finally, the PPV and NPV are determined by Bayes’ rule. For example, the

PPV is

P(D|T) =

(T|D)P(D)

P(T)

(T|D)P(D)

P(T|D)P(D) +P(T|D

)P(D

)

The posttest disease odds ratio is LRP times the pretest odds ratio. Because of

this property, the LRP is in fact Bayes’ factor in the terminology of Chaps. 3

114 4 Sensitivity, Speciﬁcity, and Relatives

and 8.

P(D|T)

P(D

|T)

(T|D)P(D)

P(T)

(T|D

)P(D

)

P(T)

(T|D)P(D)

P(T|D

)P(D

)

(T|D)

P(T|D

)

(D)

P(D

)

Thus, posterior disease odds = LRP

× prior disease odds.

The above deﬁnitions are illustrated on an example where researchers

tested for acute pulmonary embolism.

Example 4.2. D-Dimer. When a vein or artery is injured and begins to leak

blood, a sequence of clotting steps and factors (called the coagulation cascade)

are activated by the body to limit the bleeding and create a blood clot to plug

the hole. During this process, threads of a protein called ﬁbrin are produced.

These threads are cross-linked (chemically glued together) to form a ﬁbrin net

that catches platelets and helps hold the forming blood clot together at the

site of the injury. Once the area has had time to heal, the body uses a protein

called plasmin to break the clot (thrombus) into small pieces so that it can be

removed. The fragments of the disintegrating ﬁbrin in the clot are called ﬁb-

rin degradation products (FDPs). One of the FDPs produced is D-dimer, which

consists of variously sized pieces of cross-linked ﬁbrin. D-dimer is normally un-

detectable in the blood and is produced only after a clot has formed and is in

the process of being broken down. Measurement of D-dimer can indicate prob-

lems in the body’s clotting mechanisms. The data below consist of quantitative

plasma D-dimer levels among patients undergoing pulmonary angiography for

suspected pulmonary embolism (PE). The patients who exceed the threshold of

500 ng/mL are classiﬁed as positive for a PE. The gold standard for PE is the

pulmonary angiogram. Goldhaber et al. (1993), from Brigham and Women’s

Hospital at Harvard Medical School, considered a population of patients who

are suspected of PE based on a battery of symptoms. The summarized data for

173 patients are provided in the table below.

Acute PE No PE present Total

Test positive (D-dimer ≥ 500 ng/mL) 42 96 138

Test negative (D-dimer < 500 ng/mL) 3 32 35

Total 45 128 173

A simple MATLAB ﬁle sesp.m will calculate the sensitivity, speciﬁcity,

prevalence, positive and negative predictive values, and degree of agreement

between the test and gold-standard results.

function [se sp pre ppv npv ag] = sesp(tp, fp, fn, tn)

%D-dimer as a test for acute PE (Goldhaber et al, 1993)

% [s1, s2, p1, p2, p3, a, yi] = sesp(42,96,3,32)

n = tp+tn+fn+fp; %total sample size

np = tp + fp; %total positive

nn = tn + fn; %total negative

4.3 Combining Two or More Tests 115

nd = tp + fn; %total with disease

nc = tn + fp; %total control (without disease)

%--------------

se = tp/nd; %tp/(tp + fn):::sensitivity

sp = tn/nc; %tn/(tn + fp):::specificity

pre = nd/n; %(tp + fn)/(tp+tn+fn+fp):::prevalence

%only in for case when sample is random from the

% population of interest. Otherwise, the prevalence

% needed for calculating PPV and NPV is an input value

ppv = tp/np; %tp/(tp + fp):::positive predictive value

npv = tn/nn; %tn/(tn+fn):::negative predictive value

lrp = se/(1-sp); %:::likelihood ratio positive

lrn = (1-se)/sp; %:::likelihood ratio negative

ag = (tp+tn)/n; %:::agreement

yi = (se + sp - 1)/sqrt(2); %:::youden index

%---------------

disp(’ Se Sp Pre PPV NPV LRP Ag Yi’)

disp([se, sp, pre, ppv, npv, lrp, ag yi])

%spacing in disp depends on the font size.

For the D-dimer data, the result is

[a b c d e f g] = sesp(42,96,3,32);

Se Sp Pre PPV NPV LRP Ag Yi

0.9333 0.2500 0.2601 0.3043 0.9143 1.2444 0.4277 0.1296

Goldhaber et al. (1993) conclude: “The results of our study indicate that

quantitative plasma D-dimer levels can be useful in screening patients with

suspected PE who require pulmonary angiography. Plasma D-dimer values

less than 500 ng/mL may obviate the need for pulmonary angiography, partic-

ularly among medical patients for whom the clinical suspicion of PE is low. The

plasma D-dimer value, assayed using a commercially available enzyme-linked

immunosorbent assay kit, is a sensitive but nonspeciﬁc test for the presence

of acute PE.”

4.3 Combining Two or More Tests

Suppose that k independent tests for a particular condition are available and

that their sensitivities and speciﬁcities are Se

, Sp

, Se

, Sp

, ..., Se

, Sp

. If

these tests could be combined, what would be the sensitivity/speciﬁcity of the

combined test?

First, it is important to deﬁne how the tests are going to be combined.

There are two main strategies: parallel and serial. When the tests are as-

sumed independent, the calculations are similar to those in circuit problems

from Chap. 3, p. 68. Denote by Se and Sp the sensitivity and speciﬁcity of the

combined test, respectively.