RAND Corporation. Social Science for Counterterrorism
Подождите немного. Документ загружается.
Representing Social-Science Knowledge Analytically 415
Figure 11.5
A Simple Feedback Diagram
with Feedback
A
B
C
–
RAND MG849-11.5
e second item addresses the common perception that social
phenomena are too complicated to deal with analytically. Imagine
an influence diagram describing the social outcome of an encoun-
ter between two people. Such factors as physical appearance, scent,
motions, verbal comments, facial expressions, and handshake strength
are among the many that could interact. ere could also be feedbacks:
What appeared to be a grimace could cause the other person to back
off, stutter, or weaken the handshake. Or, conversely, an expression of
joy might lead to a rapidly warmer encounter. However, suppose that
these occurred in combination: a grimace (due to being interrupted),
followed by recognition, followed by joy, followed by embarrassment
at being found excessively dressed, followed by. . . . e instantaneous
dynamics might be quite complex, but, after a period of seconds, the
outcome would be determined. Figure 11.6, then, suggests that such
complex feedbacks occurring on a fine time scale may be ignorable on
a coarser time scale.
Thresholds and Ceilings. Representing threshold and ceiling
effects is important. Human beings may slough off risks, for example,
until they reach some level of apparent significance (Slovic, 2000; Shoe-
maker, 1980). is may help explain historical incidents of “unreason-
able” risk-taking, such as that of Saddam Hussein (Davis and Arquilla,
1991; National Research Council, 1996). At the same time, many
effects have a saturation point. Significantly:
416 Social Science for Counterterrorism: Putting the Pieces Together
Figure 11.6
Feedbacks and Time Scale
RAND MG849-11.6
A
B
C
D
A
B
C
D
Fine time scale Coarser time scale
ere is special value in treating thresholds and ceilings in the •
modeling of counterterrorism because they may play a role in the-
ories of victory.
To illustrate, it is probably unnecessary to reduce the materiel and
human-capability components of a terrorist group to zero before effec-
tiveness drops to negligible proportions: We may reasonably hope to
see critical-mass effects. Unless such matters are represented analyti-
cally, we could underestimate the potential value of such tactics as dis-
rupting an organization by targeting its leaders or forcing it to change
operational locations and processes frequently.
It is straightforward to represent the nonlinear phenomena of
thresholds and ceilings. e concepts can be conveyed with simple dia-
grams, such as Figure 11.7. e first curve (solid curve) has a strict
threshold, followed by a linear increase of D with A, followed by a
constant maximum value (ceiling). e alternative (dashed curve) is
generated by using a standard mathematical function called an S-curve
(or sigmoid). e shape can be tuned by adjusting parameters in the
function if precision is needed.
Representing Social-Science Knowledge Analytically 417
Figure 11.7
Thresholds and Ceilings
S-shaped curve
Step-function and linear
D
A
Threshold
Threshold
Ceiling
RAND MG849-11.7
Randomness and the Need for Humility. Even if we have done
a good job identifying factors and combining relationships, social
phenomena will often yield surprises. is may be the consequence
of “hidden variables” (the problem of omitted-variable bias in econo-
metrics), which might or might not be knowable in advance. Such
variables include the health and mood of protagonists, perceptions
about exogenous events in the world, and the order of events. Some
events are truly random, however, and humans display some inherent
inconsistency—as studied by social psychologists. ere is something
amusing in the way critics of the intelligence community expect a
higher degree of omniscience about events in faraway lands than exists
in our own nation when we try to project the winner of an upcoming
election months in advance. History should also be sobering. Prob-
ably no crisis has been better studied than the Cuban Missile Crisis of
1963, but research is still uncovering material that demonstrates how a
number of microscopic events could easily have changed the outcome,
perhaps unleashing a war that no one wanted (Dobbs, 2008).
418 Social Science for Counterterrorism: Putting the Pieces Together
How should randomness be handled in CT research? e first
principle is humility: We should aspire to estimating the odds of being
correct rather than getting predictions “right.” Analytically, we can
add explicit random variables just to remind us constantly of uncer-
tainty, which can work either positively or negatively (Davis, Bankes,
and Egner, 2007). Alternative important approaches, not discussed in
this paper, involve Bayesian nets or influence nets—methods for which
have evolved substantially over the last decade or so.
16
Figure 11.8 illustrates representation of the randomness issue. In
this, factors A, B, and C affect outcome D, but so also do factors that
have not been explicitly identified, called “other.” e presence of those
factors might either increase or decrease the likelihood of the outcome
D (hence the +/-). e heaviness of the arrows indicates that A is espe-
cially important, that B and C are less so, and hidden variables even
less so. Much can be conveyed with diagrams with only this level of
complexity.
ere are limits to what can be expressed diagrammatically with-
out excessive complication. e next step, arguably, is to use simple
outcome tables. Table 11.3 is intended to reflect more precisely the
same thinking as in Figure 11.8. Factor A is especially important; if
it is Low, then the claim is that outcome D will be Low. If factor A
is High, then outcome D will probably be High if at least one of B
and C are High, and very likely be High if both are High. A modeler
Figure 11.8
A Combining Logic Diagram with Hidden Factors
RAND MG849-11.8
A B C Other
D
or
and
±
Representing Social-Science Knowledge Analytically 419
Table 11.3
A Simple Outcome Table, Consistent with Figure 11.8
But More Precise
A B C D Confidence
High High High High High
High High Low High Moderate
High Low High High Moderate
High Low Low Low High
Low High High Low High
Low High Low Low High
Low Low High Low High
Low Low Low Low High
familiar with expert judgments could give these a bit more precision,
such as associating “moderate” with ~60 percent and “high” with ~80
percent in probability terms, or odds of, say, 3:2 and 4:1. Subject-matter
experts could ponder about whether those would be “about right.”
e continuing point is that simple representations of knowledge
may have a degree of imprecision consistent with the knowledge itself.
Even if the factors must be allowed more discrete values (say three, as
in Low, Marginal, High), experience in developing artificial-intelligence
models using highly structured rules demonstrates that sophisti-
cated but comprehensible models can be built using these techniques
(see also the appendix). With straightforward mathematical tech-
niques and appropriate spot-checking by human experts, much can
also be done to verify and even validate—relative to subjective expert
knowledge.
e logic table (Table 11.3) is equivalent to the mathematics,
described in pseudo code as:
If A is High and (B is High and C is High)
en: D is High; Confidence is High
Else
If A is High and (B is High or C is High)
420 Social Science for Counterterrorism: Putting the Pieces Together
en: D is High; Confidence is Moderate
Else: D is Low; Confidence is High.
In this case, the pseudo-code is as good as or better than the table, but
communication is usually better with table structures—especially as
dimensionality increases.
The Dynamics of Competition, Learning, and Adaptation. Com-
petition, learning, and adaptation are crucial in social phenomena, but
representing them is not always easy. Competition can in some cases be
represented by game-theoretic methods. In the simplest form, these do
not purport to describe the actual dynamics of interaction but rather
to show what outcomes would be like if competitors act most effec-
tively in their own interest. is can be done sequentially.
17
Modern
game theory includes cooperation and competition and can include
agent-based modeling (for example, Axelrod, 1997; Epstein, 2007), as
discussed below.
Agent-based modeling (ABM) is closely associated with the study
of complex adaptive systems (CAS) generally (for example, Holland
and Minmaugh, 1998; Epstein and Axtell, 1996; Epstein, 2006; Bar-
Yam, 2003; Gilbert, 2007). ABM can be included in any of a number
of CT simulation environments, such as REPAST (North and Macal,
2007), SEAS (Chatuverdi et al., 2000), and COMPOEX.
18
As described elsewhere (Davis, 2005), entity-level ABM research
should be very helpful in developing lower-resolution models useful
in policy analysis. It may not be fruitful in policy work to follow the
dynamics of complex adaptive systems to see the details of precisely
how “emergent phenomena” such as insurgencies arise. at may be
left for separate research, with the fruits of the research being reflected
in simpler models identifying when situations should be expected to be
unstable. It will probably prove necessary to have much more sophisti-
cated agents and representations of the environment to develop a good
microscopic understanding of phenomena, including emergence.
19
In
any case, making the connection between the worlds of micromodels
and macromodels is both exciting and challenging.
20
Some researchers believe that detailed agent-based simulations
can be used predicatively. ey sometimes disparage the feasibility
Representing Social-Science Knowledge Analytically 421
of social-science modeling that does not include agent-based model-
ing. Such researchers have far more faith in the validity of the current
ABMs than I do and far less confidence that the consequences of the
various “emergent phenomena” can be represented macroscopically. It
is an interesting theoretical debate that will be resolved over time with
experience.
Other classes of model may also be important, such as models
describing the consequences of a conservation law or of aspects of a
system that are constant after a steady state has been reached. Some of
these types are familiar in the physical sciences, economics, and other
social-science disciplines. I mention them because the common use of
computer simulations sometimes crowds out simpler depictions.
A Vision of Analysis Amidst Uncertainty
e preceding sections have emphasized the special difficulties asso-
ciated with uncertainty and soft, qualitative knowledge as occurs in
social science. It follows from an appreciation of these that
e objective of analysis in social science should often not be reli-•
able “prediction,” but rather an understanding of possibilities and
perhaps of rough probabilities, or odds.
is admonition applies to much policy analysis generally. e
Department of Defense has come increasingly to recognize that mas-
sive uncertainty exists about such fundamental issues as who will be
the future adversaries of the United States, where and under what cir-
cumstances future wars will occur, and what strategies and tactics will
be employed. e result has been an emphasis on capabilities-based
planning (Rumsfeld, 2001).
Much has been done to develop uncertainty-sensitive methods of
analysis (Davis, 2002a; Davis, Shaver, and Beck, 2008) with applica-
tions to defense. Similar methods have been brought to bear on policy
debates about climate change and regional natural-resource planning
(Lempert, Popper, and Bankes, 2003; Lempert, Groves, Popper, and
422 Social Science for Counterterrorism: Putting the Pieces Together
Bankes, 2006). e philosophy refl ected in these eff orts is consistent
with what has become a key element in strategic planning: prepar-
ing for adaptiveness (Davis, 2002a; Light, 2004; Alberts and Hayes,
2003).
21
All of these applications relate to people and organizations,
that is, to social phenomena.
22
A key element of these approaches is exploratory analysis, as illus-
trated in Figure 11.9. Consider fi rst the fl ow along the top of the fi gure.
Given some alternative packages of strategies, tactics, and investments,
which packages we can call Options 1, 2, 3, and 4, we seek to assess
them despite extraordinary uncertainty about “everything” (see the
assumption classes at the bottom of the fi gure). To do this, we develop
an experimental plan that systematically varies the assumptions—even
assumptions about the functional form of the model being used for
evaluation! e experimental plan then drives computational experi-
ments. In each experiment, the model (an “engine” for generating cases)
has a set of inputs and produces outputs (which may be stochastic). e
plan may call for huge numbers of runs, but actually conducting the
Figure 11.9
The Vision of Analysis
RAND MG849-11.9
Experimental plan
varying assumptions
Strategies, tactics,
investments
Costs
(economic, other)
Insights about
potential effects
on terrorism,
uncertainties,
costs
Portfolio-analysis
decision aids
t4DPSFDBSET
t$PTUFGGFDUJWFOFTT
plots by
“perspective”
Engine for exploratory
analysis
Integrate,
summarize
holistically
Assumptions varied:
t Contexts
t0CKFDUJWFTTUSBUFHJFTBOEUBDUJDTPG
protagonists (including third parties)
t$BQBCJMJUJFT
t1FSGPSNBODFPGTZTUFNTBOEPSHBOJ[BUJPOT
t$PTUT
t&MFNFOUTPGVOEFSMZJOHNPEFMT
t+VEHNFOUTBCPVUJNQPSUBODFPGEJGGFSFOU
criteria
...
Simulate CT system
behavior
0QUJPO
0QUJPO
0QUJPO
0QUJPO
Nomi-
nal
Annual
Cost
Down-
side
X
9
9
9
Up-
side
Net assessment*
*Based on a particular “perspective” of how to
average value over multiple criteria.
Effects on problem
$PNQPOFOUTUP
Representing Social-Science Knowledge Analytically 423
runs is a “mechanical” matter behind the scenes. e results, then,
must be analyzed like “data” to see if patterns emerge. e results can
be integrated and summarized for comprehensible displays such as
the colored scorecards used in RAND’s approach to portfolio analy-
sis (bottom right). Analysts examine the results in a myriad of ways
using such methods as standard statistical regressions, motivated meta-
models, and data mining (Davis, Bankes, and Egner, 2007; Lempert,
Groves, Popper, and Bankes, 2006).
is “exploratory analysis” approach represents a very different
paradigm than starting with a baseline scenario, simulating the conse-
quences for a few alternative strategies, and then conducing a handful
of excursions with different assumptions.
e fundamental concept in such work is to find strategies that •
are likely do well across much of the uncertainty space rather than
performing well only for best-estimate assumptions.
e approach seeks “FAR strategies,” that is, strategies that are Flexible,
Adaptive, and Robust (see also National Research Council, 2006).
Some traditional analysts view such an image with horror because
they are used to spending months working out details of a baseline
model and database. How could they be asked to work on an entire
scenario space? Fortunately, in exploratory analysis the premium is on
achieving a synoptic view rather than precision. Simpler models suf-
fice for initial work, so that running huge numbers of cases may be
relatively straightforward, occurring behind the scenes. e fruits of
exploratory analysis can be shown in displays that identify the circum-
stances in which outcomes are, for example, favorable or unfavorable,
and where the boundary lines lie, that is, defining different regions.
e intellectual content has to do with learning how many impor-
tantly different regions exist and where they lie in the n-dimensional
assumptions space (also called factor space, scenario space, and param-
eter space).
In some cases, the regions can be identified by clever analysts
without much computation, in which case it is even easier to identify a
small “spanning set” of analytical cases, one or two for each important
424 Social Science for Counterterrorism: Putting the Pieces Together
region. In assessing alternative courses of action, it is then necessary
to test only against the spanning-set cases because they “stress” the
alternative in all of the most important ways (Davis, Shaver, and Beck,
2008; National Research Council, 2008; Davis, Johnson, Long, and
Gompert, 2008). Under budget pressures, policymakers could decide
to deemphasize some of the case, but they would do so with recogni-
tion of the risks.
Once the synoptic view has been accomplished with low-resolution
exploratory analysis, detail can be added selectively to better under-
stand implications. is process is most compelling and rigorous when
combined with multiresolution modeling.
is vision is ambitious, but 15–20 years of experience now
exists with exploratory analysis, which has proven its viability and
usefulness—assuming sufficient relevant knowledge (even if uncer-
tain).
Illustrative Application to Integrating Social-Science
Knowledge
e previous sections have described generic methods for communicat-
ing social-science knowledge. What follows illustrates how they can be
used.
Companion papers in the larger study of which this paper is part
(Davis and Cragin, 2009) reviewed the social-science literature on
terrorism and counterterrorism with regard to root causes (Noricks,
2009), individual radicalization (Helmus, 2009), achieving and main-
taining public support (Paul, 2009), and how terrorist organizations
make decisions (Jackson, 2009). Each included a factor tree relating
the factors identified in the respective reviews. ese are reproduced
here as Figures 11.10 through 11.13.
23
Noricks’s root-causes tree (Figure 11.10) has three approximately
necessary conditions for overall root-cause pressures to be significant:
norms that tolerate violence, perceived grievances (for example, foreign
occupation), and the mobilizing structures for terrorism. ese top-