Assessing the Benefits and Costs of Homeland Security Research: A Risk-Informed Methodology with Applications for the U.S. Coast Guard.

This article describes a methodology for risk-informed benefit-cost analyses of homeland security research products. The methodology is field-tested with 10 research products developed for the U.S. Coast Guard. Risk-informed benefit-cost analysis is a tool for risk management that integrates elements of risk analysis, decision analysis, and benefit-cost analysis. The cost analysis methodology includes a full-cost accounting of research projects, starting with initial fundamental research costs and extending to the costs of implementation of the research products and, where applicable, training, maintenance, and upgrade costs. The benefits analysis methodology is driven by changes in costs and risks leading to five alternative models: cost savings at the same level of security, increased security at the same cost, signal detection improvements, risk reduction by deterrence, and value of information. The U.S. Coast Guard staff selected 10 research projects to test and generalize the methodology. Examples include tools to improve the detection of explosives, reduce the costs of harbor patrols, and provide better predictions of hurricane wind speeds and floods. Benefits models and estimates varied by research project and many input parameters of the benefit estimates were highly uncertain, so risk analysis for sensitivity testing and simulation was important. Aggregating across the 10 research products, we found an overall median net present value of about $385 million, with a range from $54 million (5th percentile) to $877 million (95th percentile). Lessons learned are provided for future applications.

Defender-Attacker Decision Tree Analysis to Combat Terrorism.

We propose a methodology, called defender-attacker decision tree analysis, to evaluate defensive actions against terrorist attacks in a dynamic and hostile environment. Like most game-theoretic formulations of this problem, we assume that the defenders act rationally by maximizing their expected utility or minimizing their expected costs. However, we do not assume that attackers maximize their expected utilities. Instead, we encode the defender's limited knowledge about the attacker's motivations and capabilities as a conditional probability distribution over the attacker's decisions. We apply this methodology to the problem of defending against possible terrorist attacks on commercial airplanes, using one of three weapons: infrared-guided MANPADS (man-portable air defense systems), laser-guided MANPADS, or visually targeted RPGs (rocket propelled grenades). We also evaluate three countermeasures against these weapons: DIRCMs (directional infrared countermeasures), perimeter control around the airport, and hardening airplanes. The model includes deterrence effects, the effectiveness of the countermeasures, and the substitution of weapons and targets once a specific countermeasure is selected. It also includes a second stage of defensive decisions after an attack occurs. Key findings are: (1) due to the high cost of the countermeasures, not implementing countermeasures is the preferred defensive alternative for a large range of parameters; (2) if the probability of an attack and the associated consequences are large, a combination of DIRCMs and ground perimeter control are preferred over any single countermeasure.

Cognitive and Motivational Biases in Decision and Risk Analysis.

Behavioral decision research has demonstrated that judgments and decisions of ordinary people and experts are subject to numerous biases. Decision and risk analysis were designed to improve judgments and decisions and to overcome many of these biases. However, when eliciting model components and parameters from decisionmakers or experts, analysts often face the very biases they are trying to help overcome. When these inputs are biased they can seriously reduce the quality of the model and resulting analysis. Some of these biases are due to faulty cognitive processes; some are due to motivations for preferred analysis outcomes. This article identifies the cognitive and motivational biases that are relevant for decision and risk analysis because they can distort analysis inputs and are difficult to correct. We also review and provide guidance about the existing debiasing techniques to overcome these biases. In addition, we describe some biases that are less relevant because they can be corrected by using logic or decomposing the elicitation task. We conclude the article with an agenda for future research.

Dealing with femtorisks in international relations.

The contemporary global community is increasingly interdependent and confronted with systemic risks posed by the actions and interactions of actors existing beneath the level of formal institutions, often operating outside effective governance structures. Frequently, these actors are human agents, such as rogue traders or aggressive financial innovators, terrorists, groups of dissidents, or unauthorized sources of sensitive or secret information about government or private sector activities. In other instances, influential "actors" take the form of climate change, communications technologies, or socioeconomic globalization. Although these individual forces may be small relative to state governments or international institutions, or may operate on long time scales, the changes they catalyze can pose significant challenges to the analysis and practice of international relations through the operation of complex feedbacks and interactions of individual agents and interconnected systems. We call these challenges "femtorisks," and emphasize their importance for two reasons. First, in isolation, they may be inconsequential and semiautonomous; but when embedded in complex adaptive systems, characterized by individual agents able to change, learn from experience, and pursue their own agendas, the strategic interaction between actors can propel systems down paths of increasing, even global, instability. Second, because their influence stems from complex interactions at interfaces of multiple systems (e.g., social, financial, political, technological, ecological, etc.), femtorisks challenge standard approaches to risk assessment, as higher-order consequences cascade across the boundaries of socially constructed complex systems. We argue that new approaches to assessing and managing systemic risk in international relations are required, inspired by principles of evolutionary theory and development of resilient ecological systems.

Bridging the gap between science and decision making.

All decisions, whether they are personal, public, or business-related, are based on the decision maker's beliefs and values. Science can and should help decision makers by shaping their beliefs. Unfortunately, science is not easily accessible to decision makers, and scientists often do not understand decision makers' information needs. This article presents a framework for bridging the gap between science and decision making and illustrates it with two examples. The first example is a personal health decision. It shows how a formal representation of the beliefs and values can reflect scientific inputs by a physician to combine with the values held by the decision maker to inform a medical choice. The second example is a public policy decision about managing a potential environmental hazard. It illustrates how controversial beliefs can be reflected as uncertainties and informed by science to make better decisions. Both examples use decision analysis to bridge science and decisions. The conclusions suggest that this can be a helpful process that requires skills in both science and decision making.

The value of environmental information without control of subsequent decisions.

The standard value of information approach of decision analysis assumes that the individual or agency that collects the information is also in control of the subsequent decisions based on the information. We refer to this situation as the "value of information with control (VOI-C)." This paradigm leads to powerful results, for example, that the value of information cannot be negative and that it is zero, when the information cannot change subsequent decisions. In many real world situations, however, the agency collecting the information is different from the one that makes the decision on the basis of that information. For example, an environmental research group may contemplate to fund a study that can affect an environmental policy decision that is made by a regulatory organization. In this two-agency formulation, the information-acquiring agency has to decide, whether an investment in research is worthwhile, while not being in control of the subsequent decision. We refer to this situation as "value of information without control (VOI-NC)." In this article, we present a framework for the VOI-NC and illustrate it with an example of a specific problem of determining the value of a research program on the health effects of power-frequency electromagnetic fields. We first compare the VOI-C approach with the VOI-NC approach. We show that the VOI-NC can be negative, but that with high-quality research (low probabilities of errors of type I and II) it is positive. We also demonstrate, both in the example and in more general mathematical terms, that the VOI-NC for environmental studies breaks down into a sum of the VOI-NC due to the possible reduction of environmental impacts and the VOI-NC due to the reduction of policy costs, with each component being positive for low environmental impacts and high-quality research. Interesting results include that the environmental and cost components of the VOI-NC move in opposite directions as a function of the probability of environmental impacts and that VOI-NC can be positive, even though the probability of environmental impacts is zero or one.

A value model for evaluating homeland security decisions.

One of the most challenging tasks of homeland security policymakers is to allocate their limited resources to reduce terrorism risks cost effectively. To accomplish this task, it is useful to develop a comprehensive set of homeland security objectives, metrics to measure each objective, a utility function, and value tradeoffs relevant for making homeland security investments. Together, these elements form a homeland security value model. This article develops a homeland security value model based on literature reviews, a survey, and experience with building value models. The purposes of the article are to motivate the use of a value model for homeland security decision making and to illustrate its use to assess terrorism risks, assess the benefits of countermeasures, and develop a severity index for terrorism attacks.

Exploring reductions in London Underground passenger journeys following the July 2005 bombings.

We examine the reduction in London Underground passenger journeys in response to the July 2005 bombings. Using entrance data for London Underground stations between 2001 and 2007, we incorporate demand and supply factors in a multivariate time-series regression model to estimate changes in passenger journeys between different Underground lines. We find that passenger journeys fell by an average of 8.3% for the 4 months following the attacks. This amounts to an overall reduction of 22.5 million passenger journeys for that period. Passenger journeys returned to predicted levels during September 2005, yet we find evidence of reduced travel until June 2006. Our estimates controlled for other factors, including reduced Underground service provision due to damage from the attacks, economic conditions, and weather, yet substantial reductions in passenger journeys remained. Around 82% of passenger journey reductions following the 2005 attacks cannot be attributed to supply-side factors or demand-side factors such as economic conditions, weather, or the summer school-break alone. We suggest that this reduction may partially be due to an increased perception of the risk of Underground travel after the attacks.

Identifying and structuring the objectives of terrorists.

The risk of terrorism is of great concern to many countries and significant resources are spent to counter this threat. A better understanding of the motivation of terrorists and their reasons for selecting certain modes and targets of attack can help improve the decisions to allocate resources in the fight against terrorism. The fundamental question addressed in this article is: "What do terrorists want?" We take the view that terrorists' preferences for actions are based on their values and beliefs. An important missing piece in our knowledge of terrorists' preferences is an understanding of their values. This article uses a novel approach to determine these values and state them as objectives, using principles from decision analysis and value-focused thinking. Instead of interviewing decisionmakers and stakeholders, as would be normal in decision analysis, we extract the values of terrorists by examining their own writings and verbal statements. To illustrate the approach, we extract the values of Al-Qaeda and structure them in terms of strategic, fundamental, and means objectives. These objectives are interrelated through a means-ends network. This information is useful for understanding terrorists' motivations, intent, and likely actions, as well as for developing policies to counter terrorism at its root causes.

Probabilistic risk analysis and terrorism risk.

Since the terrorist attacks of September 11, 2001, and the subsequent establishment of the U.S. Department of Homeland Security (DHS), considerable efforts have been made to estimate the risks of terrorism and the cost effectiveness of security policies to reduce these risks. DHS, industry, and the academic risk analysis communities have all invested heavily in the development of tools and approaches that can assist decisionmakers in effectively allocating limited resources across the vast array of potential investments that could mitigate risks from terrorism and other threats to the homeland. Decisionmakers demand models, analyses, and decision support that are useful for this task and based on the state of the art. Since terrorism risk analysis is new, no single method is likely to meet this challenge. In this article we explore a number of existing and potential approaches for terrorism risk analysis, focusing particularly on recent discussions regarding the applicability of probabilistic and decision analytic approaches to bioterrorism risks and the Bioterrorism Risk Assessment methodology used by the DHS and criticized by the National Academies and others.

Managing potential health risks from electric powerlines: a decision analysis caught in controversy.

Over the past 20 years, several epidemiological studies have found an association between exposure to electromagnetic fields (EMFs) and health effects, including childhood leukemia and adult brain cancer. However, experts strongly disagree about whether this association is causal and, if so, how strong it is. In this article, we examine several alternatives to reduce EMFs from sources of the California power grid, including undergrounding distribution and transmission lines and reconfiguring or rephasing lines. The alternatives were evaluated in terms of the potential health risk reduction, cost, impacts on service reliability, property values, and many other consequences. Because of the uncertainty about an EMF-health link, the main effort was to determine the sensitivity of the decisions to the probability and seriousness of an EMF hazard. User-friendly computer models were developed to allow stakeholders to change the model assumptions and parameters to analyze the impacts of their own assumptions and estimates on the decision. The analysis clearly demonstrated that only four of the many concerns raised by the stakeholders could make a difference in the decision: health risks, costs, service reliability, and property values. Whether undergrounding, moderate alternatives for EMF reduction, or no change was the best decision depended on a few key factors, including the probability that EMF exposure is a hazard, the severity of this hazard, how the EMF reduction measures are financed, and the impacts on property values. While the analysis did not resolve the EMF issues, it showed that even in the most controversial settings, a little analysis goes a long way to clarifying the issues and to focus the debate.