BARD: A Structured Technique for Group Elicitation of Bayesian Networks to Support Analytic Reasoning.

In many complex, real-world situations, problem solving and decision making require effective reasoning about causation and uncertainty. However, human reasoning in these cases is prone to confusion and error. Bayesian networks (BNs) are an artificial intelligence technology that models uncertain situations, supporting better probabilistic and causal reasoning and decision making. However, to date, BN methodologies and software require (but do not include) substantial upfront training, do not provide much guidance on either the model building process or on using the model for reasoning and reporting, and provide no support for building BNs collaboratively. Here, we contribute a detailed description and motivation for our new methodology and application, Bayesian ARgumentation via Delphi (BARD). BARD utilizes BNs and addresses these shortcomings by integrating (1) short, high-quality e-courses, tips, and help on demand; (2) a stepwise, iterative, and incremental BN construction process; (3) report templates and an automated explanation tool; and (4) a multiuser web-based software platform and Delphi-style social processes. The result is an end-to-end online platform, with associated online training, for groups without prior BN expertise to understand and analyze a problem, build a model of its underlying probabilistic causal structure, validate and reason with the causal model, and (optionally) use it to produce a written analytic report. Initial experiments demonstrate that, for suitable problems, BARD aids in reasoning and reporting. Comparing their effect sizes also suggests BARD's BN-building and collaboration combine beneficially and cumulatively.

Individuals vs. BARD: Experimental Evaluation of an Online System for Structured, Collaborative Bayesian Reasoning.

US intelligence analysts must weigh up relevant evidence to assess the probability of their conclusions, and express this reasoning clearly in written reports for decision-makers. Typically, they work alone with no special analytic tools, and sometimes succumb to common probabilistic and causal reasoning errors. So, the US government funded a major research program (CREATE) for four large academic teams to develop new structured, collaborative, software-based methods that might achieve better results. Our team's method (BARD) is the first to combine two key techniques: constructing causal Bayesian network models (BNs) to represent analyst knowledge, and small-group collaboration via the Delphi technique. BARD also incorporates compressed, high-quality online training allowing novices to use it, and checklist-inspired report templates with a rudimentary AI tool for generating text explanations from analysts' BNs. In two prior experiments, our team showed BARD's BN-building assists probabilistic reasoning when used by individuals, with a large effect (Glass' Δ 0.8) (Cruz et al., 2020), and even minimal Delphi-style interactions improve the BN structures individuals produce, with medium to very large effects (Glass' Δ 0.5-1.3) (Bolger et al., 2020). This experiment is the critical test of BARD as an integrated system and possible alternative to business-as-usual for intelligence analysis. Participants were asked to solve three probabilistic reasoning problems spread over 5 weeks, developed by our team to test both quantitative accuracy and susceptibility to tempting qualitative fallacies. Our 256 participants were randomly assigned to form 25 teams of 6-9 using BARD and 58 individuals using Google Suite and (if desired) the best pen-and-paper techniques. For each problem, BARD outperformed this control with very large to huge effects (Glass' Δ 1.4-2.2), greatly exceeding CREATE's initial target. We conclude that, for suitable problems, BARD already offers significant advantages over both business-as-usual and existing BN software. Our effect sizes also suggest BARD's BN-building and collaboration combined beneficially and cumulatively, although implementation differences decreased performances compared to Cruz et al. (2020), so interaction may have contributed. BARD has enormous potential for further development and testing of specific components and on more complex problems, and many potential applications beyond intelligence analysis.

Information-theoretic models of deception: Modelling cooperation and diffusion in populations exposed to "fake news".

The modelling of deceptions in game theory and decision theory has not been well studied, despite the increasing importance of this problem in social media, public discourse, and organisational management. This paper presents an improved formulation of the extant information-theoretic models of deceptions, a framework for incorporating these models of deception into game and decision theoretic models of deception, and applies these models and this framework in an agent based evolutionary simulation that models two very common deception types employed in "fake news" attacks. The simulation results for both deception types modelled show, as observed empirically in many social systems subjected to "fake news" attacks, that even a very small population of deceivers that transiently invades a much larger population of non-deceiving agents can strongly alter the equilibrium behaviour of the population in favour of agents playing an always defect strategy. The results also show that the ability of a population of deceivers to establish itself or remain present in a population is highly sensitive to the cost of the deception, as this cost reduces the fitness of deceiving agents when competing against non-deceiving agents. Diffusion behaviours observed for agents exploiting the deception producing false beliefs are very close to empirically observed behaviours in social media, when fitted to epidemiological models. We thus demonstrate, using the improved formulation of the information-theoretic models of deception, that agent based evolutionary simulations employing the Iterated Prisoner's Dilemma can accurately capture the behaviours of a population subject to deception attacks introducing uncertainty and false perceptions, and show that information-theoretic models of deception have practical applications beyond trivial taxonomical analysis.

The effects of demographic change on disease transmission and vaccine impact in a household structured population.

The demographic structure of populations in both more developed and less developed countries is changing: increases in life expectancy and declining fertility have led to older populations and smaller households. The implications of these demographic changes for the spread and control of infectious diseases are not fully understood. Here we use an individual based model with realistic and dynamic age and household structure to demonstrate the marked effect that demographic change has on disease transmission at the population and household level. The decline in fertility is associated with a decrease in disease incidence and an increase in the age of first infection, even in the absence of vaccination or other control measures. Although large households become rarer as fertility decreases, we show that there is a proportionate increase in incidence of disease in these households as the accumulation of susceptible clusters increases the potential for explosive outbreaks. By modelling vaccination, we provide a direct comparison of the relative importance of demographic change and vaccination on incidence of disease. We highlight the increased risks associated with unvaccinated households in a low fertility setting if vaccine behaviour is correlated with household membership. We suggest that models that do not account for future demographic change, and especially its effect on household structure, may potentially overestimate the impact of vaccination.

Incorporating expert knowledge when learning Bayesian network structure: a medical case study.

OBJECTIVES: Bayesian networks (BNs) are rapidly becoming a leading technology in applied Artificial Intelligence, with many applications in medicine. Both automated learning of BNs and expert elicitation have been used to build these networks, but the potentially more useful combination of these two methods remains underexplored. In this paper we examine a number of approaches to their combination when learning structure and present new techniques for assessing their results. METHODS AND MATERIALS: Using public-domain medical data, we run an automated causal discovery system, CaMML, which allows the incorporation of multiple kinds of prior expert knowledge into its search, to test and compare unbiased discovery with discovery biased with different kinds of expert opinion. We use adjacency matrices enhanced with numerical and colour labels to assist with the interpretation of the results. We present an algorithm for generating a single BN from a set of learned BNs that incorporates user preferences regarding complexity vs completeness. These techniques are presented as part of the first detailed workflow for hybrid structure learning within the broader knowledge engineering process. RESULTS: The detailed knowledge engineering workflow is shown to be useful for structuring a complex iterative BN development process. The adjacency matrices make it clear that for our medical case study using the IOWA dataset, the simplest kind of prior information (partially sorting variables into tiers) was more effective in aiding model discovery than either using no prior information or using more sophisticated and detailed expert priors. The method for generating a single BN captures relationships that would be overlooked by other approaches in the literature. CONCLUSION: Hybrid causal learning of BNs is an important emerging technology. We present methods for incorporating it into the knowledge engineering process, including visualisation and analysis of the learned networks.