Diagnostic accuracy of an artificial intelligence online engine in migraine: A multi-center study.

OBJECTIVE: This study assesses the concordance in migraine diagnosis between an online, self-administered, Computer-based, Diagnostic Engine (CDE) and semi-structured interview (SSI) by a headache specialist, both using International Classification of Headache Disorders, 3rd edition (ICHD-3) criteria. BACKGROUND: Delay in accurate diagnosis is a major barrier to headache care. Accurate computer-based algorithms may help reduce the need for SSI-based encounters to arrive at correct ICHD-3 diagnosis. METHODS: Between March 2018 and August 2019, adult participants were recruited from three academic headache centers and the community via advertising to our cross-sectional study. Participants completed two evaluations: phone interview conducted by headache specialists using the SSI and a web-based expert questionnaire and analytics, CDE. Participants were randomly assigned to either the SSI followed by the web-based questionnaire or the web-based questionnaire followed by the SSI. Participants completed protocols a few minutes apart. The concordance in migraine/probable migraine (M/PM) diagnosis between SSI and CDE was measured using Cohen's kappa statistics. The diagnostic accuracy of CDE was assessed using the SSI as reference standard. RESULTS: Of the 276 participants consented, 212 completed both SSI and CDE (study completion rate = 77%; median age = 32 years [interquartile range: 28-40], female:male ratio = 3:1). Concordance in M/PM diagnosis between SSI and CDE was: κ = 0.83 (95% confidence interval [CI]: 0.75-0.91). CDE diagnostic accuracy: sensitivity = 90.1% (118/131), 95% CI: 83.6%-94.6%; specificity = 95.8% (68/71), 95% CI: 88.1%-99.1%. Positive and negative predictive values = 97.0% (95% CI: 91.3%-99.0%) and 86.6% (95% CI: 79.3%-91.5%), respectively, using identified migraine prevalence of 60%. Assuming a general migraine population prevalence of 10%, positive and negative predictive values were 70.3% (95% CI: 43.9%-87.8%) and 98.9% (95% CI: 98.1%-99.3%), respectively. CONCLUSION: The SSI and CDE have excellent concordance in diagnosing M/PM. Positive CDE helps rule in M/PM, through high specificity and positive likelihood ratio. A negative CDE helps rule out M/PM through high sensitivity and low negative likelihood ratio. CDE that mimics SSI logic is a valid tool for migraine diagnosis.

Behavioral Determinants of Target Shifting and Deterrence in an Analog Cyber-Attack Game.

This study examines how exploiting biases in probability judgment can enhance deterrence using a fixed allocation of defensive resources. We investigate attacker anchoring heuristics for conjunctive events with missing information to distort attacker estimates of success for targets with equal defensive resources. We designed and conducted a behavioral experiment functioning as an analog cyber attack with multiple targets requiring three stages of attack to successfully acquire a target. Each stage is associated with a probability of successfully attacking a layer of defense, reflecting the allocation of resources for each layer. There are four types of targets that have nearly equal likelihood of being successfully attacked, including one type with equally distributed success probabilities over every layer and three types with success probabilities that are concentrated to be lowest in the first, second, or third layer. Players are incentivized by a payoff system that offers a reward for successfully attacked targets and a penalty for failed attacks. We collected data from a total of 1,600 separate target selections from 80 players and discovered that the target type with the lowest probability of success on the first layer was least preferred among attackers, providing the greatest deterrent. Targets with equally distributed success probabilities across layers were the next least preferred among attackers, indicating greater deterrence for uniform-layered defenses compared to defenses that are concentrated at the inner (second or third) levels. This finding is consistent with both attacker anchoring and ambiguity biases and an interpretation of failed attacks as near misses.

Workarounds to computer access in healthcare organizations: you want my password or a dead patient?

Workarounds to computer access in healthcare are sufficiently common that they often go unnoticed. Clinicians focus on patient care, not cybersecurity. We argue and demonstrate that understanding workarounds to healthcare workers' computer access requires not only analyses of computer rules, but also interviews and observations with clinicians. In addition, we illustrate the value of shadowing clinicians and conducing focus groups to understand their motivations and tradeoffs for circumvention. Ethnographic investigation of the medical workplace emerges as a critical method of research because in the inevitable conflict between even well-intended people versus the machines, it's the people who are the more creative, flexible, and motivated. We conducted interviews and observations with hundreds of medical workers and with 19 cybersecurity experts, CIOs, CMIOs, CTO, and IT workers to obtain their perceptions of computer security. We also shadowed clinicians as they worked. We present dozens of ways workers ingeniously circumvent security rules. The clinicians we studied were not "black hat" hackers, but just professionals seeking to accomplish their work despite the security technologies and regulations.