Validating a forced-choice method for eliciting quality-of-reasoning judgments.

In this paper we investigate the criterion validity of forced-choice comparisons of the quality of written arguments with normative solutions. Across two studies, novices and experts assessing quality of reasoning through a forced-choice design were both able to choose arguments supporting more accurate solutions-62.2% (SE = 1%) of the time for novices and 74.4% (SE = 1%) for experts-and arguments produced by larger teams-up to 82% of the time for novices and 85% for experts-with high inter-rater reliability, namely 70.58% (95% CI = 1.18) agreement for novices and 80.98% (95% CI = 2.26) for experts. We also explored two methods for increasing efficiency. We found that the number of comparative judgments needed could be substantially reduced with little accuracy loss by leveraging transitivity and producing quality-of-reasoning assessments using an AVL tree method. Moreover, a regression model trained to predict scores based on automatically derived linguistic features of participants' judgments achieved a high correlation with the objective accuracy scores of the arguments in our dataset. Despite the inherent subjectivity involved in evaluating differing quality of reasoning, the forced-choice paradigm allows even novice raters to perform beyond chance and can provide a valid, reliable, and efficient method for producing quality-of-reasoning assessments at scale.

Temporal variation in skin microbiota of cohabitating amphibians.

Temporal changes and transmission patterns in host-associated microbial communities have important implications for host health. The diversity of amphibian skin microbial communities is associated with disease outcome in amphibians exposed to the fungal pathogen Batrachochytrium dendrobatidis (Bd). To successfully develop conservation strategies against Bd, we need a comprehensive understanding of how skin microbes are maintained and transmitted over time within populations. We used 16S rRNA sequence analysis to compare Epipedobates anthonyi frogs housed with one conspecific to frogs housed singly at four time points over the course of 1 year. We found that both α and ß diversity of frog skin bacterial communities changed significantly over the course of the experiment. Specifically, we found that bacterial communities of cohabitating frogs became more similar over time. We also observed that some bacterial taxa were differentially abundant between frogs housed singly and frogs housed with a conspecific. These results suggest that conspecific contact may play a role in mediating amphibian skin microbial diversity and that turnover of skin microbial communities can occur across time. Our findings provide rationale for future studies exploring horizontal transmission as a potential mechanism of host-associated microbial maintenance in amphibians.

Frog Skin Microbiota Vary With Host Species and Environment but Not Chytrid Infection.

Describing the structure and function of the amphibian cutaneous microbiome has gained importance with the spread of Batrachochytrium dendrobatidis (Bd), the fungal pathogen that can cause the skin disease chytridiomycosis. Sampling amphibian skin microbiota is needed to characterize current infection status and to help predict future susceptibility to Bd based on microbial composition since some skin microbes have antifungal capabilities that may confer disease resistance. Here, I use 16S rRNA sequencing to describe the composition and structure of the cutaneous microbiota of six species of amphibians. Frog skin samples were also tested for Bd, and I found 11.8% Bd prevalence among all individuals sampled (n = 76). Frog skin microbiota varied by host species and sampling site, but did not differ among Bd-positive and Bd-negative individuals. These results suggest that bacterial composition reflects host species and the environment, but does not reflect Bd infection among the species sampled here. Of the bacterial OTUs identified using an indicator species analysis as strongly associated with amphibians, significantly more indicator OTUs were putative anti-Bd taxa than would be expected based on the proportion of anti-Bd OTUs among all frog OTUs, suggesting strong associations between host species and anti-Bd OTUs. This relationship may partially explain why some of these frogs are asymptomatic carriers of Bd, but more work is needed to determine the other factors that contribute to interspecific variation in Bd susceptibility. This work provides important insights on inter- and intra-specific variation in microbial community composition, putative function, and disease dynamics in populations of amphibians that appear to be coexisting with Bd.

The role of narrative in collaborative reasoning and intelligence analysis: A case study.

This paper explores the significance of narrative in collaborative reasoning using a qualitative case study of two teams of intelligence analysts who took part in an exercise using an online collaborative platform. Digital ethnographic methods were used to analyze the chat transcripts of analysts as they reasoned with evidence provided in a difficult, fictional intelligence-type problem and produced a final intelligence report. These chat transcripts provided a powerful "microscope" into the reasoning processes and interactions involved in complex, collaborative reasoning. We found that Individuals and teams used narrative to solve the kinds of complex problems organizations and intelligence agencies face daily. We observed that team members generated what we term "micro-narratives", which provided a means for testing, assessing and weighing alternative hypotheses through mental simulation in the context of collaborative reasoning. The creation of micro-narratives assisted in the teams' reasoning with evidence, an integral part of collaborative reasoning and intelligence analysis. Micro-narratives were combined into, and compared with, an ideal or 'virtual' narrative which informed the judgements the team came to in their final intelligence report. The case study developed in this paper provides evidence that narrative thought processes play an important role in complex collaborative problem-solving, reasoning with evidence and problem-solving. This is contrary to a widespread perception that narrative thinking is fundamentally distinct from formal, logical reasoning.

Internal Medicine Resident Work Absence During the COVID-19 Pandemic at a Large Academic Medical Center in New York City.

BACKGROUND: Montefiore Medical Center (MMC) is a large tertiary care center in the Bronx, New York City, with 245 internal medicine residents. Beginning on February 29, 2020, residents became ill with COVID-19-like illness (CLI), which required absence from work. There was initially a shortage of personal protective equipment and delays in SARS-CoV-2 testing, which gradually improved during March and April 2020. OBJECTIVE: We evaluated the relationship between CLI-related work absence rates of internal medicine residents and MMC's COVID-19 hospital census over time. METHODS: Data on resident work absence between February 29 and May 22 were reviewed along with MMC's COVID-19 hospital census data. To determine the effect of patient exposure on resident CLI incidence, we compared the mean incidence of CLI per patient exposure days (PED = daily hospital census × days pre- or post-peak) before and after peak COVID-19 hospital census. RESULTS: Forty-two percent (103 of 245) of internal medicine residents were absent from work, resulting in 875 missed workdays. At the peak of resident work absence, 16% (38 of 245) were out sick. Residents were absent for a median of 7 days (IQR 6-9.5 days). Mean resident CLI incidence per PED (CLI/PED) was 13.9-fold lower post-peak compared to pre-peak (P = .003). CONCLUSIONS: At the beginning of the COVID-19 pandemic in New York City, a large portion of internal medicine residents at this single center became ill. However, the incidence of CLI decreased over time, despite ongoing exposure to patients with COVID-19.

Functional Redundancy of Batrachochytrium dendrobatidis Inhibition in Bacterial Communities Isolated from Lithobates clamitans Skin.

The cutaneous microbial community can influence the health of amphibians exposed to Batrachochytrium dendrobatidis (Bd), a fungal pathogen that has contributed to recent amphibian declines. Resistance to Bd in amphibian populations is correlated with the presence of anti-Bd cutaneous microbes, which confer disease resistance by inhibiting Bd growth. I aimed to determine if green frogs (Lithobates clamitans), an abundant and widely distributed species in New Jersey, harbored bacteria that inhibit Bd and whether the presence and identity of these microbes varied among sites. I used in vitro challenge assays to determine if bacteria isolated from green frog skin could inhibit or enhance the growth of Bd. I found that green frogs at all sites harbored anti-Bd bacteria. However, there were differences in Bd inhibition capabilities among bacterial isolates identified as the same operational taxonomic unit (OTU), lending support to the idea that phylogenetic relatedness does not always predict Bd inhibition status. Additionally, anti-Bd bacterial richness did not vary by site, but the composition of anti-Bd bacterial taxa was distinct at each site. This suggests that there is functional redundancy of Bd inhibition across unique communities of anti-Bd symbionts found on frogs at different sites. These findings highlight the need to better elucidate the structure-function relationship of microbiomes and their role in disease resistance.

Better Together: Reliable Application of the Post-9/11 and Post-Iraq US Intelligence Tradecraft Standards Requires Collective Analysis.

Background: The events of 9/11 and the October 2002 National Intelligence Estimate on Iraq's Continuing Programs for Weapons of Mass Destruction precipitated fundamental changes within the United States Intelligence Community. As part of the reform, analytic tradecraft standards were revised and codified into a policy document - Intelligence Community Directive (ICD) 203 - and an analytic ombudsman was appointed in the newly created Office for the Director of National Intelligence to ensure compliance across the intelligence community. In this paper we investigate the untested assumption that the ICD203 criteria can facilitate reliable evaluations of analytic products. Methods: Fifteen independent raters used a rubric based on the ICD203 criteria to assess the quality of reasoning of 64 analytical reports generated in response to hypothetical intelligence problems. We calculated the intra-class correlation coefficients for single and group-aggregated assessments. Results: Despite general training and rater calibration, the reliability of individual assessments was poor. However, aggregate ratings showed good to excellent reliability. Conclusion: Given that real problems will be more difficult and complex than our hypothetical case studies, we advise that groups of at least three raters are required to obtain reliable quality control procedures for intelligence products. Our study sets limits on assessment reliability and provides a basis for further evaluation of the predictive validity of intelligence reports generated in compliance with the tradecraft standards.

Endosialin: a novel malignant cell therapeutic target for neuroblastoma.

Endosialin emerged recently as a potential therapeutic target for sarcoma. Since some sarcoma subtypes, such as Ewing's sarcoma, show characteristics of neuroendocrine differentiation, we wondered whether cancers with neuro-endocrine properties and/or neuroectodermal origin, such as neuroblastoma, small cell lung cancer and melanoma, may express endosialin. Endosialin protein expression was surveyed in neuroblastoma, small cell lung cancer and melanoma in human clinical specimens by immunohistochemistry (IHC) and in human cell lines by flow cytometry. Side population cells were examined to determine whether cancer stem cells can express endosialin. Endosialin-expressing neuroblastoma cell lines were implanted in immunodeficient mice and allowed to grow. The xenograft tumors were resected and tested for endosialin expression by IHC. In human clinical specimens, vascular endosialin staining was observed in neuroblastoma, small cell lung cancer and melanoma. Malignant cell staining was strongest in neuroblastoma, weak in melanoma and rare in small cell lung cancer. In human cell lines, endosialin was detected in neuroblastoma cell lines, including cancer stem cell-like side population (SP) cells, but was absent in melanoma and was both rare and weak in small cell lung cancer. Human neuroblastoma xenograft tumors were found to be positive for endosialin. Our work suggests that endosialin may be a suitable therapeutic target for neuroblastoma.

Characteristics of human Ewing/PNET sarcoma models.

Ewing/PNET (peripheral neuroepithelioma) tumors are rare aggressive bone sarcomas occurring in young people. Rare-disease clinical trials can require global collaborations and many years. In vivo models that as accurately as possible reflect the clinical disease are helpful in selecting therapeutics with the most promise of positive clinical impact. Human Ewing/PNET sarcoma cell lines developed over the past 45 years are described. Several of these have undergone genetic analysis and have been confirmed to be those of Ewing/PNET sarcoma. The A673 Ewing sarcoma line has proven to be particularly useful in understanding the biology of this disease in the mouse. The chromosomal translocation producing the EWS/FLI1 fusion transcript characterizes clinical Ewing sarcoma. Cell lines that express this genetic profile are confirmed to be those of Ewing sarcoma. The A673 Ewing sarcoma line grows in culture and as a xenograft in immunodeficient mice. The A673 model has been used to study Ewing sarcoma angiogenesis and response to antiangiogenic agents. Many Ewing sarcoma clinical specimens express the cell surface protein endosialin. Several Ewing sarcoma cell lines, including the A673 line, also express cell surface endosialin when grown as subcutaneous tumor nodules and as disseminated disease; thus the A673 is a useful model for the study of endosialin biology and endosialin-directed therapies. With the advent of tools that allow characterization of clinical disease to facilitate optimal treatment, it becomes imperative, especially for rare tumors, to develop preclinical models reflecting disease subsets. Ewing PNET sarcomas are a rare disease where models are available.