Evaluation and mitigation of the limitations of large language models in clinical decision-making.

Clinical decision-making is one of the most impactful parts of a physician's responsibilities and stands to benefit greatly from artificial intelligence solutions and large language models (LLMs) in particular. However, while LLMs have achieved excellent performance on medical licensing exams, these tests fail to assess many skills necessary for deployment in a realistic clinical decision-making environment, including gathering information, adhering to guidelines, and integrating into clinical workflows. Here we have created a curated dataset based on the Medical Information Mart for Intensive Care database spanning 2,400 real patient cases and four common abdominal pathologies as well as a framework to simulate a realistic clinical setting. We show that current state-of-the-art LLMs do not accurately diagnose patients across all pathologies (performing significantly worse than physicians), follow neither diagnostic nor treatment guidelines, and cannot interpret laboratory results, thus posing a serious risk to the health of patients. Furthermore, we move beyond diagnostic accuracy and demonstrate that they cannot be easily integrated into existing workflows because they often fail to follow instructions and are sensitive to both the quantity and order of information. Overall, our analysis reveals that LLMs are currently not ready for autonomous clinical decision-making while providing a dataset and framework to guide future studies.

ß-amyloid monomer scavenging by an anticalin protein prevents neuronal hyperactivity in mouse models of Alzheimer's Disease.

Hyperactivity mediated by synaptotoxic ß-amyloid (Aß) oligomers is one of the earliest forms of neuronal dysfunction in Alzheimer's disease. In the search for a preventive treatment strategy, we tested the effect of scavenging Aß peptides before Aß plaque formation. Using in vivo two-photon calcium imaging and SF-iGluSnFR-based glutamate imaging in hippocampal slices, we demonstrate that an Aß binding anticalin protein (Aß-anticalin) can suppress early neuronal hyperactivity and synaptic glutamate accumulation in the APP23xPS45 mouse model of ß-amyloidosis. Our results suggest that the sole targeting of Aß monomers is sufficient for the hyperactivity-suppressing effect of the Aß-anticalin at early disease stages. Biochemical and neurophysiological analyses indicate that the Aß-anticalin-dependent depletion of naturally secreted Aß monomers interrupts their aggregation to neurotoxic oligomers and, thereby, reverses early neuronal and synaptic dysfunctions. Thus, our results suggest that Aß monomer scavenging plays a key role in the repair of neuronal function at early stages of AD.