Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need.

OBJECTIVE: To solve major clinical natural language processing (NLP) tasks using a unified text-to-text learning architecture based on a generative large language model (LLM) via prompt tuning. METHODS: We formulated 7 key clinical NLP tasks as text-to-text learning and solved them using one unified generative clinical LLM, GatorTronGPT, developed using GPT-3 architecture and trained with up to 20 billion parameters. We adopted soft prompts (ie, trainable vectors) with frozen LLM, where the LLM parameters were not updated (ie, frozen) and only the vectors of soft prompts were updated, known as prompt tuning. We added additional soft prompts as a prefix to the input layer, which were optimized during the prompt tuning. We evaluated the proposed method using 7 clinical NLP tasks and compared them with previous task-specific solutions based on Transformer models. RESULTS AND CONCLUSION: The proposed approach achieved state-of-the-art performance for 5 out of 7 major clinical NLP tasks using one unified generative LLM. Our approach outperformed previous task-specific transformer models by ∼3% for concept extraction and 7% for relation extraction applied to social determinants of health, 3.4% for clinical concept normalization, 3.4%-10% for clinical abbreviation disambiguation, and 5.5%-9% for natural language inference. Our approach also outperformed a previously developed prompt-based machine reading comprehension (MRC) model, GatorTron-MRC, for clinical concept and relation extraction. The proposed approach can deliver the "one model for all" promise from training to deployment using a unified generative LLM.

The Virtual Child.

SUMMARY: We are building the world's first Virtual Child-a computer model of normal and cancerous human development at the level of each individual cell. The Virtual Child will "develop cancer" that we will subject to unlimited virtual clinical trials that pinpoint, predict, and prioritize potential new treatments, bringing forward the day when no child dies of cancer, giving each one the opportunity to lead a full and healthy life.

A Guideline for Open-Source Tools to Make Medical Imaging Data Ready for Artificial Intelligence Applications: A Society of Imaging Informatics in Medicine (SIIM) Survey.

In recent years, the role of Artificial Intelligence (AI) in medical imaging has become increasingly prominent, with the majority of AI applications approved by the FDA being in imaging and radiology in 2023. The surge in AI model development to tackle clinical challenges underscores the necessity for preparing high-quality medical imaging data. Proper data preparation is crucial as it fosters the creation of standardized and reproducible AI models while minimizing biases. Data curation transforms raw data into a valuable, organized, and dependable resource and is a fundamental process to the success of machine learning and analytical projects. Considering the plethora of available tools for data curation in different stages, it is crucial to stay informed about the most relevant tools within specific research areas. In the current work, we propose a descriptive outline for different steps of data curation while we furnish compilations of tools collected from a survey applied among members of the Society of Imaging Informatics (SIIM) for each of these stages. This collection has the potential to enhance the decision-making process for researchers as they select the most appropriate tool for their specific tasks.

A study of generative large language model for medical research and healthcare.

There are enormous enthusiasm and concerns in applying large language models (LLMs) to healthcare. Yet current assumptions are based on general-purpose LLMs such as ChatGPT, which are not developed for medical use. This study develops a generative clinical LLM, GatorTronGPT, using 277 billion words of text including (1) 82 billion words of clinical text from 126 clinical departments and approximately 2 million patients at the University of Florida Health and (2) 195 billion words of diverse general English text. We train GatorTronGPT using a GPT-3 architecture with up to 20 billion parameters and evaluate its utility for biomedical natural language processing (NLP) and healthcare text generation. GatorTronGPT improves biomedical natural language processing. We apply GatorTronGPT to generate 20 billion words of synthetic text. Synthetic NLP models trained using synthetic text generated by GatorTronGPT outperform models trained using real-world clinical text. Physicians' Turing test using 1 (worst) to 9 (best) scale shows that there are no significant differences in linguistic readability (p = 0.22; 6.57 of GatorTronGPT compared with 6.93 of human) and clinical relevance (p = 0.91; 7.0 of GatorTronGPT compared with 6.97 of human) and that physicians cannot differentiate them (p < 0.001). This study provides insights into the opportunities and challenges of LLMs for medical research and healthcare.

Do Gradient Inversion Attacks Make Federated Learning Unsafe?

Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. This capability makes it especially interesting for healthcare applications where patient and data privacy is of utmost concern. However, recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in FL use-cases where the clients' training involves updating the Batch Normalization (BN) statistics and provide a new baseline attack that works for such scenarios. Furthermore, we present new ways to measure and visualize potential data leakage in FL. Our work is a step towards establishing reproducible methods of measuring data leakage in FL and could help determine the optimal tradeoffs between privacy-preserving techniques, such as differential privacy, and model accuracy based on quantifiable metrics.

A large language model for electronic health records.

There is an increasing interest in developing artificial intelligence (AI) systems to process and interpret electronic health records (EHRs). Natural language processing (NLP) powered by pretrained language models is the key technology for medical AI systems utilizing clinical narratives. However, there are few clinical language models, the largest of which trained in the clinical domain is comparatively small at 110 million parameters (compared with billions of parameters in the general domain). It is not clear how large clinical language models with billions of parameters can help medical AI systems utilize unstructured EHRs. In this study, we develop from scratch a large clinical language model-GatorTron-using >90 billion words of text (including >82 billion words of de-identified clinical text) and systematically evaluate it on five clinical NLP tasks including clinical concept extraction, medical relation extraction, semantic textual similarity, natural language inference (NLI), and medical question answering (MQA). We examine how (1) scaling up the number of parameters and (2) scaling up the size of the training data could benefit these NLP tasks. GatorTron models scale up the clinical language model from 110 million to 8.9 billion parameters and improve five clinical NLP tasks (e.g., 9.6% and 9.5% improvement in accuracy for NLI and MQA), which can be applied to medical AI systems to improve healthcare delivery. The GatorTron models are publicly available at: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/models/gatortron_og .

Federated learning improves site performance in multicenter deep learning without data sharing.

OBJECTIVE: To demonstrate enabling multi-institutional training without centralizing or sharing the underlying physical data via federated learning (FL). MATERIALS AND METHODS: Deep learning models were trained at each participating institution using local clinical data, and an additional model was trained using FL across all of the institutions. RESULTS: We found that the FL model exhibited superior performance and generalizability to the models trained at single institutions, with an overall performance level that was significantly better than that of any of the institutional models alone when evaluated on held-out test sets from each institution and an outside challenge dataset. DISCUSSION: The power of FL was successfully demonstrated across 3 academic institutions while avoiding the privacy risk associated with the transfer and pooling of patient data. CONCLUSION: Federated learning is an effective methodology that merits further study to enable accelerated development of models across institutions, enabling greater generalizability in clinical use.

Janus kinase 3 inhibition with CP-690,550 prevents allograft vasculopathy.

Janus kinase 3 (JAK3) mediates signal transduction from cytokine receptors using the common gamma chain. The rationally designed inhibitor of JAK3, CP-690,550, prevents acute allograft rejection in rodents and in nonhuman primates. Here we investigated the ability of CP-690,550, to prevent allograft vasculopathy in a rodent model of aorta transplantation. Aortas from AxC Irish (RT1(a)) or Lewis (RT1(l)) rats were heterotopically transplanted into the infra-renal aorta of Lewis recipients and harvested at 28 or 56 days. Treated recipients received CP-690,550 by osmotic pumps (mean drug exposure of 110 +/- 38 ng/ml). Significant intimal hyperplasia was demonstrated in untreated allografts when compared with isografts at 28 days (2.08 +/- 0.85% vs. 0.43 +/- 0.2% luminal obliteration, respectively, P = 0.001) and 56 days (5.3 +/- 2.4% vs. 0.38 +/- 0.3%, P = 0.002). Treatment caused a 51% reduction in intimal hyperplasia at day 56. CP-690,550-treated animals also had a significant reduction of donor-specific IgG production and of the gene expression for suppressor of cytokine signaling-3 and with unchanged levels of expression of RANTES, IP-10 and transforming growth factor-beta1. These results are the first to show that JAK3 blockade by CP-690,550 effectively prevents allograft vasculopathy in this rat model of aorta transplantation.

Combined use of the JAK3 inhibitor CP-690,550 with mycophenolate mofetil to prevent kidney allograft rejection in nonhuman primates.

BACKGROUND: Immunosuppression via Janus kinase (JAK) 3 inhibition affords significant prolongation of allograft survival. We investigated the effects of an immunosuppressive regimen combining the JAK3 inhibitor CP-690,550 with mycophenolate mofetil (MMF) in nonhuman primates (NHPs). METHODS: Life-supporting kidney transplantations were performed between ABO-compatible, MLR-mismatched NHPs. Animals were treated orally twice a day with CP-690,550 and MMF (n=8) or MMF alone (n=2) and were euthanized at day 90 or earlier due to allograft rejection. RESULTS: Mean survival time (+/-SEM) in animals treated with MMF alone (23+/-1 days) was significantly extended in animals that concurrently received CP-690,550 (59.5+/-9.8 days, P=0.02). Combination animals exposed to higher levels of CP-690,550 had a significantly better survival (75.2+/-8.7 days) than animals that received less CP-690,550 (33.3+/-12.6 days, P=0.02). Three combination therapy animals were euthanized at day 90 with a subnormal renal function and early-stage acute graft rejection. Rejection, delayed by treatment, ultimately developed in other animals. Anemia and gastrointestinal intolerance was seen in combination therapy animals that otherwise did not show evidence of viral or bacterial infection besides signs consistent with subclinical pyelonephritis (n=3). One incidental lymphosarcoma was noted. CONCLUSIONS: Addition of CP-690,550 to MMF significantly improved allograft survival. The observed side effects appear amenable to improvements upon alteration of dosing strategies. Efficacy of this combination regimen suggests that it could become the backbone of calcineurin inhibitor-free regimens.

Effects of JAK3 inhibition with CP-690,550 on immune cell populations and their functions in nonhuman primate recipients of kidney allografts.

BACKGROUND: Janus Kinase (JAK) 3 is a tyrosine kinase essential for proper signal transduction downstream of selected cytokine receptors and for robust T-cell and natural killer cells activation and function. JAK3 inhibition with CP-690,550 prevents acute allograft rejection. To provide further insight into the mechanisms of efficacy, we investigated the immunomodulatory effects of CP-690,550 in vitro and in vivo in nonhuman primates. METHODS: Pharmacodynamic assessments of lymphocyte activation, function, proliferation and phenotype were performed in three settings: in vitro in whole blood isolated from untransplanted cynomolgus monkeys (cynos), in vivo in blood from untransplanted cynos dosed with CP-690,550 for 8 days, and in vivo in blood from transplanted cynos immunosuppressed with CP-690,550. Cell surface activation markers expression, IL-2- enhanced IFN-gamma production, lymphocyte proliferation and immune cell phenotype analyzes were performed with multiparametric flow cytometry. RESULTS: In vitro exposure to CP-690,550 resulted in significant reduction of IL-2-enhanced IFN-gamma production by T-cells (maximum inhibition of 55-63%), T-cell surface expression of CD25 (50% inhibitory concentration (IC50); 0.18 microM) and CD71 (IC50; 1.6 microM), and T-cell proliferative capacities measured by proliferating cell nuclear antigen expression (IC50; 0.87 microM). Similar results were observed in animals dosed with CP-690,550. In addition, transplanted animals displayed significant reduction of NK cell (90% from baseline) and T-cell numbers whereas CD8 effector memory T-cell populations were unaffected. CONCLUSIONS: Potent in vitro and in vivo immunomodulatory effects of the JAK3 inhibitor CP-690,550 likely contribute to its efficacy in the prevention of organ allograft rejection.

Appraisal of the extent of chronic allograft vasculopathy in animal models: proposition of a standardized micromorphometric method.

Arterial obstruction due to intimal myproliferation is an outcome used for investigating chronic allograft vasculopathy in animal models. Since harvested tissue may be distorted while processing, discrepancy may happen in the measure of the obstruction. We propose a standardized micromorphometric method for calculating the percentage of obstruction with eliminating the variability in the shape of the vessel cross-section. The mathematical adjustment avoids the overestimation of the percentage of obstruction and improves accuracy.

A technique of bone marrow collection from vertebral bodies of cynomolgus macaques for transplant studies.

BACKGROUND: Strategies to induce donor-specific allograft tolerance are best tested in preclinical models developed in nonhuman primates (NHPs). Most protocols prepare the recipient by infusing hematopoietic cells from the donor. We report here a procedure to isolate and characterize large numbers of bone marrow cells (BMCs) from cynomolgus monkeys (cynos) that can then successfully be transplanted into conditioned recipients. MATERIALS AND METHODS: Vertebral columns of five cynos were excised en bloc and separated into individual vertebrae. The cancelous bone was extracted with a core puncher, fractionated, filtered, centrifuged, and resuspended in transplantation media before being analyzed by flow cytometry. In two instances, the collected BMCs were reinfused into allogeneic recipients preconditioned with a nonmyeloablative regimen. Chimerism was monitored using short-tandem repeat analysis. RESULTS: The mean total BMCs yield was 25.5 x 10(9) (range of 4.00 x 10(9) to 59 x 10(9)) with mean cell viability of 93.4% (range: 90-96%). CD34+ cells and CD3+ cells averaged 0.34 and 3.91% of total BMCs, respectively. This resulted in absolute cell number yields of 1.02 x 10(8) and 1.15 x 10(9) for CD34+ and CD3+ cells, respectively. Graft-versus-host disease was absent in both bone marrow infused animals, and a maximum level of chimerism of 18% was detected at 3 weeks after BMCs infusion. CONCLUSION: We present here the first detailed report of a procedure to retrieve and characterize large numbers of BMCs from vertebral bodies of cynos and demonstrate that cells collected with this technique have the capability of engrafting in allogenic recipients.

Immunosuppression by the JAK3 inhibitor CP-690,550 delays rejection and significantly prolongs kidney allograft survival in nonhuman primates.

BACKGROUND: Janus kinase 3 (JAK3) mediates signal transduction from cytokine receptors using the common chain (gammac). Because mutations in genes encoding gammac or JAK3 result in immunodeficiency, we investigated the potential of a rationally designed inhibitor of JAK3, CP-690,550, to prevent renal allograft rejection in nonhuman primates. METHODS: Life-supporting kidney transplantations were performed between mixed leukocyte reaction-mismatched, ABO blood group-matched cynomolgus monkeys. Animals were treated with CP-690,550 (n = 18) or its vehicle (controls, n = 3) and were euthanized at day 90 or earlier if there was allograft rejection. RESULTS: Mean survival time (+/- standard error of mean) in animals treated with CP-690,550 (53 +/- 7 days) was significantly longer than in control animals (7 +/- 1 days, P=0.0003) and was positively correlated with exposure to the drug (r = 0.79, P < 0.01). Four treated animals were euthanized at 90 days with a normal renal function and low-grade rejection at final pathology. Occurrence of rejection was significantly delayed in treated animals (46 +/- 7 days from transplantation vs. 7 +/- 1 days in controls, P = 0.0003). Persistent anemia, polyoma virus-like nephritis (n = 2), and urinary calcium carbonate accretions (n = 3) were seen in animals with high exposure. Natural killer cell and CD4 and CD8 T-cell numbers were significantly reduced in treated animals. Blood glucose, serum lipid levels, and arterial blood pressure were within normal range in treated animals, and no cancers were demonstrated. CONCLUSIONS: CP-690,550 is the first reported JAK3 inhibitor combining efficacy and good tolerability in a preclinical model of allotransplantation in nonhuman primates and thus has interesting potential for immunosuppression in humans.

In vitro evaluation of the effects of candidate immunosuppressive drugs: flow cytometry and quantitative real-time PCR as two independent and correlated read-outs.

BACKGROUND: Immune monitoring may use flow cytometry or molecular biology techniques. Flow cytometry assays cells that are phenotypically characterized, whereas TaqMan RT-PCR starts with RNA extraction from unfractionated heterogeneous cell populations. We therefore wondered how the effects of immunosuppressive drugs on cytokine production in stimulated whole blood, as determined by flow cytometry, would correlate with those obtained with quantitative real-time PCR (TaqMan RT-PCR). METHODS: Blood drawn from naive cynomolgus monkeys was exposed to incremental amounts of cyclosporine (CsA; 300, 600, 900 and 1200 ng/ml) or tacrolimus (TRL; 8, 20, 40 and 80 ng/ml) before lectin stimulation in vitro. Blood was then either stained for CD3, IFN-gamma, IL-2, IL-4, and TNF-alpha and analyzed on a flow cytometer with various gating strategies, or submitted to RNA extraction for analysis of the above mentioned cytokines mRNA transcripts using TaqMan RT-PCR. RESULTS: Both methods revealed a parallel dose-dependent inhibition of cytokine production in stimulated blood. The 50% inhibitory concentrations (IC(50)'s) ranged from 511-771 ng/ml (CsA) and 15-29 ng/ml (TRL) with flow cytometry, and from 275-529 ng/ml (CsA) and 11-48 ng/ml (TRL) with TaqMan RT-PCR for T-helper 1 cytokines. Both assays correlated well with a Pearson product moment correlation of 0.76. Extending gating from a CD3(+) gate to a lymphocyte gate improved correlation (r = 0.85) for all cytokines investigated (except IL-2; unchanged) whereas further extending gating resulted, to the contrary, in lower correlations. Independent of gating strategy a high correlation (r = 0.97) was observed when drug IC(50)'s were considered. CONCLUSIONS: Flow cytometry and TaqMan RT-PCR may be used interchangeably to monitor the effects of candidate immunosuppressive drugs on cytokine mRNA production in lectin-stimulated whole blood.