Towards optimal model evaluation: enhancing active testing with actively improved estimators.

With rapid advancements in machine learning and statistical models, ensuring the reliability of these models through accurate evaluation has become imperative. Traditional evaluation methods often rely on fully labeled test data, a requirement that is becoming increasingly impractical due to the growing size of datasets. In this work, we address this issue by extending existing work on active testing (AT) methods which are designed to sequentially sample and label data for evaluating pre-trained models. We propose two novel estimators: the Actively Improved Levelled Unbiased Risk (AILUR) and the Actively Improved Inverse Probability Weighting (AIIPW) estimators which are derived from nonparametric smoothing estimation. In addition, a model recalibration process is designed for the AIIPW estimator to optimize the sampling probability within the AT framework. We evaluate the proposed estimators on four real-world datasets and demonstrate that they consistently outperform existing AT methods. Our study also shows that the proposed methods are robust to changes in subsample sizes, and effective at reducing labeling costs.

Uses and limitations of artificial intelligence for oncology.

Modern artificial intelligence (AI) tools built on high-dimensional patient data are reshaping oncology care, helping to improve goal-concordant care, decrease cancer mortality rates, and increase workflow efficiency and scope of care. However, data-related concerns and human biases that seep into algorithms during development and post-deployment phases affect performance in real-world settings, limiting the utility and safety of AI technology in oncology clinics. To this end, the authors review the current potential and limitations of predictive AI for cancer diagnosis and prognostication as well as of generative AI, specifically modern chatbots, which interfaces with patients and clinicians. They conclude the review with a discussion on ongoing challenges and regulatory opportunities in the field.

Time of Clinic Appointment and Serious Illness Communication in Oncology.

INTRODUCTION: Serious illness communication in oncology increases goal concordant care. Factors associated with the frequency of serious illness conversations are not well understood. Given prior evidence of the association between suboptimal decision-making and clinic time, we aimed to investigate the relationship between appointment time and the likelihood of serious illness conversations in oncology. METHODS: We conducted a retrospective study of electronic health record data from 55 367 patient encounters between June 2019 to April 2020, using generalized estimating equations to model the likelihood of a serious illness conversation across clinic time. RESULTS: Documentation rate decreased from 2.1 to 1.5% in the morning clinic session (8am-12pm) and from 1.2% to .9% in the afternoon clinic session (1pm-4pm). Adjusted odds ratios for Serious illness conversations documentation rates were significantly lower for all hours of each session after the earliest hour (adjusted odds ratios .91 [95% CI, .84-.97], P = .006 for overall linear trend). CONCLUSIONS: Serious illness conversations between oncologists and patients decrease considerably through the clinic day, and proactive strategies to avoid missed conversations should be investigated.

The Coming of Age of Nucleic Acid Vaccines during COVID-19.

In the 21st century, several emergent viruses have posed a global threat. Each pathogen has emphasized the value of rapid and scalable vaccine development programs. The ongoing severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has made the importance of such efforts especially clear. New biotechnological advances in vaccinology allow for recent advances that provide only the nucleic acid building blocks of an antigen, eliminating many safety concerns. During the COVID-19 pandemic, these DNA and RNA vaccines have facilitated the development and deployment of vaccines at an unprecedented pace. This success was attributable at least in part to broader shifts in scientific research relative to prior epidemics: the genome of SARS-CoV-2 was available as early as January 2020, facilitating global efforts in the development of DNA and RNA vaccines within 2 weeks of the international community becoming aware of the new viral threat. Additionally, these technologies that were previously only theoretical are not only safe but also highly efficacious. Although historically a slow process, the rapid development of vaccines during the COVID-19 crisis reveals a major shift in vaccine technologies. Here, we provide historical context for the emergence of these paradigm-shifting vaccines. We describe several DNA and RNA vaccines in terms of their efficacy, safety, and approval status. We also discuss patterns in worldwide distribution. The advances made since early 2020 provide an exceptional illustration of how rapidly vaccine development technology has advanced in the last 2 decades in particular and suggest a new era in vaccines against emerging pathogens. IMPORTANCE The SARS-CoV-2 pandemic has caused untold damage globally, presenting unusual demands on but also unique opportunities for vaccine development. The development, production, and distribution of vaccines are imperative to saving lives, preventing severe illness, and reducing the economic and social burdens caused by the COVID-19 pandemic. Although vaccine technologies that provide the DNA or RNA sequence of an antigen had never previously been approved for use in humans, they have played a major role in the management of SARS-CoV-2. In this review, we discuss the history of these vaccines and how they have been applied to SARS-CoV-2. Additionally, given that the evolution of new SARS-CoV-2 variants continues to present a significant challenge in 2022, these vaccines remain an important and evolving tool in the biomedical response to the pandemic.

Application of Traditional Vaccine Development Strategies to SARS-CoV-2.

Over the past 150 years, vaccines have revolutionized the relationship between people and disease. During the COVID-19 pandemic, technologies such as mRNA vaccines have received attention due to their novelty and successes. However, more traditional vaccine development platforms have also yielded important tools in the worldwide fight against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). A variety of approaches have been used to develop COVID-19 vaccines that are now authorized for use in countries around the world. In this review, we highlight strategies that focus on the viral capsid and outwards, rather than on the nucleic acids inside. These approaches fall into two broad categories: whole-virus vaccines and subunit vaccines. Whole-virus vaccines use the virus itself, in either an inactivated or an attenuated state. Subunit vaccines contain instead an isolated, immunogenic component of the virus. Here, we highlight vaccine candidates that apply these approaches against SARS-CoV-2 in different ways. In a companion article (H. M. Rando, R. Lordan, L. Kolla, E. Sell, et al., mSystems 8:e00928-22, 2023, https://doi.org/10.1128/mSystems.00928-22), we review the more recent and novel development of nucleic acid-based vaccine technologies. We further consider the role that these COVID-19 vaccine development programs have played in prophylaxis at the global scale. Well-established vaccine technologies have proved especially important to making vaccines accessible in low- and middle-income countries. Vaccine development programs that use established platforms have been undertaken in a much wider range of countries than those using nucleic acid-based technologies, which have been led by wealthy Western countries. Therefore, these vaccine platforms, though less novel from a biotechnological standpoint, have proven to be extremely important to the management of SARS-CoV-2. IMPORTANCE The development, production, and distribution of vaccines is imperative to saving lives, preventing illness, and reducing the economic and social burdens caused by the COVID-19 pandemic. Vaccines that use cutting-edge biotechnology have played an important role in mitigating the effects of SARS-CoV-2. However, more traditional methods of vaccine development that were refined throughout the 20th century have been especially critical to increasing vaccine access worldwide. Effective deployment is necessary to reducing the susceptibility of the world's population, which is especially important in light of emerging variants. In this review, we discuss the safety, immunogenicity, and distribution of vaccines developed using established technologies. In a separate review, we describe the vaccines developed using nucleic acid-based vaccine platforms. From the current literature, it is clear that the well-established vaccine technologies are also highly effective against SARS-CoV-2 and are being used to address the challenges of COVID-19 globally, including in low- and middle-income countries. This worldwide approach is critical for reducing the devastating impact of SARS-CoV-2.

Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic.

Sudden changes in health care utilization during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic may have impacted the performance of clinical predictive models that were trained prior to the pandemic. In this study, we evaluated the performance over time of a machine learning, electronic health record-based mortality prediction algorithm currently used in clinical practice to identify patients with cancer who may benefit from early advance care planning conversations. We show that during the pandemic period, algorithm identification of high-risk patients had a substantial and sustained decline. Decreases in laboratory utilization during the peak of the pandemic may have contributed to drift. Calibration and overall discrimination did not markedly decline during the pandemic. This argues for careful attention to the performance and retraining of predictive algorithms that use inputs from the pandemic period.

The Coming of Age of Nucleic Acid Vaccines during COVID-19.

In the 21st century, several emergent viruses have posed a global threat. Each pathogen has emphasized the value of rapid and scalable vaccine development programs. The ongoing SARS-CoV-2 pandemic has made the importance of such efforts especially clear. New biotechnological advances in vaccinology allow for recent advances that provide only the nucleic acid building blocks of an antigen, eliminating many safety concerns. During the COVID-19 pandemic, these DNA and RNA vaccines have facilitated the development and deployment of vaccines at an unprecedented pace. This success was attributable at least in part to broader shifts in scientific research relative to prior epidemics; the genome of SARS-CoV-2 was available as early as January 2020, facilitating global efforts in the development of DNA and RNA vaccines within two weeks of the international community becoming aware of the new viral threat. Additionally, these technologies that were previously only theoretical are not only safe but also highly efficacious. Although historically a slow process, the rapid development of vaccines during the COVID-19 crisis reveals a major shift in vaccine technologies. Here, we provide historical context for the emergence of these paradigm-shifting vaccines. We describe several DNA and RNA vaccines and in terms of their efficacy, safety, and approval status. We also discuss patterns in worldwide distribution. The advances made since early 2020 provide an exceptional illustration of how rapidly vaccine development technology has advanced in the last two decades in particular and suggest a new era in vaccines against emerging pathogens.

Application of Traditional Vaccine Development Strategies to SARS-CoV-2.

Over the past 150 years, vaccines have revolutionized the relationship between people and disease. During the COVID-19 pandemic, technologies such as mRNA vaccines have received attention due to their novelty and successes. However, more traditional vaccine development platforms have also yielded important tools in the worldwide fight against the SARS-CoV-2 virus. A variety of approaches have been used to develop COVID-19 vaccines that are now authorized for use in countries around the world. In this review, we highlight strategies that focus on the viral capsid and outwards, rather than on the nucleic acids inside. These approaches fall into two broad categories: whole-virus vaccines and subunit vaccines. Whole-virus vaccines use the virus itself, either in an inactivated or attenuated state. Subunit vaccines contain instead an isolated, immunogenic component of the virus. Here, we highlight vaccine candidates that apply these approaches against SARS-CoV-2 in different ways. In a companion manuscript, we review the more recent and novel development of nucleic-acid based vaccine technologies. We further consider the role that these COVID-19 vaccine development programs have played in prophylaxis at the global scale. Well-established vaccine technologies have proved especially important to making vaccines accessible in low- and middle-income countries. Vaccine development programs that use established platforms have been undertaken in a much wider range of countries than those using nucleic-acid-based technologies, which have been led by wealthy Western countries. Therefore, these vaccine platforms, though less novel from a biotechnological standpoint, have proven to be extremely important to the management of SARS-CoV-2.

The case for AI-driven cancer clinical trials - The efficacy arm in silico.

Pharmaceutical agents in oncology currently have high attrition rates from early to late phase clinical trials. Recent advances in computational methods, notably causal artificial intelligence, and availability of rich clinico-genomic databases have made it possible to simulate the efficacy of cancer drug protocols in diverse patient populations, which could inform and improve clinical trial design. Here, we review the current and potential use of in silico trials and causal AI to increase the efficacy and safety of traditional clinical trials. We conclude that in silico trials using causal AI approaches can simulate control and efficacy arms, inform patient recruitment and regimen titrations, and better enable subgroup analyses critical for precision medicine.

Identification and characterization of key long non-coding RNAs in the mouse cochlea.

The auditory system is a complex sensory network with an orchestrated multilayer regulatory programme governing its development and maintenance. Accumulating evidence has implicated long non-coding RNAs (lncRNAs) as important regulators in numerous systems, as well as in pathological pathways. However, their function in the auditory system has yet to be explored. Using a set of specific criteria, we selected four lncRNAs expressed in the mouse cochlea, which are conserved in the human transcriptome and are relevant for inner ear function. Bioinformatic characterization demonstrated a lack of coding potential and an absence of evolutionary conservation that represent properties commonly shared by their class members. RNAscope®  analysis of the spatial and temporal expression profiles revealed specific localization to inner ear cells. Sub-cellular localization analysis presented a distinct pattern for each lncRNA and mouse tissue expression evaluation displayed a large variability in terms of level and location. Our findings establish the expression of specific lncRNAs in different cell types of the auditory system and present a potential pathway by which the lncRNA Gas5 acts in the inner ear. Studying lncRNAs and deciphering their functions may deepen our knowledge of inner ear physiology and morphology and may reveal the basis of as yet unresolved genetic hearing loss-related pathologies. Moreover, our experimental design may be employed as a reference for studying other inner ear-related lncRNAs, as well as lncRNAs expressed in other sensory systems.

Characterization of the development of the mouse cochlear epithelium at the single cell level.

Mammalian hearing requires the development of the organ of Corti, a sensory epithelium comprising unique cell types. The limited number of each of these cell types, combined with their close proximity, has prevented characterization of individual cell types and/or their developmental progression. To examine cochlear development more closely, we transcriptionally profile approximately 30,000 isolated mouse cochlear cells collected at four developmental time points. Here we report on the analysis of those cells including the identification of both known and unknown cell types. Trajectory analysis for OHCs indicates four phases of gene expression while fate mapping of progenitor cells suggests that OHCs and their surrounding supporting cells arise from a distinct (lateral) progenitor pool. Tgfßr1 is identified as being expressed in lateral progenitor cells and a Tgfßr1 antagonist inhibits OHC development. These results provide insights regarding cochlear development and demonstrate the potential value and application of this data set.

High content screen for identifying small-molecule LC3B-localization modulators in a renal cancer cell line.

Forms of selective autophagy have now been recognized to regulate flux in many intracellular processes. Specific pathways and functions have been identified for mitophagy, ERphagy, and other selective autophagies; yet there is no consensus in whether and how autophagy regulates protein maintenance in and around the nucleus. Such processes are of interest for potential degradation of DNA and nuclear envelope proteins in various disease states. The mechanistic details of such nucleus-related autophagic processes remain elusive due to the lack of chemical or genetic regulators to manipulate and follow the process in vitro. Here, we describe a high content screen from which we identified small chemical compounds that can modulate the localization of the autophagy marker MAP1LC3B (LC3) in renal carcinoma cells. We also describe a pipeline designed for the execution and analysis of high content screens. The chemical tools discerned from this screen will allow for the deeper exploration of the mechanism, regulation, and molecular targets of nuclear-localized LC3 in perturbed cellular states.

SUMO targeting of a stress-tolerant Ulp1 SUMO protease.

SUMO proteases of the SENP/Ulp family are master regulators of both sumoylation and desumoylation and regulate SUMO homeostasis in eukaryotic cells. SUMO conjugates rapidly increase in response to cellular stress, including nutrient starvation, hypoxia, osmotic stress, DNA damage, heat shock, and other proteotoxic stressors. Nevertheless, little is known about the regulation and targeting of SUMO proteases during stress. To this end we have undertaken a detailed comparison of the SUMO-binding activity of the budding yeast protein Ulp1 (ScUlp1) and its ortholog in the thermotolerant yeast Kluyveromyces marxianus, KmUlp1. We find that the catalytic UD domains of both ScUlp1 and KmUlp1 show a high degree of sequence conservation, complement a ulp1Δ mutant in vivo, and process a SUMO precursor in vitro. Next, to compare the SUMO-trapping features of both SUMO proteases we produced catalytically inactive recombinant fragments of the UD domains of ScUlp1 and KmUlp1, termed ScUTAG and KmUTAG respectively. Both ScUTAG and KmUTAG were able to efficiently bind a variety of purified SUMO isoforms and bound immobilized SUMO1 with nanomolar affinity. However, KmUTAG showed a greatly enhanced ability to bind SUMO and SUMO-modified proteins in the presence of oxidative, temperature and other stressors that induce protein misfolding. We also investigated whether a SUMO-interacting motif (SIM) in the UD domain of KmULP1 that is not conserved in ScUlp1 may contribute to the SUMO-binding properties of KmUTAG. In summary, our data reveal important details about how SUMO proteases target and bind their sumoylated substrates, especially under stress conditions. We also show that the robust pan-SUMO binding features of KmUTAG can be exploited to detect and study SUMO-modified proteins in cell culture systems.