Demographic bias in misdiagnosis by computational pathology models.

Despite increasing numbers of regulatory approvals, deep learning-based computational pathology systems often overlook the impact of demographic factors on performance, potentially leading to biases. This concern is all the more important as computational pathology has leveraged large public datasets that underrepresent certain demographic groups. Using publicly available data from The Cancer Genome Atlas and the EBRAINS brain tumor atlas, as well as internal patient data, we show that whole-slide image classification models display marked performance disparities across different demographic groups when used to subtype breast and lung carcinomas and to predict IDH1 mutations in gliomas. For example, when using common modeling approaches, we observed performance gaps (in area under the receiver operating characteristic curve) between white and Black patients of 3.0% for breast cancer subtyping, 10.9% for lung cancer subtyping and 16.0% for IDH1 mutation prediction in gliomas. We found that richer feature representations obtained from self-supervised vision foundation models reduce performance variations between groups. These representations provide improvements upon weaker models even when those weaker models are combined with state-of-the-art bias mitigation strategies and modeling choices. Nevertheless, self-supervised vision foundation models do not fully eliminate these discrepancies, highlighting the continuing need for bias mitigation efforts in computational pathology. Finally, we demonstrate that our results extend to other demographic factors beyond patient race. Given these findings, we encourage regulatory and policy agencies to integrate demographic-stratified evaluation into their assessment guidelines.

Algorithmic fairness in artificial intelligence for medicine and healthcare.

In healthcare, the development and deployment of insufficiently fair systems of artificial intelligence (AI) can undermine the delivery of equitable care. Assessments of AI models stratified across subpopulations have revealed inequalities in how patients are diagnosed, treated and billed. In this Perspective, we outline fairness in machine learning through the lens of healthcare, and discuss how algorithmic biases (in data acquisition, genetic variation and intra-observer labelling variability, in particular) arise in clinical workflows and the resulting healthcare disparities. We also review emerging technology for mitigating biases via disentanglement, federated learning and model explainability, and their role in the development of AI-based software as a medical device.

A deep-learning model for transforming the style of tissue images from cryosectioned to formalin-fixed and paraffin-embedded.

Histological artefacts in cryosectioned tissue can hinder rapid diagnostic assessments during surgery. Formalin-fixed and paraffin-embedded (FFPE) tissue provides higher quality slides, but the process for obtaining them is laborious (typically lasting 12-48 h) and hence unsuitable for intra-operative use. Here we report the development and performance of a deep-learning model that improves the quality of cryosectioned whole-slide images by transforming them into the style of whole-slide FFPE tissue within minutes. The model consists of a generative adversarial network incorporating an attention mechanism that rectifies cryosection artefacts and a self-regularization constraint between the cryosectioned and FFPE images for the preservation of clinically relevant features. Transformed FFPE-style images of gliomas and of non-small-cell lung cancers from a dataset independent from that used to train the model improved the rates of accurate tumour subtyping by pathologists.

Fast and scalable search of whole-slide images via self-supervised deep learning.

The adoption of digital pathology has enabled the curation of large repositories of gigapixel whole-slide images (WSIs). Computationally identifying WSIs with similar morphologic features within large repositories without requiring supervised training can have significant applications. However, the retrieval speeds of algorithms for searching similar WSIs often scale with the repository size, which limits their clinical and research potential. Here we show that self-supervised deep learning can be leveraged to search for and retrieve WSIs at speeds that are independent of repository size. The algorithm, which we named SISH (for self-supervised image search for histology) and provide as an open-source package, requires only slide-level annotations for training, encodes WSIs into meaningful discrete latent representations and leverages a tree data structure for fast searching followed by an uncertainty-based ranking algorithm for WSI retrieval. We evaluated SISH on multiple tasks (including retrieval tasks based on tissue-patch queries) and on datasets spanning over 22,000 patient cases and 56 disease subtypes. SISH can also be used to aid the diagnosis of rare cancer types for which the number of available WSIs is often insufficient to train supervised deep-learning models.

Artificial intelligence for multimodal data integration in oncology.

In oncology, the patient state is characterized by a whole spectrum of modalities, ranging from radiology, histology, and genomics to electronic health records. Current artificial intelligence (AI) models operate mainly in the realm of a single modality, neglecting the broader clinical context, which inevitably diminishes their potential. Integration of different data modalities provides opportunities to increase robustness and accuracy of diagnostic and prognostic models, bringing AI closer to clinical practice. AI models are also capable of discovering novel patterns within and across modalities suitable for explaining differences in patient outcomes or treatment resistance. The insights gleaned from such models can guide exploration studies and contribute to the discovery of novel biomarkers and therapeutic targets. To support these advances, here we present a synopsis of AI methods and strategies for multimodal data fusion and association discovery. We outline approaches for AI interpretability and directions for AI-driven exploration through multimodal data interconnections. We examine challenges in clinical adoption and discuss emerging solutions.

Pan-cancer integrative histology-genomic analysis via multimodal deep learning.

The rapidly emerging field of computational pathology has demonstrated promise in developing objective prognostic models from histology images. However, most prognostic models are either based on histology or genomics alone and do not address how these data sources can be integrated to develop joint image-omic prognostic models. Additionally, identifying explainable morphological and molecular descriptors from these models that govern such prognosis is of interest. We use multimodal deep learning to jointly examine pathology whole-slide images and molecular profile data from 14 cancer types. Our weakly supervised, multimodal deep-learning algorithm is able to fuse these heterogeneous modalities to predict outcomes and discover prognostic features that correlate with poor and favorable outcomes. We present all analyses for morphological and molecular correlates of patient prognosis across the 14 cancer types at both a disease and a patient level in an interactive open-access database to allow for further exploration, biomarker discovery, and feature assessment.

Deep learning-enabled assessment of cardiac allograft rejection from endomyocardial biopsies.

Endomyocardial biopsy (EMB) screening represents the standard of care for detecting allograft rejections after heart transplant. Manual interpretation of EMBs is affected by substantial interobserver and intraobserver variability, which often leads to inappropriate treatment with immunosuppressive drugs, unnecessary follow-up biopsies and poor transplant outcomes. Here we present a deep learning-based artificial intelligence (AI) system for automated assessment of gigapixel whole-slide images obtained from EMBs, which simultaneously addresses detection, subtyping and grading of allograft rejection. To assess model performance, we curated a large dataset from the United States, as well as independent test cohorts from Turkey and Switzerland, which includes large-scale variability across populations, sample preparations and slide scanning instrumentation. The model detects allograft rejection with an area under the receiver operating characteristic curve (AUC) of 0.962; assesses the cellular and antibody-mediated rejection type with AUCs of 0.958 and 0.874, respectively; detects Quilty B lesions, benign mimics of rejection, with an AUC of 0.939; and differentiates between low-grade and high-grade rejections with an AUC of 0.833. In a human reader study, the AI system showed non-inferior performance to conventional assessment and reduced interobserver variability and assessment time. This robust evaluation of cardiac allograft rejection paves the way for clinical trials to establish the efficacy of AI-assisted EMB assessment and its potential for improving heart transplant outcomes.

Federated learning for computational pathology on gigapixel whole slide images.

Deep Learning-based computational pathology algorithms have demonstrated profound ability to excel in a wide array of tasks that range from characterization of well known morphological phenotypes to predicting non human-identifiable features from histology such as molecular alterations. However, the development of robust, adaptable and accurate deep learning-based models often rely on the collection and time-costly curation large high-quality annotated training data that should ideally come from diverse sources and patient populations to cater for the heterogeneity that exists in such datasets. Multi-centric and collaborative integration of medical data across multiple institutions can naturally help overcome this challenge and boost the model performance but is limited by privacy concerns among other difficulties that may arise in the complex data sharing process as models scale towards using hundreds of thousands of gigapixel whole slide images. In this paper, we introduce privacy-preserving federated learning for gigapixel whole slide images in computational pathology using weakly-supervised attention multiple instance learning and differential privacy. We evaluated our approach on two different diagnostic problems using thousands of histology whole slide images with only slide-level labels. Additionally, we present a weakly-supervised learning framework for survival prediction and patient stratification from whole slide images and demonstrate its effectiveness in a federated setting. Our results show that using federated learning, we can effectively develop accurate weakly-supervised deep learning models from distributed data silos without direct data sharing and its associated complexities, while also preserving differential privacy using randomized noise generation. We also make available an easy-to-use federated learning for computational pathology software package: http://github.com/mahmoodlab/HistoFL.

Bitbow Enables Highly Efficient Neuronal Lineage Tracing and Morphology Reconstruction in Single Drosophila Brains.

Identifying the cellular origins and mapping the dendritic and axonal arbors of neurons have been century old quests to understand the heterogeneity among these brain cells. Current Brainbow based transgenic animals take the advantage of multispectral labeling to differentiate neighboring cells or lineages, however, their applications are limited by the color capacity. To improve the analysis throughput, we designed Bitbow, a digital format of Brainbow which exponentially expands the color palette to provide tens of thousands of spectrally resolved unique labels. We generated transgenic Bitbow Drosophila lines, established statistical tools, and streamlined sample preparation, image processing, and data analysis pipelines to conveniently mapping neural lineages, studying neuronal morphology and revealing neural network patterns with unprecedented speed, scale, and resolution.

AI-based pathology predicts origins for cancers of unknown primary.

Cancer of unknown primary (CUP) origin is an enigmatic group of diagnoses in which the primary anatomical site of tumour origin cannot be determined1,2. This poses a considerable challenge, as modern therapeutics are predominantly specific to the primary tumour3. Recent research has focused on using genomics and transcriptomics to identify the origin of a tumour4-9. However, genomic testing is not always performed and lacks clinical penetration in low-resource settings. Here, to overcome these challenges, we present a deep-learning-based algorithm-Tumour Origin Assessment via Deep Learning (TOAD)-that can provide a differential diagnosis for the origin of the primary tumour using routinely acquired histology slides. We used whole-slide images of tumours with known primary origins to train a model that simultaneously identifies the tumour as primary or metastatic and predicts its site of origin. On our held-out test set of tumours with known primary origins, the model achieved a top-1 accuracy of 0.83 and a top-3 accuracy of 0.96, whereas on our external test set it achieved top-1 and top-3 accuracies of 0.80 and 0.93, respectively. We further curated a dataset of 317 cases of CUP for which a differential diagnosis was assigned. Our model predictions resulted in concordance for 61% of cases and a top-3 agreement of 82%. TOAD can be used as an assistive tool to assign a differential diagnosis to complicated cases of metastatic tumours and CUPs and could be used in conjunction with or in lieu of ancillary tests and extensive diagnostic work-ups to reduce the occurrence of CUP.

Data-efficient and weakly supervised computational pathology on whole-slide images.

Deep-learning methods for computational pathology require either manual annotation of gigapixel whole-slide images (WSIs) or large datasets of WSIs with slide-level labels and typically suffer from poor domain adaptation and interpretability. Here we report an interpretable weakly supervised deep-learning method for data-efficient WSI processing and learning that only requires slide-level labels. The method, which we named clustering-constrained-attention multiple-instance learning (CLAM), uses attention-based learning to identify subregions of high diagnostic value to accurately classify whole slides and instance-level clustering over the identified representative regions to constrain and refine the feature space. By applying CLAM to the subtyping of renal cell carcinoma and non-small-cell lung cancer as well as the detection of lymph node metastasis, we show that it can be used to localize well-known morphological features on WSIs without the need for spatial labels, that it overperforms standard weakly supervised classification algorithms and that it is adaptable to independent test cohorts, smartphone microscopy and varying tissue content.

Histological features of BRAF V600E-mutant anaplastic thyroid carcinoma.

AIMS: Treatment with a BRAF inhibitor, alone or in combination with a MEK inhibitor, may be considered for BRAF-mutant anaplastic thyroid carcinoma (ATC). The purpose of this study was to characterise the histology of BRAF V600E-mutant ATC. METHODS AND RESULTS: We identified 28 ATC that were consecutively resected between 2003 and 2019. All tumour slides for each case were evaluated for the presence of a precursor tumour and for ATC morphology (sarcomatoid, pleomorphic giant cell, epithelioid or squamous). BRAF V600E mutation status was determined by BRAF V600E IHC or molecular analysis (OncoPanel NGS). Eighteen (64%) ATC had an associated well-differentiated precursor, including 10 (36%) with associated papillary thyroid carcinoma (PTC) and eight (29%) with associated follicular thyroid carcinoma (FTC) or Hürthle cell carcinoma (HCC). Most ATC (19 cases, 68%) demonstrated a mixed anaplastic morphology. Squamous morphology was present in four cases. Ten (36%) ATC had a BRAF V600E mutation. All ATC that had a PTC precursor had a BRAF V600E mutation (and all ATC with a BRAF V600E mutation had a PTC precursor), whereas no ATC with an FTC or HCC precursor had a BRAF V600E mutation. All four cases of ATC with a squamous morphology had a PTC precursor and a BRAF V600E mutation. CONCLUSION: In our cohort, the presence of a PTC precursor predicted the presence of the BRAF V600E mutation, whereas ATC with an FTC or HCC precursor lacked a BRAF V600E mutation. A squamous morphology was associated with the presence of a PTC precursor and a BRAF V600E mutation.

Clinicopathologic predictors of survival in buccal squamous cell carcinoma.

BACKGROUND: Buccal squamous cell carcinoma (SCC) is a locoregionally aggressive malignancy, representing a small subset of oral cancers in North America. We investigated the prognostic value of several clinicopathologic factors in a cohort of patients diagnosed with buccal SCC. METHODS: Between years 1992 and 2017, 52 patients were diagnosed with conventional buccal SCC. Archival surgical pathology material was retrospectively reviewed for reportable findings according to the latest reporting guidelines published by the College of American Pathologists. Clinical data were obtained through chart review. RESULTS: The majority of patients were of older age, current or past smokers, and without specific gender predilection. Most presented at a clinically advanced stage and were treated with surgery alone, or surgery followed by adjuvant radiotherapy. The tumor recurred in about 40% of patients, and almost half of the patients died from the disease by the end of the follow-up period. The worst pattern of invasion (WPOI) was associated with greater depth of invasion (DOI) (P = .031) and perineural invasion (P < .001). In univariate analyses, older age (P = .004), positive nodal status (P = .047), lymphovascular invasion (P = .012), perineural invasion (P = .05), and WPOI-5 (P = .015) were adverse predictors of 5-year overall survival (OS). In multivariate analysis, older age (P = .011), WPOI-5 (P < .001), and perineural invasion (P = .001) remained statistically significant independent prognosticators of worse 5-year OS. CONCLUSIONS: Older age, WPOI-5, and perineural invasion are significant prognosticators of worse OS. WPOI is associated with DOI, a finding which may have important implications for the pathogenesis and biologic behavior of the disease.

A potential diagnostic pitfall for hobnail variant of papillary thyroid carcinoma.

AIMS: Hobnail variant of papillary thyroid carcinoma (PTC) is an aggressive PTC subtype characterised by a hobnail cytomorphology. However, some classic PTC have a 'hobnail-like' cytomorphology associated with thick, hyalinised, variably oedematous fibrovascular cores that appears to be a form of ischaemic/degenerative atypia. METHODS AND RESULTS: We studied three cohorts to compare the histopathological characteristics and clinical outcome of 'hobnail-like' classic PTC and true hobnail variant of PTC: cohort 1, PTC consecutively resected between 2016 and 2017 (to assess frequency of 'hobnail-like' cytomorphology); cohort 2, 20 'hobnail-like' classic PTC resected between 2005 and 2007 (to assess clinical outcome); and cohort 3, seven true hobnail variant of PTC. A 'hobnail-like' cytomorphology was identified in 16% of consecutively resected PTC. Compared with true hobnail variant, 'hobnail-like' classic PTC occurred in younger patients (mean age 40 years versus 68 years, P < 0.001), were smaller tumours (mean tumour size 2.1 cm versus 4.4 cm, P < 0.001), had a lower rate of gross extrathyroidal extension (0% versus 71%, P < 0.001), had a lower proliferative rate (≥3 mitoses per 10 high-power fields seen in 0% versus 71%, P < 0.001; Ki67 index ≥5% in 0% versus 86%, P < 0.001), a lower rate of secondary pathogenic mutations (for cases with molecular data, 0% versus 100%, P = 0.0061) and improved survival (for cases with sufficient clinical outcome data, 10-year disease-free survival of 93% versus 0%, P = 0.0016). CONCLUSION: Classic PTC can show ischaemic/degenerative atypia that mimics the hobnail cytomorphology of true hobnail variant; however, these tumours lack aggressive histopathological features and pursue an indolent clinical course.

The Effect of OnabotulinumtoxinA on Aura Frequency and Severity in Patients With Hemiplegic Migraine: Case Series of 11 Patients.

BACKGROUND: OnabotulinumtoxinA has been demonstrated to be effective for the preventive treatment of headache in individuals with chronic migraine and has been approved and recommended for this patient population. While the therapeutic effect of onabotulinumtoxinA on migraine headache is well documented, there is limited information on the effect of onabotulinumtoxinA on migraine aura. OBJECTIVE: Given the prolonged and often debilitating nature of aura in patients with hemiplegic migraine, our group sought to examine the effect of onabotulinumtoxinA on aura frequency and severity in this specific subset of migraine patients. METHODS: All clinical notes from July 1, 1994 to December 1, 2017 at our institution were retrospectively reviewed to identify patients diagnosed with hemiplegic migraine who received at least one set of onabotulinumtoxinA injections. Thirty-four patients were identified; and of those, 23 were excluded due to incomplete documentation regarding aura symptoms at follow-up visits. The clinical notes of the remaining 11 patients (4 with familial hemiplegic migraine [FHM] and 7 with sporadic hemiplegic migraine [SHM]) were reviewed, and the frequency and description of their headaches and aura before and after receiving onabotulinumtoxinA were recorded. RESULTS: Nine of the 11 patients in our series noted a decrease in the frequency, severity, and/or duration of their aura after receiving onabotulinumtoxinA. Of the 2 non-responders, one was FHM and one was SHM. Of the 9 that noticed improved aura symptoms, 6 described a "wearing off" effect of onabotulinumtoxinA around week 9 or 10 of the 12-week cycle, with subsequent improvement after the next round. CONCLUSION: For 9 of 11 patients with hemiplegic migraine, onabotulinumtoxinA was helpful not only in reducing the frequency and severity of headaches but also in reducing the frequency and severity of the aura. In this manuscript, we speculate on potential pathophysiologic mechanisms that could have contributed to this effect.

Fibroepithelial polyp causing urethral obstruction: Diagnosis by cystourethrogram.

Fibroepithelial polyps of the urethra are rare benign tumors that predominantly affect males in childhood or adolescence. In this report, we present a case of a 3-year-old boy in acute urinary retention with a urethral fibroepithelial polyp manifesting as a large filling defect on voiding cystourethrogram and successfully managed endoscopically with transurethral resection.

Evaluating the validity of subclassifying warfarin-associated nonuremic calciphylaxis: a retrospective cohort study.

BACKGROUND: Calciphylaxis is a devastating multifactorial disorder of the subcutaneous fat that is known to be associated with hypercoagulability. Recent literature has proposed subclassifying patients with calciphylaxis as having warfarin-associated or warfarin-unassociated disease. AIM: We aimed to determine whether patients with warfarin-associated calciphylaxis differ clinically from patients with warfarin-unassociated calciphylaxis. MATERIALS AND METHODS: We performed a subgroup analysis of patients with nonuremic calciphylaxis from a previously studied cohort and compared clinical and outcomes features of patients who were taking warfarin at the time of disease onset to those of patients who were not. RESULTS: Nineteen patients with nonuremic calciphylaxis were identified, including 10 (53%) who had been on warfarin at the time of disease onset and 9 (47%) who had not. Of all clinical and outcomes parameters tested, no significant differences were detected between the two groups. DISCUSSION AND CONCLUSIONS: Though this study is limited by its retrospective nature and the relatively small number of patients studied, available data do not support subclassifying patients with nonuremic calciphylaxis as having warfarin-associated or warfarin-unassociated disease. Rather, the body of literature would suggest that identification and correction of underlying disorders of hypercoagulability should be prioritized.

Histopathology of Calciphylaxis: Cohort Study With Clinical Correlations.

Calciphylaxis is a rare, painful, and life-threatening condition with a high mortality rate. Although the etiology of calciphylaxis is not well understood, it has been proposed that calcium deposition within and around subcutaneous vessels restricts blood flow chronically, thereby predisposing the patient to acute pannicular and dermal thrombosis. Given increasing recognition of the role of hypercoagulability in calciphylaxis, this retrospective cohort study sought to evaluate the presence of thromboses and dermal angioplasia in calciphylaxis. Moreover, we aimed to validate previous observations about the histopathology of calciphylaxis compared with skin biopsies from patients with end-stage renal disease but without calciphylaxis. After a meticulous clinical chart review, we assessed the corresponding skin biopsies for the presence of vessel calcification, thromboses, and dermal angioplasia in skin biopsies from patients with calciphylaxis (n = 57) and compared with those from patients with end-stage renal disease but without calciphylaxis (n = 26). Histopathologic findings were correlated with clinical features such as chronic kidney disease, diabetes, or associated malignancy. Our results validated a prior observation that calciphylaxis was significantly more likely to show calcification of dermal vessels and diffuse dermal thrombi. This study reports the frequent finding of dermal angioplasia, a potential marker of chronic low-grade ischemia, as another frequent microscopic finding in calciphylaxis. Among cases of calciphylaxis, histopathologic changes in patients with chronic kidney disease were indistinguishable from those in patients without chronic kidney disease, thereby implying a final common pathogenic pathway in both uremic and nonuremic calciphylaxis. In future, larger, prospective studies may be useful in validating these findings.