KRASFormer: a fully vision transformer-based framework for predicting KRAS gene mutations in histopathological images of colorectal cancer.

Detecting the Kirsten Rat Sarcoma Virus (KRAS) gene mutation is significant for colorectal cancer (CRC) patients. The KRAS gene encodes a protein involved in the epidermal growth factor receptor (EGFR) signaling pathway, and mutations in this gene can negatively impact the use of monoclonal antibodies in antiEGFR therapy and affect treatment decisions. Currently, commonly used methods like next-generation sequencing (NGS) identify KRAS mutations but are expensive, time-consuming, and may not be suitable for every cancer patient sample. To address these challenges, we have developed KRASFormer, a novel framework that predicts KRAS gene mutations from Haematoxylin and Eosin (H&E) stained WSIs that are widely available for most CRC patients. KRASFormer consists of two stages: the first stage filters out non-tumor regions and selects only tumour cells using a quality screening mechanism, and the second stage predicts the KRAS gene either 'wildtype' or 'mutant' using a Vision Transformer-based XCiT method. The XCiT employs cross-covariance attention to capture clinically meaningful long-range representations of textural patterns in tumour tissue and KRAS mutant cells. We evaluated the performance of the first stage using an independent CRC-5000 dataset, and the second stage included both The Cancer Genome Atlas colon and rectal cancer (TCGA-CRCDX) and in-house cohorts. The results of our experiments showed that the XCiT outperformed existing state-of-the-art methods, achieving AUCs for ROC curves of 0.691 and 0.653 on TCGA-CRC-DX and in-house datasets, respectively. Our findings emphasize three key consequences: the potential of using H&E-stained tissue slide images for predicting KRAS gene mutations as a cost-effective and time-efficient means for guiding treatment choice with CRC patients; the increase in performance metrics of a Transformer-based model; and the value of the collaboration between pathologists and data scientists in deriving a morphologically meaningful model.

Topographic analysis of pancreatic cancer by TMA and digital spatial profiling reveals biological complexity with potential therapeutic implications.

Pancreatic ductal adenocarcinoma (PDAC) remains one of the most lethal human malignancies. Tissue microarrays (TMA) are an established method of high throughput biomarker interrogation in tissues but may not capture histological features of cancer with potential biological relevance. Topographic TMAs (T-TMAs) representing pathophysiological hallmarks of cancer were constructed from representative, retrospective PDAC diagnostic material, including 72 individual core tissue samples. The T-TMA was interrogated with tissue hybridization-based experiments to confirm the accuracy of the topographic sampling, expression of pro-tumourigenic and immune mediators of cancer, totalling more than 750 individual biomarker analyses. A custom designed Next Generation Sequencing (NGS) panel and a spatial distribution-specific transcriptomic evaluation were also employed. The morphological choice of the pathophysiological hallmarks of cancer was confirmed by protein-specific expression. Quantitative analysis identified topography-specific patterns of expression in the IDO/TGF-ß axis; with a heterogeneous relationship of inflammation and desmoplasia across hallmark areas and a general but variable protein and gene expression of c-MET. NGS results highlighted underlying genetic heterogeneity within samples, which may have a confounding influence on the expression of a particular biomarker. T-TMAs, integrated with quantitative biomarker digital scoring, are useful tools to identify hallmark specific expression of biomarkers in pancreatic cancer.

Digital pathology for reporting histopathology samples, including cancer screening samples - definitive evidence from a multisite study.

AIMS: To conduct a definitive multicentre comparison of digital pathology (DP) with light microscopy (LM) for reporting histopathology slides including breast and bowel cancer screening samples. METHODS: A total of 2024 cases (608 breast, 607 GI, 609 skin, 200 renal) were studied, including 207 breast and 250 bowel cancer screening samples. Cases were examined by four pathologists (16 study pathologists across the four speciality groups), using both LM and DP, with the order randomly assigned and 6 weeks between viewings. Reports were compared for clinical management concordance (CMC), meaning identical diagnoses plus differences which do not affect patient management. Percentage CMCs were computed using logistic regression models with crossed random-effects terms for case and pathologist. The obtained percentage CMCs were referenced to 98.3% calculated from previous studies. RESULTS: For all cases LM versus DP comparisons showed the CMC rates were 99.95% [95% confidence interval (CI) = 99.90-99.97] and 98.96 (95% CI = 98.42-99.32) for cancer screening samples. In speciality groups CMC for LM versus DP showed: breast 99.40% (99.06-99.62) overall and 96.27% (94.63-97.43) for cancer screening samples; [gastrointestinal (GI) = 99.96% (99.89-99.99)] overall and 99.93% (99.68-99.98) for bowel cancer screening samples; skin 99.99% (99.92-100.0); renal 99.99% (99.57-100.0). Analysis of clinically significant differences revealed discrepancies in areas where interobserver variability is known to be high, in reads performed with both modalities and without apparent trends to either. CONCLUSIONS: Comparing LM and DP CMC, overall rates exceed the reference 98.3%, providing compelling evidence that pathologists provide equivalent results for both routine and cancer screening samples irrespective of the modality used.

True-T - Improving T-cell response quantification with holistic artificial intelligence based prediction in immunohistochemistry images.

The immune response associated with oncogenesis and potential oncological ther- apeutic interventions has dominated the field of cancer research over the last decade. T-cell lymphocytes in the tumor microenvironment are a crucial aspect of cancer's adaptive immunity, and the quantification of T-cells in specific can- cer types has been suggested as a potential diagnostic aid. However, this is cur- rently not part of routine diagnostics. To address this challenge, we present a new method called True-T, which employs artificial intelligence-based techniques to quantify T-cells in colorectal cancer (CRC) using immunohistochemistry (IHC) images. True-T analyses the chromogenic tissue hybridization signal of three widely recognized T-cell markers (CD3, CD4, and CD8). Our method employs a pipeline consisting of three stages: T-cell segmentation, density estimation from the segmented mask, and prediction of individual five-year survival rates. In the first stage, we utilize the U-Net method, where a pre-trained ResNet-34 is em- ployed as an encoder to extract clinically relevant T-cell features. The segmenta- tion model is trained and evaluated individually, demonstrating its generalization in detecting the CD3, CD4, and CD8 biomarkers in IHC images. In the second stage, the density of T-cells is estimated using the predicted mask, which serves as a crucial indicator for patient survival statistics in the third stage. This ap- proach was developed and tested in 1041 patients from four reference diagnostic institutions, ensuring broad applicability. The clinical effectiveness of True-T is demonstrated in stages II-IV CRC by offering valuable prognostic information that surpasses previous quantitative gold standards, opening possibilities for po- tential clinical applications. Finally, to evaluate the robustness and broader ap- plicability of our approach without additional training, we assessed the universal accuracy of the CD3 component of the True-T algorithm across 13 distinct solid tumors.

Encrypted federated learning for secure decentralized collaboration in cancer image analysis.

Artificial intelligence (AI) has a multitude of applications in cancer research and oncology. However, the training of AI systems is impeded by the limited availability of large datasets due to data protection requirements and other regulatory obstacles. Federated and swarm learning represent possible solutions to this problem by collaboratively training AI models while avoiding data transfer. However, in these decentralized methods, weight updates are still transferred to the aggregation server for merging the models. This leaves the possibility for a breach of data privacy, for example by model inversion or membership inference attacks by untrusted servers. Somewhat-homomorphically-encrypted federated learning (SHEFL) is a solution to this problem because only encrypted weights are transferred, and model updates are performed in the encrypted space. Here, we demonstrate the first successful implementation of SHEFL in a range of clinically relevant tasks in cancer image analysis on multicentric datasets in radiology and histopathology. We show that SHEFL enables the training of AI models which outperform locally trained models and perform on par with models which are centrally trained. In the future, SHEFL can enable multiple institutions to co-train AI models without forsaking data governance and without ever transmitting any decryptable data to untrusted servers.

Translation of tissue-based artificial intelligence into clinical practice: from discovery to adoption.

Digital pathology (DP), or the digitization of pathology images, has transformed oncology research and cancer diagnostics. The application of artificial intelligence (AI) and other forms of machine learning (ML) to these images allows for better interpretation of morphology, improved quantitation of biomarkers, introduction of novel concepts to discovery and diagnostics (such as spatial distribution of cellular elements), and the promise of a new paradigm of cancer biomarkers. The application of AI to tissue analysis can take several conceptual approaches, within the domains of language modelling and image analysis, such as Deep Learning Convolutional Neural Networks, Multiple Instance Learning approaches, or the modelling of risk scores and their application to ML. The use of different approaches solves different problems within pathology workflows, including assistive applications for the detection and grading of tumours, quantification of biomarkers, and the delivery of established and new image-based biomarkers for treatment prediction and prognostic purposes. All these AI formats, applied to digital tissue images, are also beginning to transform our approach to clinical trials. In parallel, the novelty of DP/AI devices and the related computational science pipeline introduces new requirements for manufacturers to build into their design, development, regulatory and post-market processes, which may need to be taken into account when using AI applied to tissues in cancer discovery. Finally, DP/AI represents challenge to the way we accredit new diagnostic tools with clinical applicability, the understanding of which will allow cancer patients to have access to a new generation of complex biomarkers.

Multimodal analysis and the oncology patient: Creating a hospital system for integrated diagnostics and discovery.

We propose that an information technology and computational framework that would unify access to hospital digital information silos, and enable integration of this information using machine learning methods, would bring a new paradigm to patient management and research. This is the core principle of Integrated Diagnostics (ID): the amalgamation of multiple analytical modalities, with evolved information technology, applied to a defined patient cohort, and resulting in a synergistic effect in the clinical value of the individual diagnostic tools. This has the potential to transform the practice of personalized oncology at a time at which it is very much needed. In this article we present different models from the literature that contribute to the vision of ID and we provide published exemplars of ID tools. We briefly describe ongoing efforts within a universal healthcare system to create national clinical datasets. Following this, we argue the case to create "hospital units" to leverage this multi-modal analysis, data integration and holistic clinical decision-making. Finally, we describe the joint model created in our institutions.

Exploring the immune microenvironment in small bowel adenocarcinoma using digital image analysis.

BACKGROUND: Small bowel adenocarcinoma (SBA) is a rare malignancy of the small intestine associated with late stage diagnosis and poor survival outcome. High expression of immune cells and immune checkpoint biomarkers especially programmed cell death ligand-1 (PD-L1) have been shown to significantly impact disease progression. We have analysed the expression of a subset of immune cell and immune checkpoint biomarkers in a cohort of SBA patients and assessed their impact on progression-free survival (PFS) and overall survival (OS). METHODS: 25 patient samples in the form of formalin fixed, paraffin embedded (FFPE) tissue were obtained in tissue microarray (TMAs) format. Automated immunohistochemistry (IHC) staining was performed using validated antibodies for CD3, CD4, CD8, CD68, PD-L1, ICOS, IDO1 and LAG3. Slides were scanned digitally and assessed in QuPath, an open source image analysis software, for biomarker density and percentage positivity. Survival analyses were carried out using the Kaplan Meier method. RESULTS: Varying expressions of biomarkers were recorded. High expressions of CD3, CD4 and IDO1 were significant for PFS (p = 0.043, 0.020 and 0.018 respectively). High expression of ICOS was significant for both PFS (p = 0.040) and OS (p = 0.041), while high PD-L1 expression in tumour cells was significant for OS (p = 0.033). High correlation was observed between PD-L1 and IDO1 expressions (Pearson correlation co-efficient = 1) and subsequently high IDO1 expression in tumour cells was found to be significant for PFS (p = 0.006) and OS (p = 0.034). CONCLUSIONS: High levels of immune cells and immune checkpoint proteins have a significant impact on patient survival in SBA. These data could provide an insight into the immunotherapeutic management of patients with SBA.

Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study.

Deep learning (DL) can accelerate the prediction of prognostic biomarkers from routine pathology slides in colorectal cancer (CRC). However, current approaches rely on convolutional neural networks (CNNs) and have mostly been validated on small patient cohorts. Here, we develop a new transformer-based pipeline for end-to-end biomarker prediction from pathology slides by combining a pre-trained transformer encoder with a transformer network for patch aggregation. Our transformer-based approach substantially improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training and evaluating on a large multicenter cohort of over 13,000 patients from 16 colorectal cancer cohorts, we achieve a sensitivity of 0.99 with a negative predictive value of over 0.99 for prediction of microsatellite instability (MSI) on surgical resection specimens. We demonstrate that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem.

Molecular profiling of aromatase inhibitor sensitive and resistant ER+HER2- postmenopausal breast cancers.

Aromatase inhibitors (AIs) reduce recurrences and mortality in postmenopausal patients with oestrogen receptor positive (ER+) breast cancer (BC), but >20% of patients will eventually relapse. Given the limited understanding of intrinsic resistance in these tumours, here we conduct a large-scale molecular analysis to identify features that impact on the response of ER + HER2- BC to AI. We compare the 15% of poorest responders (PRs, n = 177) as measured by proportional Ki67 changes after 2 weeks of neoadjuvant AI to good responders (GRs, n = 190) selected from the top 50% responders in the POETIC trial and matched for baseline Ki67 categories. In this work, low ESR1 levels are associated with poor response, high proliferation, high expression of growth factor pathways and non-luminal subtypes. PRs having high ESR1 expression have similar proportions of luminal subtypes to GRs but lower plasma estradiol levels, lower expression of estrogen response genes, higher levels of tumor infiltrating lymphocytes and immune markers, and more TP53 mutations.

Transcriptional Profiling of a Patient-Matched Cohort of Glioblastoma (IDH-Wildtype) for Therapeutic Target and Repurposing Drug Identification.

Glioblastoma (GBM) is the most prevalent and aggressive adult brain tumor. Despite multi-modal therapies, GBM recurs, and patients have poor survival (~14 months). Resistance to therapy may originate from a subpopulation of tumor cells identified as glioma-stem cells (GSC), and new treatments are urgently needed to target these. The biology underpinning GBM recurrence was investigated using whole transcriptome profiling of patient-matched initial and recurrent GBM (recGBM). Differential expression analysis identified 147 significant probes. In total, 24 genes were validated using expression data from four public cohorts and the literature. Functional analyses revealed that transcriptional changes to recGBM were dominated by angiogenesis and immune-related processes. The role of MHC class II proteins in antigen presentation and the differentiation, proliferation, and infiltration of immune cells was enriched. These results suggest recGBM would benefit from immunotherapies. The altered gene signature was further analyzed in a connectivity mapping analysis with QUADrATiC software to identify FDA-approved repurposing drugs. Top-ranking target compounds that may be effective against GSC and GBM recurrence were rosiglitazone, nizatidine, pantoprazole, and tolmetin. Our translational bioinformatics pipeline provides an approach to identify target compounds for repurposing that may add clinical benefit in addition to standard therapies against resistant cancers such as GBM.

Subcellular Epithelial HMGB1 Expression Is Associated with Colorectal Neoplastic Progression, Male Sex, Mismatch Repair Protein Expression, Lymph Node Positivity, and an 'Immune Cold' Phenotype Associated with Poor Survival.

New treatment targets are needed for colorectal cancer (CRC). We define expression of High Mobility Group Box 1 (HMGB1) protein throughout colorectal neoplastic progression and examine the biological consequences of aberrant expression. HMGB1 is a ubiquitously expressed nuclear protein that shuttles to the cytoplasm under cellular stress. HMGB1 impacts cellular responses, acting as a cytokine when secreted. A total of 846 human tissue samples were retrieved; 6242 immunohistochemically stained sections were reviewed. Subcellular epithelial HMGB1 expression was assessed in a CRC Tissue Microarray (n = 650), normal colonic epithelium (n = 75), adenomatous polyps (n = 52), and CRC polyps (CaP, n = 69). Stromal lymphocyte phenotype was assessed in the CRC microarray and a subgroup of CaP. Normal colonic epithelium has strong nuclear and absent cytoplasmic HMGB1. With progression to CRC, there is an emergence of strong cytoplasmic HMGB1 (p < 0.001), pronounced at the leading cancer edge within CaP (p < 0.001), and reduction in nuclear HMGB1 (p < 0.001). In CRC, absent nuclear HMGB1 is associated with mismatch repair proteins (p = 0.001). Stronger cytoplasmic HMGB1 is associated with lymph node positivity (p < 0.001) and male sex (p = 0.009). Stronger nuclear (p = 0.011) and cytoplasmic (p = 0.002) HMGB1 is associated with greater CD4+ T-cell density, stronger nuclear HMGB1 is associated with greater FOXP3+ (p < 0.001) and ICOS+ (p = 0.018) lymphocyte density, and stronger nuclear HMGB1 is associated with reduced CD8+ T-cell density (p = 0.022). HMGB1 does not directly impact survival but is associated with an 'immune cold' tumour microenvironment which is associated with poor survival (p < 0.001). HMGB1 may represent a new treatment target for CRC.

Technical note on the exploration of COVID-19 in autopsy material.

Interrogation of immune response in autopsy material from patients with SARS-CoV-2 is potentially significant. We aim to describe a validated protocol for the exploration of the molecular physiopathology of SARS-CoV-2 pulmonary disease using multiplex immunofluorescence (mIF).The application of validated assays for the detection of SARS-CoV-2 in tissues, originally developed in our laboratory in the context of oncology, was used to map the topography and complexity of the adaptive immune response at protein and mRNA levels.SARS-CoV-2 is detectable in situ by protein or mRNA, with a sensitivity that could be in part related to disease stage. In formalin-fixed, paraffin-embedded pneumonia material, multiplex immunofluorescent panels are robust, reliable and quantifiable and can detect topographic variations in inflammation related to pathological processes.Clinical autopsies have relevance in understanding diseases of unknown/complex pathophysiology. In particular, autopsy materials are suitable for the detection of SARS-CoV-2 and for the topographic description of the complex tissue-based immune response using mIF.

Clinical Trials and Digital Pathology-Toward Quantitative Therapeutic Immunohistochemistry and Tissue Hybridization.

This Viewpoint discusses immunohistochemistry and tissue-hybridization-based diagnostics delivery and compares it with digital pathology.

ICOSeg: Real-Time ICOS Protein Expression Segmentation from Immunohistochemistry Slides Using a Lightweight Conv-Transformer Network.

In this article, we propose ICOSeg, a lightweight deep learning model that accurately segments the immune-checkpoint biomarker, Inducible T-cell COStimulator (ICOS) protein in colon cancer from immunohistochemistry (IHC) slide patches. The proposed model relies on the MobileViT network that includes two main components: convolutional neural network (CNN) layers for extracting spatial features; and a transformer block for capturing a global feature representation from IHC patch images. The ICOSeg uses an encoder and decoder sub-network. The encoder extracts the positive cell's salient features (i.e., shape, texture, intensity, and margin), and the decoder reconstructs important features into segmentation maps. To improve the model generalization capabilities, we adopted a channel attention mechanism that added to the bottleneck of the encoder layer. This approach highlighted the most relevant cell structures by discriminating between the targeted cell and background tissues. We performed extensive experiments on our in-house dataset. The experimental results confirm that the proposed model achieves more significant results against state-of-the-art methods, together with an 8× reduction in parameters.

General Roadmap and Core Steps for the Development of AI Tools in Digital Pathology.

Integrating artificial intelligence (AI) tools in the tissue diagnostic workflow will benefit the pathologist and, ultimately, the patient. The generation of such AI tools has two parallel and yet interconnected processes, namely the definition of the pathologist's task to be delivered in silico, and the software development requirements. In this review paper, we demystify this process, from a viewpoint that joins experienced pathologists and data scientists, by proposing a general pathway and describing the core steps to build an AI digital pathology tool. In doing so, we highlight the importance of the collaboration between AI scientists and pathologists, from the initial formulation of the hypothesis to the final, ready-to-use product.

Swarm learning for decentralized artificial intelligence in cancer histopathology.

Artificial intelligence (AI) can predict the presence of molecular alterations directly from routine histopathology slides. However, training robust AI systems requires large datasets for which data collection faces practical, ethical and legal obstacles. These obstacles could be overcome with swarm learning (SL), in which partners jointly train AI models while avoiding data transfer and monopolistic data governance. Here, we demonstrate the successful use of SL in large, multicentric datasets of gigapixel histopathology images from over 5,000 patients. We show that AI models trained using SL can predict BRAF mutational status and microsatellite instability directly from hematoxylin and eosin (H&E)-stained pathology slides of colorectal cancer. We trained AI models on three patient cohorts from Northern Ireland, Germany and the United States, and validated the prediction performance in two independent datasets from the United Kingdom. Our data show that SL-trained AI models outperform most locally trained models, and perform on par with models that are trained on the merged datasets. In addition, we show that SL-based AI models are data efficient. In the future, SL can be used to train distributed AI models for any histopathology image analysis task, eliminating the need for data transfer.