Model-Agnostic Binary Patch Grouping for Bone Marrow Whole Slide Image Representation.

Histopathology is the reference standard for pathology diagnosis, and has evolved with the digitization of glass slides [ie, whole slide images (WSIs)]. While trained histopathologists are able to diagnose diseases by examining WSIs visually, this process is time consuming and prone to variability. To address these issues, artificial intelligence models are being developed to generate slide-level representations of WSIs, summarizing the entire slide as a single vector. This enables various computational pathology applications, including interslide search, multimodal training, and slide-level classification. Achieving expressive and robust slide-level representations hinges on patch feature extraction and aggregation steps. This study proposed an additional binary patch grouping (BPG) step, a plugin that can be integrated into various slide-level representation pipelines, to enhance the quality of slide-level representation in bone marrow histopathology. BPG excludes patches with less clinical relevance through minimal interaction with the pathologist; a one-time human intervention for the entire process. This study further investigated domain-general versus domain-specific feature extraction models based on convolution and attention and examined two different feature aggregation methods, with and without BPG, showing BPG's generalizability. The results showed that using BPG boosts the performance of WSI retrieval (mean average precision at 10) by 4% and improves WSI classification (weighted-F1) by 5% compared to not using BPG. Additionally, domain-general large models and parameterized pooling produced the best-quality slide-level representations.

Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities.

This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.

Recent Advances in the Biology and CD123-Directed Treatment of Blastic Plasmacytoid Dendritic Cell Neoplasm.

Blastic plasmacytoid dendritic cell neoplasm (BPDCN) is an aggressive myeloid malignancy of the dendritic cell lineage that affects patients of all ages, though the incidence appears to be highest in patients over the age of 60 years. Diagnosis is based on the presence of plasmacytoid dendritic cell precursors expressing CD123, the interleukin-3 (IL-3) receptor alpha, and a distinct histologic appearance. Timely diagnosis remains a challenge, due to lack of disease awareness and overlapping biologic and clinical features with other hematologic malignancies. Prognosis is poor with a median overall survival of 8 to 14 months, irrespective of disease presentation pattern. Historically, the principal treatment was remission induction therapy followed by a stem cell transplant (SCT) in eligible patients. However, bridging to SCT is often not achieved with induction chemotherapy regimens. The discovery that CD123 is universally expressed in BPDCN and is considered to have a pathogenetic role in its development paved the way for the successful introduction of tagraxofusp, a recombinant human IL-3 fused to a truncated diphtheria toxin payload, as an initial treatment for BPDCN. Tagraxofusp was approved in 2018 by the United States Food and Drug Administration for the treatment of patients aged 2 years and older with newly diagnosed and relapsed/refractory BPDCN, and by the European Medicines Agency in 2021 for first-line treatment of adults. The advent of tagraxofusp has opened a new era of precision oncology in the treatment of BPDCN. Herein, we present an overview of BPDCN biology, its diagnosis, and treatment options, illustrated by clinical cases.

Performance of externally validated machine learning models based on histopathology images for the diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer: A systematic review.

Numerous machine learning (ML) models have been developed for breast cancer using various types of data. Successful external validation (EV) of ML models is important evidence of their generalizability. The aim of this systematic review was to assess the performance of externally validated ML models based on histopathology images for diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer. A systematic search of MEDLINE, EMBASE, CINAHL, IEEE, MICCAI, and SPIE conferences was performed for studies published between January 2010 and February 2022. The Prediction Model Risk of Bias Assessment Tool (PROBAST) was employed, and the results were narratively described. Of the 2011 non-duplicated citations, 8 journal articles and 2 conference proceedings met inclusion criteria. Three studies externally validated ML models for diagnosis, 4 for classification, 2 for prognosis, and 1 for both classification and prognosis. Most studies used Convolutional Neural Networks and one used logistic regression algorithms. For diagnostic/classification models, the most common performance metrics reported in the EV were accuracy and area under the curve, which were greater than 87% and 90%, respectively, using pathologists' annotations/diagnoses as ground truth. The hazard ratios in the EV of prognostic ML models were between 1.7 (95% CI, 1.2-2.6) and 1.8 (95% CI, 1.3-2.7) to predict distant disease-free survival; 1.91 (95% CI, 1.11-3.29) for recurrence, and between 0.09 (95% CI, 0.01-0.70) and 0.65 (95% CI, 0.43-0.98) for overall survival, using clinical data as ground truth. Despite EV being an important step before the clinical application of a ML model, it hasn't been performed routinely. The large variability in the training/validation datasets, methods, performance metrics, and reported information limited the comparison of the models and the analysis of their results. Increasing the availability of validation datasets and implementing standardized methods and reporting protocols may facilitate future analyses.

Whole slide image representation in bone marrow cytology.

One of the goals of AI-based computational pathology is to generate compact representations of whole slide images (WSIs) that capture the essential information needed for diagnosis. While such approaches have been applied to histopathology, few applications have been reported in cytology. Bone marrow aspirate cytology is the basis for key clinical decisions in hematology. However, visual inspection of aspirate specimens is a tedious and complex process subject to variation in interpretation, and hematopathology expertise is scarce. The ability to generate a compact representation of an aspirate specimen may form the basis for clinical decision-support tools in hematology. In this study, we leverage our previously published end-to-end AI-based system for counting and classifying cells from bone marrow aspirate WSIs, which enables the direct use of individual cells as inputs rather than WSI patches. We then construct bags of individual cell features from each WSI, and apply multiple instance learning to extract their vector representations. To evaluate the quality of our representations, we conducted WSI retrieval and classification tasks. Our results show that we achieved a mAP@10 of 0.58 ±0.02 in WSI-level image retrieval, surpassing the random-retrieval baseline of 0.39 ±0.1. Furthermore, we predicted five diagnostic labels for individual aspirate WSIs with a weighted-average F1 score of 0.57 ±0.03 using a k-nearest-neighbors (k-NN) model, outperforming guessing using empirical class prior probabilities (0.26 ±0.02). We present the first example of exploring trainable mechanisms to generate compact, slide-level representations in bone marrow cytology with deep learning. This method has the potential to summarize complex semantic information in WSIs toward improved diagnostics in hematology, and may eventually support AI-assisted computational pathology approaches.

Cell projection plots: A novel visualization of bone marrow aspirate cytology.

Deep models for cell detection have demonstrated utility in bone marrow cytology, showing impressive results in terms of accuracy and computational efficiency. However, these models have yet to be implemented in the clinical diagnostic workflow. Additionally, the metrics used to evaluate cell detection models are not necessarily aligned with clinical goals and targets. In order to address these issues, we introduce novel, automatically generated visual summaries of bone marrow aspirate specimens called cell projection plots (CPPs). Encompassing relevant biological patterns such as neutrophil maturation, CPPs provide a compact summary of bone marrow aspirate cytology. To gauge clinical relevance, CPPs were inspected by 3 hematopathologists, who decided whether corresponding diagnostic synopses matched with generated CPPs. Pathologists were able to match CPPs to the correct synopsis with a matching degree of 85%. Our finding suggests CPPs can represent clinically relevant information from bone marrow aspirate specimens and may be used to efficiently summarize bone marrow cytology to pathologists. CPPs could be a step toward human-centered implementation of artificial intelligence (AI) in hematopathology, and a basis for a diagnostic-support tool for digital pathology workflows.

Applied machine learning in hematopathology.

An increasing number of machine learning applications are being developed and applied to digital pathology, including hematopathology. The goal of these modern computerized tools is often to support diagnostic workflows by extracting and summarizing information from multiple data sources, including digital images of human tissue. Hematopathology is inherently multimodal and can serve as an ideal case study for machine learning applications. However, hematopathology also poses unique challenges compared to other pathology subspecialities when applying machine learning approaches. By modeling the pathologist workflow and thinking process, machine learning algorithms may be designed to address practical and tangible problems in hematopathology. In this article, we discuss the current trends in machine learning in hematopathology. We review currently available machine learning enabled medical devices supporting hematopathology workflows. We then explore current machine learning research trends of the field with a focus on bone marrow cytology and histopathology, and how adoption of new machine learning tools may be enabled through the transition to digital pathology.

Biased data, biased AI: deep networks predict the acquisition site of TCGA images.

BACKGROUND: Deep learning models applied to healthcare applications including digital pathology have been increasing their scope and importance in recent years. Many of these models have been trained on The Cancer Genome Atlas (TCGA) atlas of digital images, or use it as a validation source. One crucial factor that seems to have been widely ignored is the internal bias that originates from the institutions that contributed WSIs to the TCGA dataset, and its effects on models trained on this dataset. METHODS: 8,579 paraffin-embedded, hematoxylin and eosin stained, digital slides were selected from the TCGA dataset. More than 140 medical institutions (acquisition sites) contributed to this dataset. Two deep neural networks (DenseNet121 and KimiaNet were used to extract deep features at 20× magnification. DenseNet was pre-trained on non-medical objects. KimiaNet has the same structure but trained for cancer type classification on TCGA images. The extracted deep features were later used to detect each slide's acquisition site, and also for slide representation in image search. RESULTS: DenseNet's deep features could distinguish acquisition sites with 70% accuracy whereas KimiaNet's deep features could reveal acquisition sites with more than 86% accuracy. These findings suggest that there are acquisition site specific patterns that could be picked up by deep neural networks. It has also been shown that these medically irrelevant patterns can interfere with other applications of deep learning in digital pathology, namely image search. This study shows that there are acquisition site specific patterns that can be used to identify tissue acquisition sites without any explicit training. Furthermore, it was observed that a model trained for cancer subtype classification has exploited such medically irrelevant patterns to classify cancer types. Digital scanner configuration and noise, tissue stain variation and artifacts, and source site patient demographics are among factors that likely account for the observed bias. Therefore, researchers should be cautious of such bias when using histopathology datasets for developing and training deep networks.

Evolutionary deep feature selection for compact representation of gigapixel images in digital pathology.

Despite the recent progress in Deep Neural Networks (DNNs) to characterize histopathology images, compactly representing a gigapixel whole-slide image (WSI) via salient features to enable computational pathology is still an urgent need and a significant challenge. In this paper, we propose a novel WSI characterization approach to represent, search and classify biopsy specimens using a compact feature vector (CFV) extracted from a multitude of deep feature vectors. Since the non-optimal design and training of deep networks may result in many irrelevant and redundant features and also cause computational bottlenecks, we proposed a low-cost stochastic method to optimize the output of pre-trained deep networks using evolutionary algorithms to generate a very small set of features to accurately represent each tissue/biopsy. The performance of the proposed method has been assessed using WSIs from the publicly available TCGA image data. In addition to acquiring a very compact representation (i.e., 11,000 times smaller than the initial set of features), the optimized features achieved 93% classification accuracy resulting in 11% improvement compared to the published benchmarks. The experimental results reveal that the proposed method can reliably select salient features of the biopsy sample. Furthermore, the proposed approach holds the potential to immensely facilitate the adoption of digital pathology by enabling a new generation of WSI representation for efficient storage and more user-friendly visualization.

Automated bone marrow cytology using deep learning to generate a histogram of cell types.

Background: Bone marrow cytology is required to make a hematological diagnosis, influencing critical clinical decision points in hematology. However, bone marrow cytology is tedious, limited to experienced reference centers and associated with inter-observer variability. This may lead to a delayed or incorrect diagnosis, leaving an unmet need for innovative supporting technologies. Methods: We develop an end-to-end deep learning-based system for automated bone marrow cytology. Starting with a bone marrow aspirate digital whole slide image, our system rapidly and automatically detects suitable regions for cytology, and subsequently identifies and classifies all bone marrow cells in each region. This collective cytomorphological information is captured in a representation called Histogram of Cell Types (HCT) quantifying bone marrow cell class probability distribution and acting as a cytological patient fingerprint. Results: Our system achieves high accuracy in region detection (0.97 accuracy and 0.99 ROC AUC), and cell detection and cell classification (0.75 mean average precision, 0.78 average F1-score, Log-average miss rate of 0.31). Conclusions: HCT has potential to eventually support more efficient and accurate diagnosis in hematology, supporting AI-enabled computational pathology.

DMG26: A Targeted Sequencing Panel for Mutation Profiling to Address Gaps in the Prognostication of Multiple Myeloma.

Multiple myeloma presents with numerous primary genomic lesions that broadly dichotomize cases into hyperdiploidy or IgH translocated. Clinically, these large alterations are assessed by fluorescence in situ hybridization (FISH) for risk stratification at diagnosis. Secondary focal events, including indels and single-nucleotide variants, are also reported; however, their clinical correlates are poorly described, and FISH has insufficient resolution to assess many of them. This study examined the exonic sequences of 26 genes reported to be mutated in >1% of patients with myeloma using a custom panel. These exons were sequenced to approximately 1000 times in a cohort of 76 patients from Atlantic Canada with detailed clinical correlates and in four multiple myeloma cell lines. Across the 76 patients, 255 mutations and 33 focal copy number variations were identified. High-severity mutations and mutations predicted by FATHMM-XF to be pathogenic identified patients with significantly reduced progression-free survival. These mutations were mutually exclusive from the Revised International Staging System high-risk FISH markers and were independent of all biochemical parameters of the Revised International Staging System. Applying our panel to patients classified by FISH to be standard risk successfully reclassified patients into high- and standard-risk groups. Furthermore, three patients in our cohort each had two high-risk markers; two of these patients developed plasma cell leukemia, a rare and severe clinical sequela of multiple myeloma.

Searching Images for Consensus: Can AI Remove Observer Variability in Pathology?

One of the major obstacles in reaching diagnostic consensus is observer variability. With the recent success of artificial intelligence, particularly the deep networks, the question emerges as to whether the fundamental challenge of diagnostic imaging can now be resolved. This article briefly reviews the problem and how eventually both supervised and unsupervised AI technologies could help to overcome it.

A BERT model generates diagnostically relevant semantic embeddings from pathology synopses with active learning.

Background: Pathology synopses consist of semi-structured or unstructured text summarizing visual information by observing human tissue. Experts write and interpret these synopses with high domain-specific knowledge to extract tissue semantics and formulate a diagnosis in the context of ancillary testing and clinical information. The limited number of specialists available to interpret pathology synopses restricts the utility of the inherent information. Deep learning offers a tool for information extraction and automatic feature generation from complex datasets. Methods: Using an active learning approach, we developed a set of semantic labels for bone marrow aspirate pathology synopses. We then trained a transformer-based deep-learning model to map these synopses to one or more semantic labels, and extracted learned embeddings (i.e., meaningful attributes) from the model's hidden layer. Results: Here we demonstrate that with a small amount of training data, a transformer-based natural language model can extract embeddings from pathology synopses that capture diagnostically relevant information. On average, these embeddings can be used to generate semantic labels mapping patients to probable diagnostic groups with a micro-average F1 score of 0.779 Â ± 0.025. Conclusions: We provide a generalizable deep learning model and approach to unlock the semantic information inherent in pathology synopses toward improved diagnostics, biodiscovery and AI-assisted computational pathology.

Humanized zebrafish enhance human hematopoietic stem cell survival and promote acute myeloid leukemia clonal diversity.

Xenograft models are invaluable tools in establishing the current paradigms of hematopoiesis and leukemogenesis. The zebrafish has emerged as a robust alternative xenograft model but, like mice, lack specific cytokines that mimic the microenvironment found in human patients. To address this critical gap, we generated the first humanized zebrafish that express human hematopoietic-specific cytokines (GM-CSF, SCF, and SDF1α). Termed GSS fish, these zebrafish promote survival, self-renewal and multilineage differentiation of human hematopoietic stem and progenitor cells and result in enhanced proliferation and hematopoietic niche-specific homing of primary human leukemia cells. Using error-corrected RNA sequencing, we determined that patient-derived leukemias transplanted into GSS zebrafish exhibit broader clonal representation compared to transplants into control hosts. GSS zebrafish incorporating error-corrected RNA sequencing establish a new standard for zebrafish xenotransplantation that more accurately recapitulates the human context, providing a more representative cost-effective preclinical model system for evaluating personalized response-based treatment in leukemia and therapies to expand human hematopoietic stem and progenitor cells in the transplant setting.

Yottixel - An Image Search Engine for Large Archives of Histopathology Whole Slide Images.

With the emergence of digital pathology, searching for similar images in large archives has gained considerable attention. Image retrieval can provide pathologists with unprecedented access to the evidence embodied in already diagnosed and treated cases from the past. This paper proposes a search engine specialized for digital pathology, called Yottixel, a portmanteau for "one yotta pixel," alluding to the big-data nature of histopathology images. The most impressive characteristic of Yottixel is its ability to represent whole slide images (WSIs) in a compact manner. Yottixel can perform millions of searches in real-time with a high search accuracy and low storage profile. Yottixel uses an intelligent indexing algorithm capable of representing WSIs with a mosaic of patches which are then converted into barcodes, called "Bunch of Barcodes" (BoB), the most prominent performance enabler of Yottixel. The performance of the prototype platform is qualitatively tested using 300 WSIs from the University of Pittsburgh Medical Center (UPMC) and 2,020 WSIs from The Cancer Genome Atlas Program (TCGA) provided by the National Cancer Institute. Both datasets amount to more than 4,000,000 patches of 1000 × 1000 pixels. We report three sets of experiments that show that Yottixel can accurately retrieve organs and malignancies, and its semantic ordering shows good agreement with the subjective evaluation of human observers.

Pan-cancer diagnostic consensus through searching archival histopathology images using artificial intelligence.

The emergence of digital pathology has opened new horizons for histopathology. Artificial intelligence (AI) algorithms are able to operate on digitized slides to assist pathologists with different tasks. Whereas AI-involving classification and segmentation methods have obvious benefits for image analysis, image search represents a fundamental shift in computational pathology. Matching the pathology of new patients with already diagnosed and curated cases offers pathologists a new approach to improve diagnostic accuracy through visual inspection of similar cases and computational majority vote for consensus building. In this study, we report the results from searching the largest public repository (The Cancer Genome Atlas, TCGA) of whole-slide images from almost 11,000 patients. We successfully indexed and searched almost 30,000 high-resolution digitized slides constituting 16 terabytes of data comprised of 20 million 1000 × 1000 pixels image patches. The TCGA image database covers 25 anatomic sites and contains 32 cancer subtypes. High-performance storage and GPU power were employed for experimentation. The results were assessed with conservative "majority voting" to build consensus for subtype diagnosis through vertical search and demonstrated high accuracy values for both frozen section slides (e.g., bladder urothelial carcinoma 93%, kidney renal clear cell carcinoma 97%, and ovarian serous cystadenocarcinoma 99%) and permanent histopathology slides (e.g., prostate adenocarcinoma 98%, skin cutaneous melanoma 99%, and thymoma 100%). The key finding of this validation study was that computational consensus appears to be possible for rendering diagnoses if a sufficiently large number of searchable cases are available for each cancer subtype.

Low-depth sequencing for copy number abnormalities in multiple myeloma supersedes fluorescent in situ hybridization in scope and resolution.

Multiple myeloma (MM) is an incurable hematological malignancy that relies on cytogenetic determination of copy number abnormalities (CNAs) for prognosis and management. Low-depth whole genome sequencing (LD-WGS) is a cost-effective alternative to targeted genomics for CNA detection, but its value has yet to be explored in MM. DNA from CD138+ cells from MM patients were sequenced using an Illumina NextSeq at <1x depth (ultralow-depth). Subsampling analysis and window size adjustment were performed for determining sensitivity limits and results compared to fluorescent in-Situ hybridization (FISH). CNA calls made down to 5 million (M) reads were comparable to those at 20 M reads at a window size of 100 kb had a sensitivity and specificity of 93%, 92% and an area under the curve of 0.94. All CNAs detected by FISH on the MM samples were also detected by LD-WGS; the latter detected a further 36 focal CNAs not detected by FISH. Cost per sample of LD-WGS was significantly lower for our organization than FISH testing. LD-WGS for MM is significantly more sensitive than targeted technologies such as FISH in CNA detection and resolution, provides a more cost-effective option for clinical purposes and potential for exploring prognostically relevant and drug discovery targets.

Acute myeloid leukaemia disrupts endogenous myelo-erythropoiesis by compromising the adipocyte bone marrow niche.

Acute myeloid leukaemia (AML) is distinguished by the generation of dysfunctional leukaemic blasts, and patients characteristically suffer from fatal infections and anaemia due to insufficient normal myelo-erythropoiesis. Direct physical crowding of bone marrow (BM) by accumulating leukaemic cells does not fully account for this haematopoietic failure. Here, analyses from AML patients were applied to both in vitro co-culture platforms and in vivo xenograft modelling, revealing that human AML disease specifically disrupts the adipocytic niche in BM. Leukaemic suppression of BM adipocytes led to imbalanced regulation of endogenous haematopoietic stem and progenitor cells, resulting in impaired myelo-erythroid maturation. In vivo administration of PPARγ agonists induced BM adipogenesis, which rescued healthy haematopoietic maturation while repressing leukaemic growth. Our study identifies a previously unappreciated axis between BM adipogenesis and normal myelo-erythroid maturation that is therapeutically accessible to improve symptoms of BM failure in AML via non-cell autonomous targeting of the niche.

Niche displacement of human leukemic stem cells uniquely allows their competitive replacement with healthy HSPCs.

Allogeneic hematopoietic stem cell (HSC) transplantation (HSCT) is currently the leading strategy to manage acute myeloid leukemia (AML). However, treatment-related morbidity limits the patient generalizability of HSCT use, and the survival of leukemic stem cells (LSCs) within protective areas of the bone marrow (BM) continues to lead to high relapse rates. Despite growing appreciation for the significance of the LSC microenvironment, it has remained unresolved whether LSCs preferentially situate within normal HSC niches or whether their niche requirements are more promiscuous. Here, we provide functional evidence that the spatial localization of phenotypically primitive human AML cells is restricted to niche elements shared with their normal counterparts, and that their intrinsic ability to initiate and retain occupancy of these niches can be rivaled by healthy hematopoietic stem and progenitor cells (HSPCs). When challenged in competitive BM repopulation assays, primary human leukemia-initiating cells (L-ICs) can be consistently outperformed by HSPCs for BM niche occupancy in a cell dose-dependent manner that ultimately compromises long-term L-IC renewal and subsequent leukemia-initiating capacity. The effectiveness of this approach could be demonstrated using cytokine-induced mobilization of established leukemia from the BM that facilitated the replacement of BM niches with transplanted HSPCs. These findings identify a functional vulnerability of primitive leukemia cells, and suggest that clinical development of these novel transplantation techniques should focus on the dissociation of L-IC-niche interactions to improve competitive replacement with healthy HSPCs during HSCT toward increased survival of patients.

Regional localization within the bone marrow influences the functional capacity of human HSCs.

Numerous studies have shown that the bone marrow (BM) niche plays a key role in mouse hematopoietic stem cell (HSC) function and involves contributions from a broad array of cell types. However, the composition and role of the human BM HSC niche have not been investigated. Here, using human bone biopsy specimens, we provide evidence of HSC propensity to localize to endosteal regions of the trabecular bone area (TBA). Through functional xenograft transplantation, we found that human HSCs localizing to the TBA have superior regenerative and self-renewal capacity and are molecularly distinct from those localizing to the long bone area (LBA). In addition, osteoblasts in the TBA possess unique characteristics and express a key network of factors that regulate TBA- versus LBA-localized human HSCs in vivo. Our study reveals that BM localization and architecture play a critical role in defining the functional and molecular properties of human HSCs.