Joining forces for pathology diagnostics with AI assistance: The EMPAIA initiative.

Over the past decade, artificial intelligence (AI) methods in pathology have advanced substantially. However, integration into routine clinical practice has been slow due to numerous challenges, including technical and regulatory hurdles in translating research results into clinical diagnostic products and the lack of standardized interfaces. The open and vendor-neutral EMPAIA initiative addresses these challenges. Here, we provide an overview of EMPAIA's achievements and lessons learned. EMPAIA integrates various stakeholders of the pathology AI ecosystem, i.e., pathologists, computer scientists, and industry. In close collaboration, we developed technical interoperability standards, recommendations for AI testing and product development, and explainability methods. We implemented the modular and open-source EMPAIA Platform and successfully integrated 14 AI-based image analysis apps from eight different vendors, demonstrating how different apps can use a single standardized interface. We prioritized requirements and evaluated the use of AI in real clinical settings with 14 different pathology laboratories in Europe and Asia. In addition to technical developments, we created a forum for all stakeholders to share information and experiences on digital pathology and AI. Commercial, clinical, and academic stakeholders can now adopt EMPAIA's common open-source interfaces, providing a unique opportunity for large-scale standardization and streamlining of processes. Further efforts are needed to effectively and broadly establish AI assistance in routine laboratory use. To this end, a sustainable infrastructure, the non-profit association EMPAIA International, has been established to continue standardization and support broad implementation and advocacy for an AI-assisted digital pathology future.

National Cancer Institute Imaging Data Commons: Toward Transparency, Reproducibility, and Scalability in Imaging Artificial Intelligence.

The remarkable advances of artificial intelligence (AI) technology are revolutionizing established approaches to the acquisition, interpretation, and analysis of biomedical imaging data. Development, validation, and continuous refinement of AI tools requires easy access to large high-quality annotated datasets, which are both representative and diverse. The National Cancer Institute (NCI) Imaging Data Commons (IDC) hosts large and diverse publicly available cancer image data collections. By harmonizing all data based on industry standards and colocalizing it with analysis and exploration resources, the IDC aims to facilitate the development, validation, and clinical translation of AI tools and address the well-documented challenges of establishing reproducible and transparent AI processing pipelines. Balanced use of established commercial products with open-source solutions, interconnected by standard interfaces, provides value and performance, while preserving sufficient agility to address the evolving needs of the research community. Emphasis on the development of tools, use cases to demonstrate the utility of uniform data representation, and cloud-based analysis aim to ease adoption and help define best practices. Integration with other data in the broader NCI Cancer Research Data Commons infrastructure opens opportunities for multiomics studies incorporating imaging data to further empower the research community to accelerate breakthroughs in cancer detection, diagnosis, and treatment. Published under a CC BY 4.0 license.

The NCI Imaging Data Commons as a platform for reproducible research in computational pathology.

BACKGROUND AND OBJECTIVES: Reproducibility is a major challenge in developing machine learning (ML)-based solutions in computational pathology (CompPath). The NCI Imaging Data Commons (IDC) provides >120 cancer image collections according to the FAIR principles and is designed to be used with cloud ML services. Here, we explore its potential to facilitate reproducibility in CompPath research. METHODS: Using the IDC, we implemented two experiments in which a representative ML-based method for classifying lung tumor tissue was trained and/or evaluated on different datasets. To assess reproducibility, the experiments were run multiple times with separate but identically configured instances of common ML services. RESULTS: The results of different runs of the same experiment were reproducible to a large extent. However, we observed occasional, small variations in AUC values, indicating a practical limit to reproducibility. CONCLUSIONS: We conclude that the IDC facilitates approaching the reproducibility limit of CompPath research (i) by enabling researchers to reuse exactly the same datasets and (ii) by integrating with cloud ML services so that experiments can be run in identically configured computing environments.

Digitization of Pathology Labs: A Review of Lessons Learned.

Pathology laboratories are increasingly using digital workflows. This has the potential of increasing laboratory efficiency, but the digitization process also involves major challenges. Several reports have been published describing the individual experiences of specific laboratories with the digitization process. However, a comprehensive overview of the lessons learned is still lacking. We provide an overview of the lessons learned for different aspects of the digitization process, including digital case management, digital slide reading, and computer-aided slide reading. We also cover metrics used for monitoring performance and pitfalls and corresponding values observed in practice. The overview is intended to help pathologists, information technology decision makers, and administrators to benefit from the experiences of others and to implement the digitization process in an optimal way to make their own laboratory future-proof.

Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology.

Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recommendations are missing. A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations on compiling test datasets. We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries? The recommendations are intended to help AI developers demonstrate the utility of their products and to help pathologists and regulatory agencies verify reported performance measures. Further research is needed to formulate criteria for sufficiently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic workflows in the future.

EMPAIA App Interface: An open and vendor-neutral interface for AI applications in pathology.

BACKGROUND AND OBJECTIVE: Artificial intelligence (AI) apps hold great potential to make pathological diagnoses more accurate and time efficient. Widespread use of AI in pathology is hampered by interface incompatibilities between pathology software. We studied the existing interfaces in order to develop the EMPAIA App Interface, an open standard for the integration of pathology AI apps. METHODS: The EMPAIA App Interface relies on widely-used web communication protocols and containerization. It consists of three parts: A standardized format to describe the semantics of an app, a mechanism to deploy and execute apps in computing environments, and a web API through which apps can exchange data with a host application. RESULTS: Five commercial AI app manufacturers successfully adapted their products to the EMPAIA App Interface and helped improve it with their feedback. Open source tools facilitate the adoption of the interface by providing reusable data access and scheduling functionality and enabling automatic validation of app compliance. CONCLUSIONS: Existing AI apps and pathology software can be adapted to the EMPAIA App Interface with little effort. It is a viable alternative to the proprietary interfaces of current software. If enough vendors join in, the EMPAIA App Interface can help to advance the use of AI in pathology.

Additional partial hepatectomy at the time of portal vein ligation accelerates the regeneration of the future liver remnant.

Portal vein ligation (PVL) has been adopted to induce hypertrophy of the future liver remnant (FLR) in patients with primarily irresectable liver tumor. However, regeneration of the FLR is not always sufficient to allow curative resection of the portally-deprived tumor-bearing liver lobe. We hypothesize that simultaneous hepatectomy (PHx) and PVL augments regeneration of the FLR and that the effect is related to the extent of the additional resection. Seventy-two Lewis rats were enrolled into 3 groups: 20%PVL + 70%PHx; 70%PVL + 20%PHx; 90%PVL. Animals were observed for 1, 2, 3 and 7 days postoperatively (n = 6/time point). Liver enzymes, caudate liver/body-weight-ratio, BrdU-proliferation-index (PI), proliferating-cell-nuclear-antigen (PCNA)-mRNA-expression level and autophagy-related-proteins were evaluated. Compared with 90% PVL, additional PHx induced significantly more hypertrophy during the observation time, which was confirmed by significantly higher PI and higher level of PCNA-mRNA expression. Similarly, the additional PHx induced more autophagy in the FLR compared with PVL alone. However, both effects were not clearly related to the extent of additional resection. Additional resection augmented liver regeneration and autophagy substantially compared with PVL alone. Therefore, we concluded that autophagy might play a critical role in regulating hepatocyte proliferation and the size of the FLR after simultaneous PVL + PHx.

NCI Imaging Data Commons.

The National Cancer Institute (NCI) Cancer Research Data Commons (CRDC) aims to establish a national cloud-based data science infrastructure. Imaging Data Commons (IDC) is a new component of CRDC supported by the Cancer Moonshot. The goal of IDC is to enable a broad spectrum of cancer researchers, with and without imaging expertise, to easily access and explore the value of deidentified imaging data and to support integrated analyses with nonimaging data. We achieve this goal by colocating versatile imaging collections with cloud-based computing resources and data exploration, visualization, and analysis tools. The IDC pilot was released in October 2020 and is being continuously populated with radiology and histopathology collections. IDC provides access to curated imaging collections, accompanied by documentation, a user forum, and a growing number of analysis use cases that aim to demonstrate the value of a data commons framework applied to cancer imaging research. SIGNIFICANCE: This study introduces NCI Imaging Data Commons, a new repository of the NCI Cancer Research Data Commons, which will support cancer imaging research on the cloud.

Artificial Intelligence in Pathology: From Prototype to Product.

Modern image analysis techniques based on artificial intelligence (AI) have great potential to improve the quality and efficiency of diagnostic procedures in pathology and to detect novel biomarkers. Despite thousands of published research papers on applications of AI in pathology, hardly any research implementations have matured into commercial products for routine use. Bringing an AI solution for pathology to market poses significant technological, business, and regulatory challenges. In this paper, we provide a comprehensive overview and advice on how to meet these challenges. We outline how research prototypes can be turned into a product-ready state and integrated into the IT infrastructure of clinical laboratories. We also discuss business models for profitable AI solutions and reimbursement options for computer assistance in pathology. Moreover, we explain how to obtain regulatory approval so that AI solutions can be launched as in vitro diagnostic medical devices. Thus, this paper offers computer scientists, software companies, and pathologists a road map for transforming prototypes of AI solutions into commercial products.

Size of portally deprived liver lobe after portal vein ligation and additional partial hepatectomy: Result of balancing proliferation and apoptosis.

The liver has the ability to maintain its total size by adjusting the size of the individual liver lobes differently in response to regeneration- and atrophy-stimuli. Portal vein ligation (PVL) drives the ligated lobe to undergo atrophy whereas partial hepatectomy (PHx) drives the total remnant liver to regenerate. We hypothesize that the size of the PVL-lobe is dependent on the balance between the extent of PVL and the extent of PHx inducing a complex interplay between hepatocyte proliferation, apoptosis and autophagy. Lewis-rats were subjected to either 20%PVL + 70%PHx or 70%PVL + 20%PHx. Control groups consisted of 20%PVL and 70%PVL. Liver lobe weight, BrdU-proliferation-index, proliferating-cell-nuclear-antigen-mRNA-expression level, apoptotic density and autophagy-related-proteins were investigated. The PVL-liver lobe adjusted its weight differently, increasing by 40% after 20%PVL + 70%PHx, but decreasing by 25% after 70%PVL + 20%PHx. Additional resection induced a low, but substantial size-dependent hepatocyte proliferation rate (maximal 6.3% and 3.6% vs. 0.3% and significantly suppressed apoptotic density in the deportalized-liver-lobe (3 and 14 cells/mm2 comparing with above 26 cells/mm2, p < 0.01). Autophagy was more activated in PVL-liver lobe after simultaneous PHx than after PVL only. In summary, atrophy of the PVL-liver lobe after simultaneous PHx was counteracted by promoting hepatocyte proliferation, inducing autophagy and suppressing apoptosis in a PHx-extent-dependent manner.

Automated density-based counting of FISH amplification signals for HER2 status assessment.

BACKGROUND: Automated image analysis can make quantification of FISH signals in histological sections more efficient and reproducible. Current detection-based methods, however, often fail to accurately quantify densely clustered FISH signals. METHODS: We propose a novel density-based approach to quantifying FISH signals. Instead of detecting individual signals, this approach quantifies FISH signals in terms of the integral over a density map predicted by Deep Learning. We apply the density-based approach to the task of counting and determining ratios of ERBB2 and CEN17 signals and compare it to common detection-based and area-based approaches. RESULTS: The ratios determined by our approach were strongly correlated with results obtained by manual annotation of individual FISH signals (Pearson's r = 0.907). In addition, they were highly consistent with cutoff-scores determined by a pathologist (balanced concordance = 0.971). The density-based approach generally outperformed the other approaches. Its superiority was particularly evident in the presence of dense signal clusters. CONCLUSIONS: The presented approach enables accurate and efficient automated quantification of FISH signals. Since signals in clusters can hardly be detected individually even by human observers, the density-based quantification performs better than detection-based approaches.

Deep learning nuclei detection: A simple approach can deliver state-of-the-art results.

BACKGROUND: Deep convolutional neural networks have become a widespread tool for the detection of nuclei in histopathology images. Many implementations share a basic approach that includes generation of an intermediate map indicating the presence of a nucleus center, which we refer to as PMap. Nevertheless, these implementations often still differ in several parameters, resulting in different detection qualities. METHODS: We identified several essential parameters and configured the basic PMap approach using combinations of them. We thoroughly evaluated and compared various configurations on multiple datasets with respect to detection quality, efficiency and training effort. RESULTS: Post-processing of the PMap was found to have the largest impact on detection quality. Also, two different network architectures were identified that improve either detection quality or runtime performance. The best-performing configuration yields f1-measures of 0.816 on H&E stained images of colorectal adenocarcinomas and 0.819 on Ki-67 stained images of breast tumor tissue. On average, it was fully trained in less than 15,000 iterations and processed 4.15 megapixels per second at prediction time. CONCLUSIONS: The basic PMap approach is greatly affected by certain parameters. Our evaluation provides guidance on their impact and best settings. When configured properly, this simple and efficient approach can yield equal detection quality as more complex and time-consuming state-of-the-art approaches.

Focused scores enable reliable discrimination of small differences in steatosis.

BACKGROUND: Automated image analysis enables quantitative measurement of steatosis in histological images. However, spatial heterogeneity of steatosis can make quantitative steatosis scores unreliable. To improve the reliability, we have developed novel scores that are "focused" on steatotic tissue areas. METHODS: Focused scores use concepts of tile-based hotspot analysis in order to compute statistics about steatotic tissue areas in an objective way. We evaluated focused scores on three data sets of images of rodent liver sections exhibiting different amounts of dietary-induced steatosis. The same evaluation was conducted with the standard steatosis score computed by most image analysis methods. RESULTS: The standard score reliably discriminated large differences in steatosis (intraclass correlation coefficient ICC = 0.86), but failed to discriminate small (ICC = 0.54) and very small (ICC = 0.14) differences. With an appropriate tile size, mean-based focused scores reliably discriminated large (ICC = 0.92), small (ICC = 0.86) and very small (ICC = 0.83) differences. Focused scores based on high percentiles showed promise in further improving the discrimination of very small differences (ICC = 0.93). CONCLUSIONS: Focused scores enable reliable discrimination of small differences in steatosis in histological images. They are conceptually simple and straightforward to use in research studies.

Data-Driven Discovery of Immune Contexture Biomarkers.

Background: Features characterizing the immune contexture (IC) in the tumor microenvironment can be prognostic and predictive biomarkers. Identifying novel biomarkers can be challenging due to complex interactions between immune and tumor cells and the abundance of possible features. Methods: We describe an approach for the data-driven identification of IC biomarkers. For this purpose, we provide mathematical definitions of different feature classes, based on cell densities, cell-to-cell distances, and spatial heterogeneity thereof. Candidate biomarkers are ranked according to their potential for the predictive stratification of patients. Results: We evaluated the approach on a dataset of colorectal cancer patients with variable amounts of microsatellite instability. The most promising features that can be explored as biomarkers were based on cell-to-cell distances and spatial heterogeneity. Both the tumor and non-tumor compartments yielded features that were potentially predictive for therapy response and point in direction of further exploration. Conclusion: The data-driven approach simplifies the identification of promising IC biomarker candidates. Researchers can take guidance from the described approach to accelerate their biomarker research.

Automated quantification of steatosis: agreement with stereological point counting.

BACKGROUND: Steatosis is routinely assessed histologically in clinical practice and research. Automated image analysis can reduce the effort of quantifying steatosis. Since reproducibility is essential for practical use, we have evaluated different analysis methods in terms of their agreement with stereological point counting (SPC) performed by a hepatologist. METHODS: The evaluation was based on a large and representative data set of 970 histological images from human patients with different liver diseases. Three of the evaluated methods were built on previously published approaches. One method incorporated a new approach to improve the robustness to image variability. RESULTS: The new method showed the strongest agreement with the expert. At 20× resolution, it reproduced steatosis area fractions with a mean absolute error of 0.011 for absent or mild steatosis and 0.036 for moderate or severe steatosis. At 10× resolution, it was more accurate than and twice as fast as all other methods at 20× resolution. When compared with SPC performed by two additional human observers, its error was substantially lower than one and only slightly above the other observer. CONCLUSIONS: The results suggest that the new method can be a suitable automated replacement for SPC. Before further improvements can be verified, it is necessary to thoroughly assess the variability of SPC between human observers.

Training Nuclei Detection Algorithms with Simple Annotations.

BACKGROUND: Generating good training datasets is essential for machine learning-based nuclei detection methods. However, creating exhaustive nuclei contour annotations, to derive optimal training data from, is often infeasible. METHODS: We compared different approaches for training nuclei detection methods solely based on nucleus center markers. Such markers contain less accurate information, especially with regard to nuclear boundaries, but can be produced much easier and in greater quantities. The approaches use different automated sample extraction methods to derive image positions and class labels from nucleus center markers. In addition, the approaches use different automated sample selection methods to improve the detection quality of the classification algorithm and reduce the run time of the training process. We evaluated the approaches based on a previously published generic nuclei detection algorithm and a set of Ki-67-stained breast cancer images. RESULTS: A Voronoi tessellation-based sample extraction method produced the best performing training sets. However, subsampling of the extracted training samples was crucial. Even simple class balancing improved the detection quality considerably. The incorporation of active learning led to a further increase in detection quality. CONCLUSIONS: With appropriate sample extraction and selection methods, nuclei detection algorithms trained on the basis of simple center marker annotations can produce comparable quality to algorithms trained on conventionally created training sets.

Intrahepatic Size Regulation in a Surgical Model: Liver Resection-Induced Liver Regeneration Counteracts the Local Atrophy following Simultaneous Portal Vein Ligation.

BACKGROUND/AIM: Liver size regulation is based on the balance between hepatic regeneration and atrophy. To achieve a better understanding of intrahepatic size regulation, we explored the size regulation of a portally deprived liver lobe on a liver subjected to concurrent portal vein ligation (PVL) and partial hepatectomy (PHx). MATERIALS AND METHODS: Using a surgical rat model consisting of right PVL (rPVL) plus 70% PHx, we evaluated the size regulation of liver lobes 1, 2, 3, and 7 days after the operation in terms of liver weight and hepatocyte proliferation. Portal hyperperfusion was confirmed by measuring portal flow. The portal vascular tree was visualized by injection of a contrast agent followed by CT imaging of explanted livers. Control groups consisted of 70% PHx, rPVL, and sham operation. RESULTS: The size of the ligated right lobe increased to 1.4-fold on postoperative day 7 when subjected to rPVL + 70% PHx. The right lobe increased to 3-fold when subjected to 70% PHx alone and decreased to 0.3-fold when subjected to rPVL only. The small but significant increase in liver weight after the combined procedure was accompanied by a low proliferative response. In contrast, hepatocyte proliferation was undetectable in the right lobe undergoing atrophy after PVL only. The caudate lobe in the rPVL + 70% PHx group increased to 4.6-fold, which is significantly more than in the other groups. This increase in liver weight was paralleled by persisting portal hyperperfusion and a prolonged proliferative phase of 3 days. CONCLUSIONS: A discontinued portal blood supply does not always result in atrophy of the ligated lobe. The concurrent regenerative stimulus induced by 70% PHx seemed to counteract the local atrophy after a simultaneously performed rPVL, leading to a low but prolonged regenerative response of the portally deprived liver lobe. This observation supports the conclusion that portal flow is not necessary for liver regeneration. The persisting portal hyperperfusion may be crucial for the specific kinetics of prolonged liver regeneration after rPVL + 70% PHx in the portally supplied caudate lobe. Both observations deserve more attention regarding the underlying mechanism in further studies.

Zonated quantification of steatosis in an entire mouse liver.

Many physiological processes and pathological conditions in livers are spatially heterogeneous, forming patterns at the lobular length scale or varying across the organ. Steatosis, a common liver disease characterized by lipids accumulating in hepatocytes, exhibits heterogeneity at both these spatial scales. The main goal of the present study was to provide a method for zonated quantification of the steatosis patterns found in an entire mouse liver. As an example application, the results were employed in a pharmacokinetics simulation. For the analysis, an automatic detection of the lipid vacuoles was used in multiple slides of histological serial sections covering an entire mouse liver. Lobuli were determined semi-automatically and zones were defined within the lobuli. Subsequently, the lipid content of each zone was computed. The steatosis patterns were found to be predominantly periportal, with a notable organ-scale heterogeneity. The analysis provides a quantitative description of the extent of steatosis in unprecedented detail. The resulting steatosis patterns were successfully used as a perturbation to the liver as part of an exemplary whole-body pharmacokinetics simulation for the antitussive drug dextromethorphan. The zonated quantification is also applicable to other pathological conditions that can be detected in histological images. Besides being a descriptive research tool, this quantification could perspectively complement diagnosis based on visual assessment of histological images.

Stain Specific Standardization of Whole-Slide Histopathological Images.

Variations in the color and intensity of hematoxylin and eosin (H&E) stained histological slides can potentially hamper the effectiveness of quantitative image analysis. This paper presents a fully automated algorithm for standardization of whole-slide histopathological images to reduce the effect of these variations. The proposed algorithm, called whole-slide image color standardizer (WSICS), utilizes color and spatial information to classify the image pixels into different stain components. The chromatic and density distributions for each of the stain components in the hue-saturation-density color model are aligned to match the corresponding distributions from a template whole-slide image (WSI). The performance of the WSICS algorithm was evaluated on two datasets. The first originated from 125 H&E stained WSIs of lymph nodes, sampled from 3 patients, and stained in 5 different laboratories on different days of the week. The second comprised 30 H&E stained WSIs of rat liver sections. The result of qualitative and quantitative evaluations using the first dataset demonstrate that the WSICS algorithm outperforms competing methods in terms of achieving color constancy. The WSICS algorithm consistently yields the smallest standard deviation and coefficient of variation of the normalized median intensity measure. Using the second dataset, we evaluated the impact of our algorithm on the performance of an already published necrosis quantification system. The performance of this system was significantly improved by utilizing the WSICS algorithm. The results of the empirical evaluations collectively demonstrate the potential contribution of the proposed standardization algorithm to improved diagnostic accuracy and consistency in computer-aided diagnosis for histopathology data.