Cloud-based large-scale curation of medical imaging data using AI segmentation.

Rapid advances in medical imaging Artificial Intelligence (AI) offer unprecedented opportunities for automatic analysis and extraction of data from large imaging collections. Computational demands of such modern AI tools may be difficult to satisfy with the capabilities available on premises. Cloud computing offers the promise of economical access and extreme scalability. Few studies examine the price/performance tradeoffs of using the cloud, in particular for medical image analysis tasks. We investigate the use of cloud-provisioned compute resources for AI-based curation of the National Lung Screening Trial (NLST) Computed Tomography (CT) images available from the National Cancer Institute (NCI) Imaging Data Commons (IDC). We evaluated NCI Cancer Research Data Commons (CRDC) Cloud Resources - Terra (FireCloud) and Seven Bridges-Cancer Genomics Cloud (SB-CGC) platforms - to perform automatic image segmentation with TotalSegmentator and pyradiomics feature extraction for a large cohort containing >126,000 CT volumes from >26,000 patients. Utilizing >21,000 Virtual Machines (VMs) over the course of the computation we completed analysis in under 9 hours, as compared to the estimated 522 days that would be needed on a single workstation. The total cost of utilizing the cloud for this analysis was $1,011.05. Our contributions include: 1) an evaluation of the numerous tradeoffs towards optimizing the use of cloud resources for large-scale image analysis; 2) CloudSegmentator, an open source reproducible implementation of the developed workflows, which can be reused and extended; 3) practical recommendations for utilizing the cloud for large-scale medical image computing tasks. We also share the results of the analysis: the total of 9,565,554 segmentations of the anatomic structures and the accompanying radiomics features in IDC as of release v18.

Enrichment of lung cancer computed tomography collections with AI-derived annotations.

Public imaging datasets are critical for the development and evaluation of automated tools in cancer imaging. Unfortunately, many do not include annotations or image-derived features, complicating downstream analysis. Artificial intelligence-based annotation tools have been shown to achieve acceptable performance and can be used to automatically annotate large datasets. As part of the effort to enrich public data available within NCI Imaging Data Commons (IDC), here we introduce AI-generated annotations for two collections containing computed tomography images of the chest, NSCLC-Radiomics, and a subset of the National Lung Screening Trial. Using publicly available AI algorithms, we derived volumetric annotations of thoracic organs-at-risk, their corresponding radiomics features, and slice-level annotations of anatomical landmarks and regions. The resulting annotations are publicly available within IDC, where the DICOM format is used to harmonize the data and achieve FAIR (Findable, Accessible, Interoperable, Reusable) data principles. The annotations are accompanied by cloud-enabled notebooks demonstrating their use. This study reinforces the need for large, publicly accessible curated datasets and demonstrates how AI can aid in cancer imaging.

National Cancer Institute Imaging Data Commons: Toward Transparency, Reproducibility, and Scalability in Imaging Artificial Intelligence.

The remarkable advances of artificial intelligence (AI) technology are revolutionizing established approaches to the acquisition, interpretation, and analysis of biomedical imaging data. Development, validation, and continuous refinement of AI tools requires easy access to large high-quality annotated datasets, which are both representative and diverse. The National Cancer Institute (NCI) Imaging Data Commons (IDC) hosts large and diverse publicly available cancer image data collections. By harmonizing all data based on industry standards and colocalizing it with analysis and exploration resources, the IDC aims to facilitate the development, validation, and clinical translation of AI tools and address the well-documented challenges of establishing reproducible and transparent AI processing pipelines. Balanced use of established commercial products with open-source solutions, interconnected by standard interfaces, provides value and performance, while preserving sufficient agility to address the evolving needs of the research community. Emphasis on the development of tools, use cases to demonstrate the utility of uniform data representation, and cloud-based analysis aim to ease adoption and help define best practices. Integration with other data in the broader NCI Cancer Research Data Commons infrastructure opens opportunities for multiomics studies incorporating imaging data to further empower the research community to accelerate breakthroughs in cancer detection, diagnosis, and treatment. Published under a CC BY 4.0 license.

The NCI Imaging Data Commons as a platform for reproducible research in computational pathology.

BACKGROUND AND OBJECTIVES: Reproducibility is a major challenge in developing machine learning (ML)-based solutions in computational pathology (CompPath). The NCI Imaging Data Commons (IDC) provides >120 cancer image collections according to the FAIR principles and is designed to be used with cloud ML services. Here, we explore its potential to facilitate reproducibility in CompPath research. METHODS: Using the IDC, we implemented two experiments in which a representative ML-based method for classifying lung tumor tissue was trained and/or evaluated on different datasets. To assess reproducibility, the experiments were run multiple times with separate but identically configured instances of common ML services. RESULTS: The results of different runs of the same experiment were reproducible to a large extent. However, we observed occasional, small variations in AUC values, indicating a practical limit to reproducibility. CONCLUSIONS: We conclude that the IDC facilitates approaching the reproducibility limit of CompPath research (i) by enabling researchers to reuse exactly the same datasets and (ii) by integrating with cloud ML services so that experiments can be run in identically configured computing environments.

Interoperable slide microscopy viewer and annotation tool for imaging data science and computational pathology.

The exchange of large and complex slide microscopy imaging data in biomedical research and pathology practice is impeded by a lack of data standardization and interoperability, which is detrimental to the reproducibility of scientific findings and clinical integration of technological innovations. We introduce Slim, an open-source, web-based slide microscopy viewer that implements the internationally accepted Digital Imaging and Communications in Medicine (DICOM) standard to achieve interoperability with a multitude of existing medical imaging systems. We showcase the capabilities of Slim as the slide microscopy viewer of the NCI Imaging Data Commons and demonstrate how the viewer enables interactive visualization of traditional brightfield microscopy and highly-multiplexed immunofluorescence microscopy images from The Cancer Genome Atlas and Human Tissue Atlas Network, respectively, using standard DICOMweb services. We further show how Slim enables the collection of standardized image annotations for the development or validation of machine learning models and the visual interpretation of model inference results in the form of segmentation masks, spatial heat maps, or image-derived measurements.

Highdicom: a Python Library for Standardized Encoding of Image Annotations and Machine Learning Model Outputs in Pathology and Radiology.

Machine learning (ML) is revolutionizing image-based diagnostics in pathology and radiology. ML models have shown promising results in research settings, but the lack of interoperability between ML systems and enterprise medical imaging systems has been a major barrier for clinical integration and evaluation. The DICOM® standard specifies information object definitions (IODs) and services for the representation and communication of digital images and related information, including image-derived annotations and analysis results. However, the complexity of the standard represents an obstacle for its adoption in the ML community and creates a need for software libraries and tools that simplify working with datasets in DICOM format. Here we present the highdicom library, which provides a high-level application programming interface (API) for the Python programming language that abstracts low-level details of the standard and enables encoding and decoding of image-derived information in DICOM format in a few lines of Python code. The highdicom library leverages NumPy arrays for efficient data representation and ties into the extensive Python ecosystem for image processing and machine learning. Simultaneously, by simplifying creation and parsing of DICOM-compliant files, highdicom achieves interoperability with the medical imaging systems that hold the data used to train and run ML models, and ultimately communicate and store model outputs for clinical use. We demonstrate through experiments with slide microscopy and computed tomography imaging, that, by bridging these two ecosystems, highdicom enables developers and researchers to train and evaluate state-of-the-art ML models in pathology and radiology while remaining compliant with the DICOM standard and interoperable with clinical systems at all stages. To promote standardization of ML research and streamline the ML model development and deployment process, we made the library available free and open-source at https://github.com/herrmannlab/highdicom .

Interactive Multimedia Reporting Technical Considerations: HIMSS-SIIM Collaborative White Paper.

Despite technological advances in the analysis of digital images for medical consultations, many health information systems lack the ability to correlate textual descriptions of image findings linked to the actual images. Images and reports often reside in separate silos in the medical record throughout the process of image viewing, report authoring, and report consumption. Forward-thinking centers and early adopters have created interactive reports with multimedia elements and embedded hyperlinks in reports that connect the narrative text with the related source images and measurements. Most of these solutions rely on proprietary single-vendor systems for viewing and reporting in the absence of any encompassing industry standards to facilitate interoperability with the electronic health record (EHR) and other systems. International standards have enabled the digitization of image acquisition, storage, viewing, and structured reporting. These provide the foundation to discuss enhanced reporting. Lessons learned in the digital transformation of radiology and pathology can serve as a basis for interactive multimedia reporting (IMR) across image-centric medical specialties. This paper describes the standard-based infrastructure and communications to fulfill recently defined clinical requirements through a consensus from an international workgroup of multidisciplinary medical specialists, informaticists, and industry participants. These efforts have led toward the development of an Integrating the Healthcare Enterprise (IHE) profile that will serve as a foundation for interoperable interactive multimedia reporting.

Multispecialty Enterprise Imaging Workgroup Consensus on Interactive Multimedia Reporting Current State and Road to the Future: HIMSS-SIIM Collaborative White Paper.

Diagnostic and evidential static image, video clip, and sound multimedia are captured during routine clinical care in cardiology, dermatology, ophthalmology, pathology, physiatry, radiation oncology, radiology, endoscopic procedural specialties, and other medical disciplines. Providers typically describe the multimedia findings in contemporaneous electronic health record clinical notes or associate a textual interpretative report. Visual communication aids commonly used to connect, synthesize, and supplement multimedia and descriptive text outside medicine remain technically challenging to integrate into patient care. Such beneficial interactive elements may include hyperlinks between text, multimedia elements, alphanumeric and geometric annotations, tables, graphs, timelines, diagrams, anatomic maps, and hyperlinks to external educational references that patients or provider consumers may find valuable. This HIMSS-SIIM Enterprise Imaging Community workgroup white paper outlines the current and desired clinical future state of interactive multimedia reporting (IMR). The workgroup adopted a consensus definition of IMR as "interactive medical documentation that combines clinical images, videos, sound, imaging metadata, and/or image annotations with text, typographic emphases, tables, graphs, event timelines, anatomic maps, hyperlinks, and/or educational resources to optimize communication between medical professionals, and between medical professionals and their patients." This white paper also serves as a precursor for future efforts toward solving technical issues impeding routine interactive multimedia report creation and ingestion into electronic health records.

NCI Imaging Data Commons.

The National Cancer Institute (NCI) Cancer Research Data Commons (CRDC) aims to establish a national cloud-based data science infrastructure. Imaging Data Commons (IDC) is a new component of CRDC supported by the Cancer Moonshot. The goal of IDC is to enable a broad spectrum of cancer researchers, with and without imaging expertise, to easily access and explore the value of deidentified imaging data and to support integrated analyses with nonimaging data. We achieve this goal by colocating versatile imaging collections with cloud-based computing resources and data exploration, visualization, and analysis tools. The IDC pilot was released in October 2020 and is being continuously populated with radiology and histopathology collections. IDC provides access to curated imaging collections, accompanied by documentation, a user forum, and a growing number of analysis use cases that aim to demonstrate the value of a data commons framework applied to cancer imaging research. SIGNIFICANCE: This study introduces NCI Imaging Data Commons, a new repository of the NCI Cancer Research Data Commons, which will support cancer imaging research on the cloud.

Design and Implementation of the Pre-Clinical DICOM Standard in Multi-Cohort Murine Studies.

The small animal imaging Digital Imaging and Communications in Medicine (DICOM) acquisition context structured report (SR) was developed to incorporate pre-clinical data in an established DICOM format for rapid queries and comparison of clinical and non-clinical datasets. Established terminologies (i.e., anesthesia, mouse model nomenclature, veterinary definitions, NCI Metathesaurus) were utilized to assist in defining terms implemented in pre-clinical imaging and new codes were added to integrate the specific small animal procedures and handling processes, such as housing, biosafety level, and pre-imaging rodent preparation. In addition to the standard DICOM fields, the small animal SR includes fields specific to small animal imaging such as tumor graft (i.e., melanoma), tissue of origin, mouse strain, and exogenous material, including the date and site of injection. Additionally, the mapping and harmonization developed by the Mouse-Human Anatomy Project were implemented to assist co-clinical research by providing cross-reference human-to-mouse anatomies. Furthermore, since small animal imaging performs multi-mouse imaging for high throughput, and queries for co-clinical research requires a one-to-one relation, an imaging splitting routine was developed, new Unique Identifiers (UID's) were created, and the original patient name and ID were saved for reference to the original dataset. We report the implementation of the small animal SR using MRI datasets (as an example) of patient-derived xenograft mouse models and uploaded to The Cancer Imaging Archive (TCIA) for public dissemination, and also implemented this on PET/CT datasets. The small animal SR enhancement provides researchers the ability to query any DICOM modality pre-clinical and clinical datasets using standard vocabularies and enhances co-clinical studies.

Standardization of neurophysiology signal data into the DICOM® standard.

A standard format for neurophysiology data is urgently needed to improve clinical care and promote research data exchange. Previous neurophysiology format standardization projects have provided valuable insights into how to accomplish the project. In medical imaging, the Digital Imaging and Communication in Medicine (DICOM) standard is widely adopted. DICOM offers a unique environment to accomplish neurophysiology format standardization because neurophysiology data can be easily integrated with existing DICOM-supported elements such as video, ECG, and images and also because it provides easy integration into hospital Picture Archiving and Communication Systems (PACS) long-term storage systems. Through the support of the International Federation of Clinical Neurophysiology (IFCN) and partners in industry, DICOM Working Group 32 (WG-32) has created an initial set of standards for routine electroencephalography (EEG), polysomnography (PSG), electromyography (EMG), and electrooculography (EOG). Longer and more complex neurophysiology data types such as high-definition EEG, long-term monitoring EEG, intracranial EEG, magnetoencephalography, advanced EMG, and evoked potentials will be added later. In order to provide for efficient data compression, a DICOM neurophysiology codec design competition will be held by the IFCN and this is currently being planned. We look forward to a future when a common DICOM neurophysiology data format makes data sharing and storage much simpler and more efficient.

What is the abdomen? Rationalising clinical and anatomical perspectives using formal semantics.

The meaning of the term 'abdomen' has become increasingly ambiguous, as it has to satisfy the contemporary requirements of natural language discourse, literature, gross and radiological anatomy and its role in ontologies supporting electronic records and data modelling. It is critical that there is an agreed understanding of the semantics of the abdominopelvic cavity, its component volumes including the abdomen proper, true and false pelvic cavities, and its boundaries and regional contents. The expression of part-whole (meronymic) relationships is essential for inferences to be drawn by computer algorithms, but unless these are rigorously reviewed and tested incorrect assumptions are drawn. The SNOMED CT terminology descriptions and hierarchy of anatomical concepts relating to the trunk were scrutinised for ambiguity and sub-optimal relationships using a panel of reference sources. Any identified errors were corrected and the impact of any changes reviewed iteratively by evaluating their effect on dependant hierarchies (modelled with the associated anatomical concepts). Anatomical concepts are generally structured according to a traditional gross standpoint, but in clinical practice covert complex regional notions are frequently used and during the evaluation process a new viewpoint relating to projectional (transmissive) or emissive radiological perspective was identified. The subtle but important differences in the boundaries, volumes and contents of these distinctive perspectives of the 'abdomen' are presented. Three significant complex variants have been identified which relate to the most common uses of the word 'abdomen'. The merits and disadvantages of using 'abdomen' as common synonym to more than one concept (polysemy) are briefly discussed and the solution adopted by SNOMED International described. The review of existing ontologies and academic literature confirmed the frequent varied use of the word 'abdomen', which raises concerns when derived data are increasingly being used remotely from the point of clinical contact, potentially leading to incorrect inferences. The documented regional truncal volumes from an anatomical regional, segmental and cross-sectional perspective have been integrated into a logical and comprehensive model suitable for computer processing. The robust modelling of meronymic hierarchies has to be rigorous to avoid systematic errors and it is thus timely that a proposed standard description of these subtly related volumes and structures is made available for discussion and comment.

The Importance of Body Part Labeling to Enable Enterprise Imaging: A HIMSS-SIIM Enterprise Imaging Community Collaborative White Paper.

In order for enterprise imaging to be successful across a multitude of specialties, systems, and sites, standards are essential to categorize and classify imaging data. The HIMSS-SIIM Enterprise Imaging Community believes that the Digital Imaging Communications in Medicine (DICOM) Anatomic Region Sequence, or its equivalent in other data standards, is a vital data element for this role, when populated with standard coded values. We believe that labeling images with standard Anatomic Region Sequence codes will enhance the user's ability to consume data, facilitate interoperability, and allow greater control of privacy. Image consumption-when a user views a patient's images, he or she often wants to see relevant comparison images of the same lesion or anatomic region for the same patient automatically presented. Relevant comparison images may have been acquired from a variety of modalities and specialties. The Anatomic Region Sequence data element provides a basis to allow for efficient comparison in both instances. Interoperability-as patients move between health care systems, it is important to minimize friction for data transfer. Health care providers and facilities need to be able to consume and review the increasingly large and complex volume of data efficiently. The use of Anatomic Region Sequence, or its equivalent, populated with standard values enables seamless interoperability of imaging data regardless of whether images are used within a site or across different sites and systems. Privacy-as more visible light photographs are integrated into electronic systems, it becomes apparent that some images may need to be sequestered. Although additional work is needed to protect sensitive images, standard coded values in Anatomic Region Sequence support the identification of potentially sensitive images, enable facilities to create access control policies, and can be used as an interim surrogate for more sophisticated rule-based or attribute-based access control mechanisms. To satisfy such use cases, the HIMSS-SIIM Enterprise Imaging Community encourages the use of a pre-existing body part ontology. Through this white paper, we will identify potential challenges in employing this standard and provide potential solutions for these challenges.

DICOM Format and Protocol Standardization-A Core Requirement for Digital Pathology Success.

As the use of digital techniques in toxicologic pathology expands, challenges of scalability and interoperability come to the fore. Proprietary formats and closed single-vendor platforms prevail but depend on the availability and maintenance of multiformat conversion libraries. Expedient for small deployments, this is not sustainable at an industrial scale. Primarily known as a standard for radiology, the Digital Imaging and Communications in Medicine (DICOM) standard has been evolving to support other specialties since its inception, to become the single ubiquitous standard throughout medical imaging. The adoption of DICOM for whole slide imaging (WSI) has been sluggish. Prospects for widespread commercially viable clinical use of digital pathology change the incentives. Connectathons using DICOM have demonstrated its feasibility for WSI and virtual microscopy. Adoption of DICOM for digital and computational pathology will allow the reuse of enterprise-wide infrastructure for storage, security, and business continuity. The DICOM embedded metadata allows detached files to remain useful. Bright-field and multichannel fluorescence, Z-stacks, cytology, and sparse and fully tiled encoding are supported. External terminologies and standard compression schemes are supported. Color consistency is defined using International Color Consortium profiles. The DICOM files can be dual personality Tagged Image File Format (TIFF) for legacy support. Annotations for computational pathology results can be encoded.

DICOM re-encoding of volumetrically annotated Lung Imaging Database Consortium (LIDC) nodules.

PURPOSE: The dataset contains annotations for lung nodules collected by the Lung Imaging Data Consortium and Image Database Resource Initiative (LIDC) stored as standard DICOM objects. The annotations accompany a collection of computed tomography (CT) scans for over 1000 subjects annotated by multiple expert readers, and correspond to "nodules ≥ 3 mm", defined as any lesion considered to be a nodule with greatest in-plane dimension in the range 3-30 mm regardless of presumed histology. The present dataset aims to simplify reuse of the data with the readily available tools, and is targeted towards researchers interested in the analysis of lung CT images. ACQUISITION AND VALIDATION METHODS: Open source tools were utilized to parse the project-specific XML representation of LIDC-IDRI annotations and save the result as standard DICOM objects. Validation procedures focused on establishing compliance of the resulting objects with the standard, consistency of the data between the DICOM and project-specific representation, and evaluating interoperability with the existing tools. DATA FORMAT AND USAGE NOTES: The dataset utilizes DICOM Segmentation objects for storing annotations of the lung nodules, and DICOM Structured Reporting objects for communicating qualitative evaluations (nine attributes) and quantitative measurements (three attributes) associated with the nodules. The total of 875 subjects contain 6859 nodule annotations. Clustering of the neighboring annotations resulted in 2651 distinct nodules. The data are available in TCIA at https://doi.org/10.7937/TCIA.2018.h7umfurq. POTENTIAL APPLICATIONS: The standardized dataset maintains the content of the original contribution of the LIDC-IDRI consortium, and should be helpful in developing automated tools for characterization of lung lesions and image phenotyping. In addition to those properties, the representation of the present dataset makes it more FAIR (Findable, Accessible, Interoperable, Reusable) for the research community, and enables its integration with other standardized data collections.

Quantitative Imaging Informatics for Cancer Research.

PURPOSE: We summarize Quantitative Imaging Informatics for Cancer Research (QIICR; U24 CA180918), one of the first projects funded by the National Cancer Institute (NCI) Informatics Technology for Cancer Research program. METHODS: QIICR was motivated by the 3 use cases from the NCI Quantitative Imaging Network. 3D Slicer was selected as the platform for implementation of open-source quantitative imaging (QI) tools. Digital Imaging and Communications in Medicine (DICOM) was chosen for standardization of QI analysis outputs. Support of improved integration with community repositories focused on The Cancer Imaging Archive (TCIA). Priorities included improved capabilities of the standard, toolkits and tools, reference datasets, collaborations, and training and outreach. RESULTS: Fourteen new tools to support head and neck cancer, glioblastoma, and prostate cancer QI research were introduced and downloaded over 100,000 times. DICOM was amended, with over 40 correction proposals addressing QI needs. Reference implementations of the standard in a popular toolkit and standalone tools were introduced. Eight datasets exemplifying the application of the standard and tools were contributed. An open demonstration/connectathon was organized, attracting the participation of academic groups and commercial vendors. Integration of tools with TCIA was improved by implementing programmatic communication interface and by refining best practices for QI analysis results curation. CONCLUSION: Tools, capabilities of the DICOM standard, and datasets we introduced found adoption and utility within the cancer imaging community. A collaborative approach is critical to addressing challenges in imaging informatics at the national and international levels. Numerous challenges remain in establishing and maintaining the infrastructure of analysis tools and standardized datasets for the imaging community. Ideas and technology developed by the QIICR project are contributing to the NCI Imaging Data Commons currently being developed.