Strategies to develop radiomics and machine learning models for lung cancer stage and histology prediction using small data samples.

Predictive models based on radiomics and machine-learning (ML) need large and annotated datasets for training, often difficult to collect. We designed an operative pipeline for model training to exploit data already available to the scientific community. The aim of this work was to explore the capability of radiomic features in predicting tumor histology and stage in patients with non-small cell lung cancer (NSCLC). We analyzed the radiotherapy planning thoracic CT scans of a proprietary sample of 47 subjects (L-RT) and integrated this dataset with a publicly available set of 130 patients from the MAASTRO NSCLC collection (Lung1). We implemented intra- and inter-sample cross-validation strategies (CV) for evaluating the ML predictive model performances with not so large datasets. We carried out two classification tasks: histology classification (3 classes) and overall stage classification (two classes: stage I and II). In the first task, the best performance was obtained by a Random Forest classifier, once the analysis has been restricted to stage I and II tumors of the Lung1 and L-RT merged dataset (AUC = 0.72 ± 0.11). For the overall stage classification, the best results were obtained when training on Lung1 and testing of L-RT dataset (AUC = 0.72 ± 0.04 for Random Forest and AUC = 0.84 ± 0.03 for linear-kernel Support Vector Machine). According to the classification task to be accomplished and to the heterogeneity of the available dataset(s), different CV strategies have to be explored and compared to make a robust assessment of the potential of a predictive model based on radiomics and ML.

Evaluation of the intra- and inter-method agreement of brain MRI segmentation software packages: A comparison between SPM12 and FreeSurfer v6.0.

PURPOSE: The lack of inter-method agreement can produce inconsistent results in neuroimaging studies. We evaluated the intra-method repeatability and the inter-method reproducibility of two widely-used automatic segmentation methods for brain MRI: the FreeSurfer (FS) and the Statistical Parametric Mapping (SPM) software packages. METHODS: We segmented the gray matter (GM), the white matter (WM) and subcortical structures in test-retest MRI data of healthy volunteers from Kirby-21 and OASIS datasets. We used Pearson's correlation (r), Bland-Altman plot and Dice index to study intra-method repeatability and inter-method reproducibility. In order to test whether different processing methods affect the results of a neuroimaging-based group study, we carried out a statistical comparison between male and female volume measures. RESULTS: A high correlation was found between test-retest volume measures for both SPM (r in the 0.98-0.99 range) and FS (r in the 0.95-0.99 range). A non-null bias between test-retest FS volumes was detected for GM and WM in the OASIS dataset. The inter-method reproducibility analysis measured volume correlation values in the 0.72-0.98 range and the overlap between the segmented structures assessed by the Dice index was in the 0.76-0.83 range. SPM systematically provided significantly greater GM volumes and lower WM and subcortical volumes with respect to FS. In the male vs. female brain volume comparisons, inconsistencies arose for the OASIS dataset, where the gender-related differences appear subtler with respect to the Kirby dataset. CONCLUSIONS: The inter-method reproducibility should be evaluated before interpreting the results of neuroimaging studies.

Large scale validation of the M5L lung CAD on heterogeneous CT datasets.

PURPOSE: M5L, a fully automated computer-aided detection (CAD) system for the detection and segmentation of lung nodules in thoracic computed tomography (CT), is presented and validated on several image datasets. METHODS: M5L is the combination of two independent subsystems, based on the Channeler Ant Model as a segmentation tool [lung channeler ant model (lungCAM)] and on the voxel-based neural approach. The lungCAM was upgraded with a scan equalization module and a new procedure to recover the nodules connected to other lung structures; its classification module, which makes use of a feed-forward neural network, is based of a small number of features (13), so as to minimize the risk of lacking generalization, which could be possible given the large difference between the size of the training and testing datasets, which contain 94 and 1019 CTs, respectively. The lungCAM (standalone) and M5L (combined) performance was extensively tested on 1043 CT scans from three independent datasets, including a detailed analysis of the full Lung Image Database Consortium/Image Database Resource Initiative database, which is not yet found in literature. RESULTS: The lungCAM and M5L performance is consistent across the databases, with a sensitivity of about 70% and 80%, respectively, at eight false positive findings per scan, despite the variable annotation criteria and acquisition and reconstruction conditions. A reduced sensitivity is found for subtle nodules and ground glass opacities (GGO) structures. A comparison with other CAD systems is also presented. CONCLUSIONS: The M5L performance on a large and heterogeneous dataset is stable and satisfactory, although the development of a dedicated module for GGOs detection could further improve it, as well as an iterative optimization of the training procedure. The main aim of the present study was accomplished: M5L results do not deteriorate when increasing the dataset size, making it a candidate for supporting radiologists on large scale screenings and clinical programs.

Pleural nodule identification in low-dose and thin-slice lung computed tomography.

A completely automated system for the identification of pleural nodules in low-dose and thin-slice computed tomography (CT) of the lung has been developed. The directional-gradient concentration method has been applied to the pleura surface and combined with a morphological opening-based procedure to generate a list of nodule candidates. Each nodule candidate is characterized by 12 morphological and textural features, which are analyzed by a rule-based filter and a neural classifier. This detection system has been developed and validated on a dataset of 42 annotated CT scans. The k-fold cross validation has been used to evaluate the neural classifier performance. The system performance variability due to different ground truth agreement levels is discussed. In particular, the poor 44% sensitivity obtained on the ground truth with agreement level 1 (nodules annotated by only one radiologist) with six FP per scan grows up to the 72% if the underlying ground truth is changed to the agreement level 2 (nodules annotated by two radiologists).

Characterization of a mammographic system based on single photon counting pixel arrays coupled to GaAs x-ray detectors.

The authors report on the imaging capabilities of a mammographic system demonstrator based on GaAs pixel detectors operating in single photon counting (SPC) mode. The system imaging performances have been assessed by means of the transfer functions: The modulation transfer function (MTF), the normalized noise power spectrum, and the detective quantum efficiency (DQE) have been measured following the guidelines of the IEC 62220-1-2 protocol. The transfer function analysis has shown the high spatial resolution capabilities of the GaAs detectors. The MTF calculated at the Nyquist frequency (2.94 cycles/mm) is indeed 60%. The DQE, measured with a standard mammographic beam setup (Mo/Mo, 28 kVp, with 4 mm Al added filter) and calculated at zero frequency, is 46%. Aiming to further improve the system's image quality, the authors investigate the DQE limiting factors and show that they are mainly related to system engineering. For example, the authors show that optimization of the image equalization procedure increases the DQE(0) up to 74%, which is better than the DQE(0) of most clinical mammographic systems. The authors show how the high detection efficiency of GaAs detectors and the noise discrimination associated with the SPC technology allow optimizing the image quality in mammography. In conclusion, the authors propose technological solutions to exploit to the utmost the potentiality of GaAs detectors coupled to SPC electronics.

MAGIC-5: an Italian mammographic database of digitised images for research.

The implementation of a database of digitised mammograms is discussed. The digitised images were collected beginning in 1999 by a community of physicists in collaboration with radiologists in several Italian hospitals as a first step in developing and implementing a computer-aided detection (CAD) system. All 3,369 mammograms were collected from 967 patients and classified according to lesion type and morphology, breast tissue and pathology type. A dedicated graphical user interface was developed to visualise and process mammograms to support the medical diagnosis directly on a high-resolution screen. The database has been the starting point for developing other medical imaging applications, such as a breast CAD, currently being upgraded and optimised for use in a distributed environment with grid services, in the framework of the Instituto Nazionale di Fisicia Nucleare (INFN)-funded Medical Applications on a Grid Infrastructure Connection (MAGIC)-5 project.

A pixel detector-based single photon-counting system as fast spectrometer for diagnostic X-ray beams.

Recent advances in semiconductor pixel detectors and read-out electronics allowed to build the first prototypes of single photon-counting imaging systems that represent the last frontier of digital radiography. Among the advantages with respect to commercially available digital imaging systems, there are direct conversion of photon energy into electrical charge and the effective rejection of electronic noise by means of a thresholding process. These features allow the photon-counting systems to achieve high imaging performances in terms of spatial and contrast resolution. Moreover, the now available deep integration techniques allow the reduction of the pixel size and the improvement of the functionality of the single cell and the read-out speed so as to cope with the high fluxes found in diagnostic radiology. In particular, the single photon-counting system presented in this paper is based on a 300-microm thick silicon pixel detector bump-bonded to the Medipix2 read-out chip to form an assembly of 256 x 256 square pixels at a pitch of 55 microm. Each cell comprises a low-noise preamplifier, two pulse height discriminators and a 14-bit counter. The maximum counting rate per pixel is 1 MHz. The chip can operate in two modalities: it records the events with energy above a threshold (single mode) or between two energy thresholds (window mode). Exploiting this latter feature, a possible application of such a system as a fast spectrometer is presented to study the energy spectrum of diagnostic beams produced by X-ray tubes.

Dental radiology dosimetric data as routinely collected in an Italian hospital.

The work presented here was developed in the framework of the SENTINEL Project and is devoted to the analysis of dental radiology dosimetric data. The procedure of data processing allows the analysis of some important aspects related to the protection of the patient and the staff because of the position of the operators near the patient and their exposure to the radiation scattered by the patient. Dental radiology data was collected in an Italian hospital. Following the Italian quality assurance (QA) protocols and suggestions by the leaders of the SENTINEL Project, X-ray equipment performances have been analysed in terms of: kVp accuracy, exposure time accuracy and precision, tube output, dose reproducibility and linearity, beam collimation, artefacts and light tightness. Referring to these parameters the physical quality index (QI) was analysed. In a single numerical value between 0 and 1, QI summarises the results of quality tests for radiological devices. The actual impact of such a figure (as suggested by international QA protocols or as adopted by local QA routine) on the policy of machine maintenance and replacement is discussed.

Lung nodule detection in low-dose and thin-slice computed tomography.

A computer-aided detection (CAD) system for the identification of small pulmonary nodules in low-dose and thin-slice CT scans has been developed. The automated procedure for selecting the nodule candidates is mainly based on a filter enhancing spherical-shaped objects. A neural approach based on the classification of each single voxel of a nodule candidate has been purposely developed and implemented to reduce the amount of false-positive findings per scan. The CAD system has been trained to be sensitive to small internal and sub-pleural pulmonary nodules collected in a database of low-dose and thin-slice CT scans. The system performance has been evaluated on a data set of 39 CT containing 75 internal and 27 sub-pleural nodules. The FROC curve obtained on this data set shows high values of sensitivity to lung nodules (80-85% range) at an acceptable level of false positive findings per patient (10-13 FP/scan).

Comparison of two portable solid state detectors with an improved collimation and alignment device for mammographic x-ray spectroscopy.

We describe a portable system for mammographic x-ray spectroscopy, based on a 2 X 2 X 1 mm3 cadmium telluride (CdTe) solid state detector, that is greatly improved over a similar system based on a 3 X 3 X 2 mm3 cadmium zinc telluride (CZT) solid state detector evaluated in an earlier work. The CdTe system utilized new pinhole collimators and an alignment device that facilitated measurement of mammographic x-ray spectra. Mammographic x-ray spectra acquired by each system were comparable. Half value layer measurements obtained using an ion chamber agreed closely with those derived from the x-ray spectra measured by either detector. The faster electronics and other features of the CdTe detector allowed its use with a larger pinhole collimator than could be used with the CZT detector. Additionally, the improved pinhole collimator and alignment features of the apparatus permitted much more rapid setup for acquisition of x-ray spectra than was possible on the system described in the earlier work. These improvements in detector technology, collimation and ease of alignment, as well as low cost, make this apparatus attractive as a tool for both laboratory research and advanced mammography quality control.

A completely automated CAD system for mass detection in a large mammographic database.

Mass localization plays a crucial role in computer-aided detection (CAD) systems for the classification of suspicious regions in mammograms. In this article we present a completely automated classification system for the detection of masses in digitized mammographic images. The tool system we discuss consists in three processing levels: (a) Image segmentation for the localization of regions of interest (ROIs). This step relies on an iterative dynamical threshold algorithm able to select iso-intensity closed contours around gray level maxima of the mammogram. (b) ROI characterization by means of textural features computed from the gray tone spatial dependence matrix (GTSDM), containing second-order spatial statistics information on the pixel gray level intensity. As the images under study were recorded in different centers and with different machine settings, eight GTSDM features were selected so as to be invariant under monotonic transformation. In this way, the images do not need to be normalized, as the adopted features depend on the texture only, rather than on the gray tone levels, too. (c) ROI classification by means of a neural network, with supervision provided by the radiologist's diagnosis. The CAD system was evaluated on a large database of 3369 mammographic images [2307 negative, 1062 pathological (or positive), containing at least one confirmed mass, as diagnosed by an expert radiologist]. To assess the performance of the system, receiver operating characteristic (ROC) and free-response ROC analysis were employed. The area under the ROC curve was found to be Az = 0.783 +/- 0.008 for the ROI-based classification. When evaluating the accuracy of the CAD against the radiologist-drawn boundaries, 4.23 false positives per image are found at 80% of mass sensitivity.

A study on two different CAD systems for mammography as an aid to radiological diagnosis in the search of microcalcification clusters.

OBJECTIVE: The aim of the present study was to evaluate the efficacy of two different computer aided detection (CAD) systems for mammography in improving radiological diagnosis in the search of microcalcification clusters. The CAD systems used are: the SecondLooktrade mark (CADx Medical Systems, Canada) commercial system and the CALMA (computer assisted library in MAmmography) research CAD system. Three radiologists were asked to read mammographic images with and without the support of the CAD systems. MATERIAL AND METHODS: Three radiologists with respectively 3, 5 and 7 years of practice in mammogram reading in an Italian public hospital analysed a dataset composed of 120 digitized mammograms of healthy subjects with no lesion (proven by a radiological follow up of at least 3 years) and 70 images of patients with malignant cluster of microcalcification (proven by histopathological examination) both with no CAD support as well as with the help of the SecondLooktrade mark system. After 3 months they were asked to observe the same digitized mammograms with the assistance of the CALMA system. The radiologists worked independently and were unaware of the final diagnosis. The values of the area A(z) under the ROC curve, diagnostic sensitivity, specificity, positive and negative predictive values, and diagnostic accuracy were evaluated with and without the support of the CAD systems. The reading time and qualitative evaluations of each radiologist were also reported. RESULTS: With the support of the two CAD systems an improvement in A(z) area was obtained ranging from 0.01 to 0.04. Sensitivity increased from +8.6 to +15.7% and specificity decreased from 0.8 to 4.2%. CONCLUSION: In our study, not conditioned by the dataset, the CAD systems as second reader produced an increase in overall sensitivity of up to 15.7%, with a little decrease in specificity of up to 4.2%. Based on these results both CAD systems might be used in the current practise to improve the sensitivity values of conventional reading (radiologist alone). The results of this study show that no significant differences exist in term of A(z), sensitivity and specificity between CALMA and CADx.

GPCALMA: a Grid-based tool for mammographic screening.

OBJECTIVES: The next generation of high energy physics (HEP) experiments requires a GRID approach to a distributed computing system: the key concept is the Virtual ORGANISATION (VO), a group of distributed users with a common goal and the will to share their resources. METHODS: A similar approach, applied to a group of hospitals that joined the GPCALMA project (Grid Platform for Computer Assisted Library for MAmmography), will allow common screening programs for early diagnosis of breast and, in the future, lung cancer. The application code makes use of neural networks for the image analysis and is useful in improving the radiologists' diagnostic performance. GRID services allow remote image analysis and interactive online diagnosis, with a potential for a relevant reduction of the delays presently associated with screening programs. RESULTS AND CONCLUSIONS: A prototype of the system, based on AliEn GRID Services [1], is already available, with a central server running common services [2] and several clients connecting to it. Mammograms can be acquired in any location; the related information required to select and access them at any time is stored in a common service called Data Catalogue, which can be queried by any client. Thanks to the PROOF facility [3], the result of a query can be used as input for analysis algorithms, which are executed on the nodes where the input images are stored,. The selected approach avoids data transfers for all the images with a negative diagnosis and allows an almost real time diagnosis for the set of images with high cancer probability.