Deep-Manager: a versatile tool for optimal feature selection in live-cell imaging analysis.

One of the major problems in bioimaging, often highly underestimated, is whether features extracted for a discrimination or regression task will remain valid for a broader set of similar experiments or in the presence of unpredictable perturbations during the image acquisition process. Such an issue is even more important when it is addressed in the context of deep learning features due to the lack of a priori known relationship between the black-box descriptors (deep features) and the phenotypic properties of the biological entities under study. In this regard, the widespread use of descriptors, such as those coming from pre-trained Convolutional Neural Networks (CNNs), is hindered by the fact that they are devoid of apparent physical meaning and strongly subjected to unspecific biases, i.e., features that do not depend on the cell phenotypes, but rather on acquisition artifacts, such as brightness or texture changes, focus shifts, autofluorescence or photobleaching. The proposed Deep-Manager software platform offers the possibility to efficiently select those features having lower sensitivity to unspecific disturbances and, at the same time, a high discriminating power. Deep-Manager can be used in the context of both handcrafted and deep features. The unprecedented performances of the method are proven using five different case studies, ranging from selecting handcrafted green fluorescence protein intensity features in chemotherapy-related breast cancer cell death investigation to addressing problems related to the context of Deep Transfer Learning. Deep-Manager, freely available at https://github.com/BEEuniroma2/Deep-Manager , is suitable for use in many fields of bioimaging and is conceived to be constantly upgraded with novel image acquisition perturbations and modalities.

Machine learning phenomics (MLP) combining deep learning with time-lapse-microscopy for monitoring colorectal adenocarcinoma cells gene expression and drug-response.

High-throughput phenotyping is becoming increasingly available thanks to analytical and bioinformatics approaches that enable the use of very high-dimensional data and to the availability of dynamic models that link phenomena across levels: from genes to cells, from cells to organs, and through the whole organism. The combination of phenomics, deep learning, and machine learning represents a strong potential for the phenotypical investigation, leading the way to a more embracing approach, called machine learning phenomics (MLP). In particular, in this work we present a novel MLP platform for phenomics investigation of cancer-cells response to therapy, exploiting and combining the potential of time-lapse microscopy for cell behavior data acquisition and robust deep learning software architectures for the latent phenotypes extraction. A two-step proof of concepts is designed. First, we demonstrate a strict correlation among gene expression and cell phenotype with the aim to identify new biomarkers and targets for tailored therapy in human colorectal cancer onset and progression. Experiments were conducted on human colorectal adenocarcinoma cells (DLD-1) and their profile was compared with an isogenic line in which the expression of LOX-1 transcript was knocked down. In addition, we also evaluate the phenotypic impact of the administration of different doses of an antineoplastic drug over DLD-1 cells. Under the omics paradigm, proteomics results are used to confirm the findings of the experiments.

Accelerating the experimental responses on cell behaviors: a long-term prediction of cell trajectories using Social Generative Adversarial Network.

The incremented uptake provided by time-lapse microscopy in Organ-on-a-Chip (OoC) devices allowed increased attention to the dynamics of the co-cultured systems. However, the amount of information stored in long-time experiments may constitute a serious bottleneck of the experimental pipeline. Forward long-term prediction of cell trajectories may reduce the spatial-temporal burden of video sequences storage. Cell trajectory prediction becomes crucial especially to increase the trustworthiness in software tools designed to conduct a massive analysis of cell behavior under chemical stimuli. To address this task, we transpose here the exploitation of the presence of "social forces" from the human to the cellular level for motion prediction at microscale by adapting the potential of Social Generative Adversarial Network predictors to cell motility. To demonstrate the effectiveness of the approach, we consider here two case studies: one related to PC-3 prostate cancer cells cultured in 2D Petri dishes under control and treated conditions and one related to an OoC experiment of tumor-immune interaction in fibrosarcoma cells. The goodness of the proposed strategy has been verified by successfully comparing the distributions of common descriptors (kinematic descriptors and mean interaction time for the two scenarios respectively) from the trajectories obtained by video analysis and the predicted counterparts.

Discovering the hidden messages within cell trajectories using a deep learning approach for in vitro evaluation of cancer drug treatments.

We describe a novel method to achieve a universal, massive, and fully automated analysis of cell motility behaviours, starting from time-lapse microscopy images. The approach was inspired by the recent successes in application of machine learning for style recognition in paintings and artistic style transfer. The originality of the method relies i) on the generation of atlas from the collection of single-cell trajectories in order to visually encode the multiple descriptors of cell motility, and ii) on the application of pre-trained Deep Learning Convolutional Neural Network architecture in order to extract relevant features to be used for classification tasks from this visual atlas. Validation tests were conducted on two different cell motility scenarios: 1) a 3D biomimetic gels of immune cells, co-cultured with breast cancer cells in organ-on-chip devices, upon treatment with an immunotherapy drug; 2) Petri dishes of clustered prostate cancer cells, upon treatment with a chemotherapy drug. For each scenario, single-cell trajectories are very accurately classified according to the presence or not of the drugs. This original approach demonstrates the existence of universal features in cell motility (a so called "motility style") which are identified by the DL approach in the rationale of discovering the unknown message in cell trajectories.

The influence of spatial and temporal resolutions on the analysis of cell-cell interaction: a systematic study for time-lapse microscopy applications.

Cell-cell interactions are an observable manifestation of underlying complex biological processes occurring in response to diversified biochemical stimuli. Recent experiments with microfluidic devices and live cell imaging show that it is possible to characterize cell kinematics via computerized algorithms and unravel the effects of targeted therapies. We study the influence of spatial and temporal resolutions of time-lapse videos on motility and interaction descriptors with computational models that mimic the interaction dynamics among cells. We show that the experimental set-up of time-lapse microscopy has a direct impact on the cell tracking algorithm and on the derived numerical descriptors. We also show that, when comparing kinematic descriptors in two diverse experimental conditions, too low resolutions may alter the descriptors' discriminative power, and so the statistical significance of the difference between the two compared distributions. The conclusions derived from the computational models were experimentally confirmed by a series of video-microscopy acquisitions of co-cultures of unlabelled human cancer and immune cells embedded in 3D collagen gels within microfluidic devices. We argue that the experimental protocol of acquisition should be adapted to the specific kind of analysis involved and to the chosen descriptors in order to derive reliable conclusions and avoid biasing the interpretation of results.

Towards localization of malignant sites of asymmetry across bilateral mammograms.

BACKGROUND AND OBJECTIVES: The analysis of patterns of asymmetry between the left and right mammograms of a patient can provide meaningful insights into the presence of an underlying tumor in its early stage. However, the identification of breast cancer by investigating bilateral asymmetry is difficult to perform due to the indistinct and borderline nature of the asymmetric signs as they appear on mammograms. METHODS: In this study, to increase the positive-predictive value of asymmetry in mammographic screening, a novel computerized approach for the automatic localization of malignant sites of asymmetry in mammograms is proposed. The sites of anatomical correspondence between the right and left regions of each radiographic projection were extracted by means of two bilateral masking procedures, inspired by radiologists' criteria in interpreting mammograms and based on the use of detected landmarking structures. Relative variations of spatial patterns of intensity values and of orientations of directional components within each site were quantified by combining multidirectional Gabor filters and indices of structural similarity. The localization of the sites of malignant asymmetry was performed by coupling two quadratic discriminant analysis classifiers, one for each masking procedure, that assigned the likelihood of malignancy to each site of correspondence. RESULTS: The performance of the proposed method was assessed on 94 mammographic images from two publicly available databases and containing at least one asymmetric site. Sensitivity, specificity and balanced accuracy levels of 0.83 (0.09), 0.75 (0.06), and 0.79 (0.04), respectively were obtained in the classification of malignant asymmetric sites vs benign/normal sites using cross-validation. In addition, a further blind test on a dataset of Full Field Digital Mammograms achieved levels of sensitivity, specificity, and balanced accuracy of 0.86, 0.65, and 0.75, respectively. CONCLUSIONS: The achieved performance indicates that the proposed system is effective in localizing sites of malignant asymmetry and it is expected to improve computer-aided diagnosis of breast cancer.

Estimation of the breast skin-line in mammograms using multidirectional Gabor filters.

Segmentation of the breast region is a fundamental step in any system for computerized analysis of mammograms. In this work, we propose a novel procedure for the estimation of the breast skin-line based upon multidirectional Gabor filtering. The method includes an adaptive values-of-interest (VOI) transformation, extraction of the skin-air ribbon by Otsu's thresholding method and the Euclidean distance transform, Gabor filtering with 18 real kernels, and a step for suppression of false edge points using the magnitude and phase responses of the filters. On a test set of 361 images from different acquisition modalities (screen-film and full-field digital mammograms), the average Hausdorff and polyline distances obtained were 2.85 mm and 0.84 mm, respectively, with reference to the ground-truth boundaries provided by an expert radiologist. When compared with the results obtained by other state-of-the-art methods on the same set of images and with respect to the same ground-truth boundaries, our method mostly outperformed the other approaches. The results demonstrate the effectiveness and robustness of the proposed algorithm.