Deep learning identifies morphological patterns of homologous recombination deficiency in luminal breast cancers from whole slide images.

Homologous recombination DNA-repair deficiency (HRD) is becoming a well-recognized marker of platinum salt and polyADP-ribose polymerase inhibitor chemotherapies in ovarian and breast cancers. While large-scale screening for HRD using genomic markers is logistically and economically challenging, stained tissue slides are routinely acquired in clinical practice. With the objectives of providing a robust deep-learning method for HRD prediction from tissue slides and identifying related morphological phenotypes, we first show that digital pathology workflows are sensitive to potential biases in the training set, then we propose a method to overcome the influence of these biases, and we develop an interpretation method capable of identifying complex phenotypes. Application to our carefully curated in-house dataset allows us to predict HRD with high accuracy (area under the receiver-operator characteristics curve 0.86) and to identify morphological phenotypes related to HRD. In particular, the presence of laminated fibrosis and clear tumor cells associated with HRD open new hypotheses regarding its phenotypic impact.

In vivo multiphoton multiparametric 3D quantification of human skin aging on forearm and face.

Quantifying skin aging changes and characterizing its 3D structure and function in a non-invasive way is still a challenging area of research, constantly evolving with the development of imaging methods and image analysis tools. In vivo multiphoton imaging offers means to assess skin constituents in 3D, however prior skin aging studies mostly focused on 2D analyses of dermal fibers through their signals' intensities or densities. In this work, we designed and implemented multiphoton multiparametric 3D quantification tools for in vivo human skin pigmentation and aging characterization. We first demonstrated that despite the limited field of view of the technic, investigation of 2 regions of interest (ROIs) per zone per volunteer is a good compromise in assessing 3D skin constituents in both epidermis and superficial dermis. We then characterized skin aging on different UV exposed areas-ventral and dorsal forearms, face. The three major facts of aging that are epidermal atrophy, the dermal-epidermal junction (DEJ) flattening and dermal elastosis can be non-invasively quantified and compared. Epidermal morphological changes occur late and were only objectified between extreme age groups. Melanin accumulation in suprabasal layers with age and chronic exposure on ventral and dorsal forearms is less known and appears earlier. Superficial dermal aging changes are mainly elastin density increase, with no obvious change in collagen density, reflected by SHGto2PEF ratio and SAAID index decrease and ImbrN index increase on all skin areas. Analysis of the z-dermal distribution of these parameters highlighted the 2nd 20 µm thickness normalized dermal sub-layer, that follows the DEJ shape, as exhibiting the highest aging differences. Moreover, the 3D ImbrN index allows refining the share of photoaging in global aging on face and the 3D SAAID index on forearm, which elastin or fibrillar collagens densities alone do not allow. Photoaging of the temple area evolves as a function of chronic exposure with a more pronounced increase in elastin density, also structurally modified from thin and straight elastic fibers in young volunteers to dense and compact pattern in older ones. More generally, multiphoton multiparametric 3D skin quantification offers rich spatial information of interest in assessing normal human skin condition and its pathological, external environment or product induced changes.

In vivo melanin 3D quantification and z-epidermal distribution by multiphoton FLIM, phasor and Pseudo-FLIM analyses.

Characterizing melanins in situ and determining their 3D z-epidermal distribution is paramount for understanding physiological/pathological processes of melanin neosynthesis, transfer, degradation or modulation with external UV exposure or cosmetic/pharmaceutical products. Multiphoton fluorescence intensity- and lifetime-based approaches have been shown to afford melanin detection, but how can one quantify melanin in vivo in 3D from multiphoton fluorescence lifetime (FLIM) data, especially since FLIM imaging requires long image acquisition times not compatible with 3D imaging in a clinical setup? We propose an approach combining (i) multiphoton FLIM, (ii) fast image acquisition times, and (iii) a melanin detection method called Pseudo-FLIM, based on slope analysis of autofluorescence intensity decays from temporally binned data. We compare Pseudo-FLIM to FLIM bi-exponential and phasor analyses of synthetic melanin, melanocytes/keratinocytes coculture and in vivo human skin. Using parameters of global 3D epidermal melanin density and z-epidermal distribution profile, we provide first insights into the in vivo knowledge of 3D melanin modulations with constitutive pigmentation versus ethnicity, with seasonality over 1 year and with topical application of retinoic acid or retinol on human skin. Applications of Pseudo-FLIM based melanin detection encompass physiological, pathological, or environmental factors-induced pigmentation modulations up to whitening, anti-photoaging, or photoprotection products evaluation.

In vivo multiphoton imaging for non-invasive time course assessment of retinoids effects on human skin.

BACKGROUND: In vivo multiphoton imaging and automatic 3D image processing tools provide quantitative information on human skin constituents. These multiphoton-based tools allowed evidencing retinoids epidermal effects in the occlusive patch test protocol developed for antiaging products screening. This study aimed at investigating their relevance for non-invasive, time course assessment of retinoids cutaneous effects under real-life conditions for one year. MATERIALS AND METHODS: Thirty women, 55-65 y, applied either retinol (RO 0.3%) or retinoic acid (RA 0.025%) on one forearm dorsal side versus a control product on the other forearm once a day for 1 year. In vivo multiphoton imaging was performed every three months, and biopsies were taken after 1 year. Epidermal thickness and dermal-epidermal junction undulation were estimated in 3D with multiphoton and in 2D with histology, whereas global melanin density and its z-epidermal distribution were estimated using 3D multiphoton image processing tools. RESULTS: Main results after one year were as follows: a) epidermal thickening with RO (+30%); b) slight increase in dermal-epidermal junction undulation with RO; c) slight decrease in 3D melanin density with RA; d) limitation of the melanin ascent observed with seasonality and time within supra-basal layers with both retinoids, using multiphoton 3D-melanin z-epidermal profile. CONCLUSIONS: With a novel 3D descriptor of melanin z-epidermal distribution, in vivo multiphoton imaging allows demonstrating that daily usage of retinoids counteracts aging by acting not only on epidermal morphology, but also on melanin that is shown to accumulate in the supra-basal layers with time.

Waterpixels.

Many approaches for image segmentation rely on a first low-level segmentation step, where an image is partitioned into homogeneous regions with enforced regularity and adherence to object boundaries. Methods to generate these superpixels have gained substantial interest in the last few years, but only a few have made it into applications in practice, in particular because the requirements on the processing time are essential but are not met by most of them. Here, we propose waterpixels as a general strategy for generating superpixels which relies on the marker controlled watershed transformation. We introduce a spatially regularized gradient to achieve a tunable tradeoff between the superpixel regularity and the adherence to object boundaries. The complexity of the resulting methods is linear with respect to the number of image pixels. We quantitatively evaluate our approach on the Berkeley segmentation database and compare it against the state-of-the-art.

Determination of collagen fiber orientation in histological slides using Mueller microscopy and validation by second harmonic generation imaging.

We studied the azimuthal orientations of collagen fibers in histological slides of uterine cervical tissue by two different microscopy techniques, namely Mueller polarimetry (MP) and Second Harmonic Generation (SHG). SHG provides direct visualization of the fibers with high specificity, which orientations is then obtained by suitable image processing. MP provides images of retardation (among other polarimetric parameters) due to the optical anisotropy of the fibers, which is enhanced by Picrosirius Red staining. The fiber orientations are then assumed to be those of the retardation slow axes. The two methods, though fully different from each other, provide quite similar maps of average fiber orientations. Overall, our results confirm that MP microscopy provides reliable images of dominant fiber orientations at a much lower cost that SHG, which remains the "gold standard" for specific imaging of collagen fibers using optical microscopy.

Fibrillogenesis from nanosurfaces: multiphoton imaging and stereological analysis of collagen 3D self-assembly dynamics.

The assembly of proteins into fibrillar structures is an important process that concerns different biological contexts, including molecular medicine and functional biomaterials. Engineering of hybrid biomaterials can advantageously provide synergetic interactions of the biopolymers with an inorganic component to ensure specific supramolecular organization and dynamics. To this aim, we designed hybrid systems associating collagen and surface-functionalized silica particles and we built a new strategy to investigate fibrillogenesis processes in such multicomponents systems, working at the crossroads of chemistry, physics and mathematics. The self-assembly process was investigated by bimodal multiphoton imaging coupling second harmonic generation (SHG) and 2 photon excited fluorescence (2PEF). The in-depth spatial characterization of the system was further achieved using the three-dimensional analysis of the SHG/2PEF data via mathematical morphology processing. Quantitation of collagen distribution around particles offers strong evidence that the chemically induced confinement of the protein on the silica nanosurfaces has a key influence on the spatial extension of fibrillogenesis. This new approach is unique in the information it can provide on 3D dynamic hybrid systems and may be extended to other associations of fibrillar molecules with optically responsive nano-objects.

Exudate detection in color retinal images for mass screening of diabetic retinopathy.

The automatic detection of exudates in color eye fundus images is an important task in applications such as diabetic retinopathy screening. The presented work has been undertaken in the framework of the TeleOphta project, whose main objective is to automatically detect normal exams in a tele-ophthalmology network, thus reducing the burden on the readers. A new clinical database, e-ophtha EX, containing precisely manually contoured exudates, is introduced. As opposed to previously available databases, e-ophtha EX is very heterogeneous. It contains images gathered within the OPHDIAT telemedicine network for diabetic retinopathy screening. Image definition, quality, as well as patients condition or the retinograph used for the acquisition, for example, are subject to important changes between different examinations. The proposed exudate detection method has been designed for this complex situation. We propose new preprocessing methods, which perform not only normalization and denoising tasks, but also detect reflections and artifacts in the image. A new candidates segmentation method, based on mathematical morphology, is proposed. These candidates are characterized using classical features, but also novel contextual features. Finally, a random forest algorithm is used to detect the exudates among the candidates. The method has been validated on the e-ophtha EX database, obtaining an AUC of 0.95. It has been also validated on other databases, obtaining an AUC between 0.93 and 0.95, outperforming state-of-the-art methods.

Parsimonious path openings and closings.

Path openings and closings are morphological tools used to preserve long, thin, and tortuous structures in gray level images. They explore all paths from a defined class, and filter them with a length criterion. However, most paths are redundant, making the process generally slow. Parsimonious path openings and closings are introduced in this paper to solve this problem. These operators only consider a subset of the paths considered by classical path openings, thus achieving a substantial speed-up, while obtaining similar results. In addition, a recently introduced 1D opening algorithm is applied along each selected path. Its complexity is linear with respect to the number of pixels, independent of the size of the opening. Furthermore, it is fast for any input data accuracy (integer or floating point) and works in stream. Parsimonious path openings are also extended to incomplete paths, i.e., paths containing gaps. Noise-corrupted paths can thus be processed with the same approach and complexity. These parsimonious operators achieve a several orders of magnitude speed-up. Examples are shown for incomplete path openings, where computing times are brought from minutes to tens of milliseconds, while obtaining similar results.

Multimedia data mining for automatic diabetic retinopathy screening.

This paper presents TeleOphta, an automatic system for screening diabetic retinopathy in teleophthalmology networks. Its goal is to reduce the burden on ophthalmologists by automatically detecting non referable examination records, i.e. examination records presenting no image quality problems and no pathological signs related to diabetic retinopathy or any other retinal pathology. TeleOphta is an attempt to put into practice years of algorithmic developments from our groups. It combines image quality metrics, specific lesion detectors and a generic pathological pattern miner to process the visual content of eye fundus photographs. This visual information is further combined with contextual data in order to compute an abnormality risk for each examination record. The TeleOphta system was trained and tested on a large dataset of 25,702 examination records from the OPHDIAT screening network in Paris. It was able to automatically detect 68% of the non referable examination records while achieving the same sensitivity as a second ophthalmologist. This suggests that it could safely reduce the burden on ophthalmologists by 56%.

Automatic 3D segmentation of multiphoton images: a key step for the quantification of human skin.

BACKGROUND/PURPOSE: Multiphoton microscopy has emerged in the past decade as a useful noninvasive imaging technique for in vivo human skin characterization. However, it has not been used until now in evaluation clinical trials, mainly because of the lack of specific image processing tools that would allow the investigator to extract pertinent quantitative three-dimensional (3D) information from the different skin components. METHODS: We propose a 3D automatic segmentation method of multiphoton images which is a key step for epidermis and dermis quantification. This method, based on the morphological watershed and graph cuts algorithms, takes into account the real shape of the skin surface and of the dermal-epidermal junction, and allows separating in 3D the epidermis and the superficial dermis. RESULTS: The automatic segmentation method and the associated quantitative measurements have been developed and validated on a clinical database designed for aging characterization. The segmentation achieves its goals for epidermis-dermis separation and allows quantitative measurements inside the different skin compartments with sufficient relevance. CONCLUSIONS: This study shows that multiphoton microscopy associated with specific image processing tools provides access to new quantitative measurements on the various skin components. The proposed 3D automatic segmentation method will contribute to build a powerful tool for characterizing human skin condition. To our knowledge, this is the first 3D approach to the segmentation and quantification of these original images.

A multiple-instance learning framework for diabetic retinopathy screening.

A novel multiple-instance learning framework, for automated image classification, is presented in this paper. Given reference images marked by clinicians as relevant or irrelevant, the image classifier is trained to detect patterns, of arbitrary size, that only appear in relevant images. After training, similar patterns are sought in new images in order to classify them as either relevant or irrelevant images. Therefore, no manual segmentations are required. As a consequence, large image datasets are available for training. The proposed framework was applied to diabetic retinopathy screening in 2-D retinal image datasets: Messidor (1200 images) and e-ophtha, a dataset of 25,702 examination records from the Ophdiat screening network (107,799 images). In this application, an image (or an examination record) is relevant if the patient should be referred to an ophthalmologist. Trained on one half of Messidor, the classifier achieved high performance on the other half of Messidor (A(z)=0.881) and on e-ophtha (A(z)=0.761). We observed, in a subset of 273 manually segmented images from e-ophtha, that all eight types of diabetic retinopathy lesions are detected.