ArtSeg-Artifact segmentation and removal in brightfield cell microscopy images without manual pixel-level annotations.

Brightfield cell microscopy is a foundational tool in life sciences. The acquired images are prone to contain visual artifacts that hinder downstream analysis, and automatically removing them is therefore of great practical interest. Deep convolutional neural networks are state-of-the-art for image segmentation, but require pixel-level annotations, which are time-consuming to produce. Here, we propose ScoreCAM-U-Net, a pipeline to segment artifactual regions in brightfield images with limited user input. The model is trained using only image-level labels, so the process is faster by orders of magnitude compared to pixel-level annotation, but without substantially sacrificing the segmentation performance. We confirm that artifacts indeed exist with different shapes and sizes in three different brightfield microscopy image datasets, and distort downstream analyses such as nuclei segmentation, morphometry and fluorescence intensity quantification. We then demonstrate that our automated artifact removal ameliorates this problem. Such rapid cleaning of acquired images using the power of deep learning models is likely to become a standard step for all large scale microscopy experiments.

Evaluating Very Deep Convolutional Neural Networks for Nucleus Segmentation from Brightfield Cell Microscopy Images.

Advances in microscopy have increased output data volumes, and powerful image analysis methods are required to match. In particular, finding and characterizing nuclei from microscopy images, a core cytometry task, remains difficult to automate. While deep learning models have given encouraging results on this problem, the most powerful approaches have not yet been tested for attacking it. Here, we review and evaluate state-of-the-art very deep convolutional neural network architectures and training strategies for segmenting nuclei from brightfield cell images. We tested U-Net as a baseline model; considered U-Net++, Tiramisu, and DeepLabv3+ as latest instances of advanced families of segmentation models; and propose PPU-Net, a novel light-weight alternative. The deeper architectures outperformed standard U-Net and results from previous studies on the challenging brightfield images, with balanced pixel-wise accuracies of up to 86%. PPU-Net achieved this performance with 20-fold fewer parameters than the comparably accurate methods. All models perform better on larger nuclei and in sparser images. We further confirmed that in the absence of plentiful training data, augmentation and pretraining on other data improve performance. In particular, using only 16 images with data augmentation is enough to achieve a pixel-wise F1 score that is within 5% of the one achieved with a full data set for all models. The remaining segmentation errors are mainly due to missed nuclei in dense regions, overlapping cells, and imaging artifacts, indicating the major outstanding challenges.

Practical segmentation of nuclei in brightfield cell images with neural networks trained on fluorescently labelled samples.

Identifying nuclei is a standard first step when analysing cells in microscopy images. The traditional approach relies on signal from a DNA stain, or fluorescent transgene expression localised to the nucleus. However, imaging techniques that do not use fluorescence can also carry useful information. Here, we used brightfield and fluorescence images of fixed cells with fluorescently labelled DNA, and confirmed that three convolutional neural network architectures can be adapted to segment nuclei from the brightfield channel, relying on fluorescence signal to extract the ground truth for training. We found that U-Net achieved the best overall performance, Mask R-CNN provided an additional benefit of instance segmentation, and that DeepCell proved too slow for practical application. We trained the U-Net architecture on over 200 dataset variations, established that accurate segmentation is possible using as few as 16 training images, and that models trained on images from similar cell lines can extrapolate well. Acquiring data from multiple focal planes further helps distinguish nuclei in the samples. Overall, our work helps to liberate a fluorescence channel reserved for nuclear staining, thus providing more information from the specimen, and reducing reagents and time required for preparing imaging experiments.

Flat field correction for high-throughput imaging of fluorescent samples.

Vignetting of microscopic images impacts both the visual impression of the images and any image analysis applied to it. Especially in high-throughput screening high demands are made on an automated image analysis. In our work we focused on fluorescent samples and found that two profiles (background and foreground) for each imaging channel need to be estimated to achieve a sufficiently flat image after correction. We have developed a method which runs completely unsupervised on a wide range of assays. By adding a reliable internal quality control we mitigate the risk of introducing artefacts into sample images through correction. The method requires hundreds of images for the foreground profile, thus limiting its application to high-throughput screening where this requirement is fulfilled in routine operation.

A novel method for analyzing [Ca2+] flux kinetics in high-throughput screening.

Driven by multiparameter fluorescence readouts and the analysis of kinetic responses from biological assay systems, the amount and complexity of high-throughput screening data are constantly increasing. As a consequence, the reduction of data to a simple number, reflecting a percentage activity/inhibition, is no longer an adequate approach because valuable additional information, for example, about compound-or process-induced artifacts, is lost. Time series data such as the transient calcium flux observed after activation of Gq-coupled G protein-coupled receptors (GPCRs), are especially challenging with respect to quantity of data; typically, responses are followed for several minutes. Based on measurements taken on the fluorometric imaging plate reader, the authors have introduced a mathematical model to describe the time traces of cellular calcium fluxes mediated by the activation of GPCRs. The model describes the time series using 13 parameters, reducing the amount of data by 90% while guiding the detection of compound-induced artifacts as well as the selection of compounds for further characterization.

Calculation of photon-count number distributions via master equations.

Fitting of photon-count number histograms is a way of analysis of fluorescence intensity fluctuations, a successor to fluorescence correlation spectroscopy. First versions of the theory for calculating photon-count number distributions have assumed constant emission intensity by a molecule during a counting time interval. For a long time a question has remained unanswered: to what extent is this assumption violated in experiments? Here we present a theory of photon-count number distributions that takes account of intensity fluctuations during a counting time interval. Theoretical count-number distributions are calculated via a numerical solution of Master equations (ME), which is a set of differential equations describing diffusion, singlet-triplet transitions, and photon emission. Detector afterpulsing and dead-time corrections are also included. The ME-theory is tested by fitting a series of photon-count number histograms corresponding to different lengths of the counting time interval. Compared to the first version of fluorescence intensity multiple distribution analysis theory introduced in 2000, the fit quality is significantly improved. It is discussed how a theory of photon-count number distributions, which assumes constant emission intensity during a counting time interval, may also yield a good fit quality. We argue that the spatial brightness distribution used in calculations of the fit curve is not the true spatial brightness distribution. Instead, a number of dynamic processes, which cause fluorescence intensity fluctuations, are indirectly taken into account via the profile adjustment parameters.

Fluorescence intensity and lifetime distribution analysis: toward higher accuracy in fluorescence fluctuation spectroscopy.

Fluorescence fluctuation methods such as fluorescence correlation spectroscopy and fluorescence intensity distribution analysis (FIDA) have proven to be versatile tools for studying molecular interactions with single molecule sensitivity. Another well-known fluorescence technique is the measurement of the fluorescence lifetime. Here, we introduce a method that combines the benefits of both FIDA and fluorescence lifetime analysis. It is based on fitting the two-dimensional histogram of the number of photons detected in counting time intervals of given width and the sum of excitation to detection delay times of these photons. Referred to as fluorescence intensity and lifetime distribution analysis (FILDA), the technique distinguishes fluorescence species on the basis of both their specific molecular brightness and the lifetime of the excited state and is also able to determine absolute fluorophore concentrations. The combined information yielded by FILDA results in significantly increased accuracy compared to that of FIDA or fluorescence lifetime analysis alone. In this paper, the theory of FILDA is elaborated and applied to both simulated and experimental data. The outstanding power of this technique in resolving different species is shown by quantifying the binding of calmodulin to a peptide ligand, thus indicating the potential for application of FILDA to similar problems in the life sciences.