Establishing a reference focal plane using convolutional neural networks and beads for brightfield imaging.

Repeatability of measurements from image analytics is difficult, due to the heterogeneity and complexity of cell samples, exact microscope stage positioning, and slide thickness. We present a method to define and use a reference focal plane that provides repeatable measurements with very high accuracy, by relying on control beads as reference material and a convolutional neural network focused on the control bead images. Previously we defined a reference effective focal plane (REFP) based on the image gradient of bead edges and three specific bead image features. This paper both generalizes and improves on this previous work. First, we refine the definition of the REFP by fitting a cubic spline to describe the relationship between the distance from a bead's center and pixel intensity and by sharing information across experiments, exposures, and fields of view. Second, we remove our reliance on image features that behave differently from one instrument to another. Instead, we apply a convolutional regression neural network (ResNet 18) trained on cropped bead images that is generalizable to multiple microscopes. Our ResNet 18 network predicts the location of the REFP with only a single inferenced image acquisition that can be taken across a wide range of focal planes and exposure times. We illustrate the different strategies and hyperparameter optimization of the ResNet 18 to achieve a high prediction accuracy with an uncertainty for every image tested coming within the microscope repeatability measure of 7.5 µm from the desired focal plane. We demonstrate the generalizability of this methodology by applying it to two different optical systems and show that this level of accuracy can be achieved using only 6 beads per image.

High-volume, label-free imaging for quantifying single-cell dynamics in induced pluripotent stem cell colonies.

To facilitate the characterization of unlabeled induced pluripotent stem cells (iPSCs) during culture and expansion, we developed an AI pipeline for nuclear segmentation and mitosis detection from phase contrast images of individual cells within iPSC colonies. The analysis uses a 2D convolutional neural network (U-Net) plus a 3D U-Net applied on time lapse images to detect and segment nuclei, mitotic events, and daughter nuclei to enable tracking of large numbers of individual cells over long times in culture. The analysis uses fluorescence data to train models for segmenting nuclei in phase contrast images. The use of classical image processing routines to segment fluorescent nuclei precludes the need for manual annotation. We optimize and evaluate the accuracy of automated annotation to assure the reliability of the training. The model is generalizable in that it performs well on different datasets with an average F1 score of 0.94, on cells at different densities, and on cells from different pluripotent cell lines. The method allows us to assess, in a non-invasive manner, rates of mitosis and cell division which serve as indicators of cell state and cell health. We assess these parameters in up to hundreds of thousands of cells in culture for more than 36 hours, at different locations in the colonies, and as a function of excitation light exposure.

Harmonizing the Generation and Pre-publication Stewardship of FAIR Image data.

Together with the molecular knowledge of genes and proteins, biological images promise to significantly enhance the scientific understanding of complex cellular systems and to advance predictive and personalized therapeutic products for human health. For this potential to be realized, quality-assured image data must be shared among labs at a global scale to be compared, pooled, and reanalyzed, thus unleashing untold potential beyond the original purpose for which the data was generated. There are two broad sets of requirements to enable image data sharing in the life sciences. One set of requirements is articulated in the companion White Paper entitled "Enabling Global Image Data Sharing in the Life Sciences," which is published in parallel and addresses the need to build the cyberinfrastructure for sharing the digital array data (arXiv:2401.13023 [q-bio.OT], https://doi.org/10.48550/arXiv.2401.13023). In this White Paper, we detail a broad set of requirements, which involves collecting, managing, presenting, and propagating contextual information essential to assess the quality, understand the content, interpret the scientific implications, and reuse image data in the context of the experimental details. We start by providing an overview of the main lessons learned to date through international community activities, which have recently made considerable progress toward generating community standard practices for imaging Quality Control (QC) and metadata. We then provide a clear set of recommendations for amplifying this work. The driving goal is to address remaining challenges, and democratize access to common practices and tools for a spectrum of biomedical researchers, regardless of their expertise, access to resources, and geographical location.

Contemporaneous sample data tracking for the generation of genome edited cell lines.

It is difficult to capture the large numbers of steps and details that often characterize research in the biomedical sciences. We present an approach that is based on commercial spreadsheet software so it is easily adaptable by the experimentalist. The approach is designed to be compatible with an experimentalist's workflow and allows the capture in real time of detailed information associated, in this use case, with laboratory actions involved in the process of editing, enriching and isolating clonal gene-edited pluripotent stem cell (PSC) lines. Intuitive features and flexibility allow an experimentalist without extensive programming knowledge to modify spreadsheets in response to changes in protocols and to perform simple queries. The experimental details are collated in a table format from which they can be exported in open standard formats (e.g., Extensible Markup Language (XML) or Comma Separated Values (CSV) for ingestion into a data repository supporting interoperability with other applications. We demonstrate a sample- and file-naming convention that enables the automated creation of file directory folders with human readable semantic titles within a local file system. These operations facilitate the local organization of documentation and data for each cell line derived from each transfection in designated folder/file locations. This approach is generalizable to experimental applications beyond this use case.

Establishing a reference focal plane using beads for trypan-blue-based viability measurements.

Trypan blue dye exclusion-based cell viability measurements are highly dependent upon image quality and consistency. In order to make measurements repeatable, one must be able to reliably capture images at a consistent focal plane, and with signal-to-noise ratio within appropriate limits to support proper execution of image analysis routines. Imaging chambers and imaging systems used for trypan blue analysis can be inconsistent or can drift over time, leading to a need to assure the acquisition of images prior to automated image analysis. Although cell-based autofocus techniques can be applied, the heterogeneity and complexity of the cell samples can make it difficult to assure the effectiveness, repeatability and accuracy of the routine for each measurement. Instead of auto-focusing on cells in our images, we add control beads to the images, and use them to repeatedly return to a reference focal plane. We use bead image features that have stable profiles across a wide range of focal values and exposure levels. We created a predictive model based on image quality features computed over reference datasets. Because the beads have little variation, we can determine the reference plane from bead image features computed over a single-shot image and can reproducibly return to that reference plane with each sample. The achieved accuracy (over 95%) is within the limits of the actuator repeatability. We demonstrate that a small number of beads (less than 3 beads per image) is needed to achieve this accuracy. We have also developed an open-source Graphical User Interface called Bead Benchmarking-Focus And Intensity Tool (BB-FAIT) to implement these methods for a semi-automated cell viability analyser.

It is critical for the manufacturing and release of living cell-based therapies to determine the viability, the ratio of living cells to the total number of cells (live and dead), in the therapy. Dead cells can be a safety concern for the patient, and dosing is often based on the number of living cells which are the active ingredient of the drug product. Currently, the most common approach to evaluating cell viability is based on the staining of cell samples with the trypan blue marker of cell membrane integrity: a loss in cell membrane integrity with cell death allows the dye into the cell, which can be seen using brightfield microscopy. To classify cells as live/dead, the brightness of the cells is evaluated and cells with bright centres are considered live, while those with dark centres are considered dead. Unfortunately, this approach of staining, imaging and classification is very sensitive to image acquisition settings, including image focus and brightness. This paper introduces a method to establish the required image quality for image viability analysis, providing a tool to return to image acquisition settings that will ensure image quality even when there is variability from sample to sample. In this method, polymeric beads are added to each cell sample prior to cell viability analysis. Using image processing, we extract key features from the beads in the image such as sharpness of the edges of the beads. The image features of the cells can vary significantly from sample to sample and under different cell conditions, but image features of beads have proved to be consistent across samples. We are thus able to collect reference datasets quantifying bead features over a wide range of image acquisition settings (brightness and focus), allowing us to establish a reference focal plan for image acquisition for any cell sample based on bead features. We show that with as few as three beads per image, the reference focal plane can be found from a single acquisition of beads image data over a wide range of image focuses and brightness, allowing users to consistently acquire images for cell viability that meet pre-defined quality requirements.

Information Thermodynamics and Reducibility of Large Gene Networks.

Gene regulatory networks (GRNs) control biological processes like pluripotency, differentiation, and apoptosis. Omics methods can identify a large number of putative network components (on the order of hundreds or thousands) but it is possible that in many cases a small subset of genes control the state of GRNs. Here, we explore how the topology of the interactions between network components may indicate whether the effective state of a GRN can be represented by a small subset of genes. We use methods from information theory to model the regulatory interactions in GRNs as cascading and superposing information channels. We propose an information loss function that enables identification of the conditions by which a small set of genes can represent the state of all the other genes in the network. This information-theoretic analysis extends to a measure of free energy change due to communication within the network, which provides a new perspective on the reducibility of GRNs. Both the information loss and relative free energy depend on the density of interactions and edge communication error in a network. Therefore, this work indicates that a loss in mutual information between genes in a GRN is directly coupled to a thermodynamic cost, i.e., a reduction of relative free energy, of the system.

Practical application of microsphere samples for benchmarking a quantitative phase imaging system.

Quantitative phase imaging (QPI) provides an approach for monitoring the dry mass of individual cells by measuring the optical pathlength of visible light as it passes through cells. A distinct advantage of QPI is that the measurements result in optical path length quantities that are, in principle, instrument independent. Reference materials that induce a well-defined optical pathlength shift and are compatible with QPI imaging systems will be valuable in assuring the accuracy of such measurements on different instruments. In this study, we evaluate seven combinations of microspheres embedded in index refraction matching media as candidate reference materials for benchmarking the performance of a QPI system and as calibration standards for the optical pathlength measurement. Poly(methyl metharylate) microspheres and mineral oil were used to evaluate the range of illumination apertures, signal-to-noise ratios, and focus positions that allow an accurate quantitative optical pathlength measurement. The microsphere-based reference material can be used to verify settings on an instrument that are suitable for obtaining an accurate pathlength measurement from biological cells. The microsphere/media reference material is applied to QPI-based dry mass measurements of a population of HEK293 cells to benchmark and provide evidence that the QPI image data are accurate.

Probing pluripotency gene regulatory networks with quantitative live cell imaging.

Live cell imaging uniquely enables the measurement of dynamic events in single cells, but it has not been used often in the study of gene regulatory networks. Network components can be examined in relation to one another by quantitative live cell imaging of fluorescent protein reporter cell lines that simultaneously report on more than one network component. A series of dual-reporter cell lines would allow different combinations of network components to be examined in individual cells. Dynamical information about interacting network components in individual cells is critical to predictive modeling of gene regulatory networks, and such information is not accessible through omics and other end point techniques. Achieving this requires that gene-edited cell lines are appropriately designed and adequately characterized to assure the validity of the biological conclusions derived from the expression of the reporters. In this brief review we discuss what is known about the importance of dynamics to network modeling and review some recent advances in optical microscopy methods and image analysis approaches that are making the use of quantitative live cell imaging for network analysis possible. We also discuss how strategies for genetic engineering of reporter cell lines can influence the biological relevance of the data.

The role of fluctuations in determining cellular network thermodynamics.

The steady state distributions of phenotypic responses within an isogenic population of cells result from both deterministic and stochastic characteristics of biochemical networks. A biochemical network can be characterized by a multidimensional potential landscape based on the distribution of responses and a diffusion matrix of the correlated dynamic fluctuations between N-numbers of intracellular network variables. In this work, we develop a thermodynamic description of biological networks at the level of microscopic interactions between network variables. The Boltzmann H-function defines the rate of free energy dissipation of a network system and provides a framework for determining the heat associated with the nonequilibrium steady state and its network components. The magnitudes of the landscape gradients and the dynamic correlated fluctuations of network variables are experimentally accessible. We describe the use of Fokker-Planck dynamics to calculate housekeeping heat from the experimental data by a method that we refer to as Thermo-FP. The method provides insight into the composition of the network and the relative thermodynamic contributions from network components. We surmise that these thermodynamic quantities allow determination of the relative importance of network components to overall network control. We conjecture that there is an upper limit to the rate of dissipative heat produced by a biological system that is associated with system size or modularity, and we show that the dissipative heat has a lower bound.

Mass Measurements of Focal Adhesions in Single Cells Using High Resolution Surface Plasmon Resonance Microscopy.

Surface plasmon resonance microscopy (SPRM) is a powerful label-free imaging technique with spatial resolution approaching the optical diffraction limit. The high sensitivity of SPRM to small changes in index of refraction at an interface allows imaging of dynamic protein structures within a cell. Visualization of subcellular features, such as focal adhesions (FAs), can be performed on live cells using a high numerical aperture objective lens with a digital light projector to precisely position the incident angle of the excitation light. Within the cell-substrate region of the SPRM image, punctate regions of high contrast are putatively identified as the cellular FAs. Optical parameter analysis is achieved by application of the Fresnel model to the SPRM data and resulting refractive index measurements are used to calculate protein density and mass. FAs are known to be regions of high protein density that reside at the cell-substratum interface. Comparing SPRM with fluorescence images of antibody stained for vinculin, a component in FAs, reveals similar measurements of FA size. In addition, a positive correlation between FA size and protein density is revealed by SPRM. Comparing SPRM images for two cell types reveals a distinct difference in the protein density and mass of their respective FAs. Application of SPRM to quantify mass can greatly aid monitoring basic processes that control FA mass and growth and contribute to accurate models that describe cell-extracellular interactions.

A novel measure and significance testing in data analysis of cell image segmentation.

BACKGROUND: Cell image segmentation (CIS) is an essential part of quantitative imaging of biological cells. Designing a performance measure and conducting significance testing are critical for evaluating and comparing the CIS algorithms for image-based cell assays in cytometry. Many measures and methods have been proposed and implemented to evaluate segmentation methods. However, computing the standard errors (SE) of the measures and their correlation coefficient is not described, and thus the statistical significance of performance differences between CIS algorithms cannot be assessed. RESULTS: We propose the total error rate (TER), a novel performance measure for segmenting all cells in the supervised evaluation. The TER statistically aggregates all misclassification error rates (MER) by taking cell sizes as weights. The MERs are for segmenting each single cell in the population. The TER is fully supported by the pairwise comparisons of MERs using 106 manually segmented ground-truth cells with different sizes and seven CIS algorithms taken from ImageJ. Further, the SE and 95% confidence interval (CI) of TER are computed based on the SE of MER that is calculated using the bootstrap method. An algorithm for computing the correlation coefficient of TERs between two CIS algorithms is also provided. Hence, the 95% CI error bars can be used to classify CIS algorithms. The SEs of TERs and their correlation coefficient can be employed to conduct the hypothesis testing, while the CIs overlap, to determine the statistical significance of the performance differences between CIS algorithms. CONCLUSIONS: A novel measure TER of CIS is proposed. The TER's SEs and correlation coefficient are computed. Thereafter, CIS algorithms can be evaluated and compared statistically by conducting the significance testing.

Lineage mapper: A versatile cell and particle tracker.

The ability to accurately track cells and particles from images is critical to many biomedical problems. To address this, we developed Lineage Mapper, an open-source tracker for time-lapse images of biological cells, colonies, and particles. Lineage Mapper tracks objects independently of the segmentation method, detects mitosis in confluence, separates cell clumps mistakenly segmented as a single cell, provides accuracy and scalability even on terabyte-sized datasets, and creates division and/or fusion lineages. Lineage Mapper has been tested and validated on multiple biological and simulated problems. The software is available in ImageJ and Matlab at isg.nist.gov.

Surface plasmon resonance microscopy: Achieving a quantitative optical response.

Surface plasmon resonance (SPR) imaging allows real-time label-free imaging based on index of refraction and changes in index of refraction at an interface. Optical parameter analysis is achieved by application of the Fresnel model to SPR data typically taken by an instrument in a prism based figuration. We carry out SPR imaging on a microscope by launching light into a sample and collecting reflected light through a high numerical aperture microscope objective. The SPR microscope enables spatial resolution that approaches the diffraction limit and has a dynamic range that allows detection of subnanometer to submicrometer changes in thickness of biological material at a surface. However, unambiguous quantitative interpretation of SPR changes using the microscope system could not be achieved using the Fresnel model because of polarization dependent attenuation and optical aberration that occurs in the high numerical aperture objective. To overcome this problem, we demonstrate a model to correct for polarization diattenuation and optical aberrations in the SPR data and develop a procedure to calibrate reflectivity to index of refraction values. The calibration and correction strategy for quantitative analysis was validated by comparing the known indices of refraction of bulk materials with corrected SPR data interpreted with the Fresnel model. Subsequently, we applied our SPR microscopy method to evaluate the index of refraction for a series of polymer microspheres in aqueous media and validated the quality of the measurement with quantitative phase microscopy.

Large-scale time-lapse microscopy of Oct4 expression in human embryonic stem cell colonies.

Identification and quantification of the characteristics of stem cell preparations is critical for understanding stem cell biology and for the development and manufacturing of stem cell based therapies. We have developed image analysis and visualization software that allows effective use of time-lapse microscopy to provide spatial and dynamic information from large numbers of human embryonic stem cell colonies. To achieve statistically relevant sampling, we examined >680 colonies from 3 different preparations of cells over 5days each, generating a total experimental dataset of 0.9 terabyte (TB). The 0.5 Giga-pixel images at each time point were represented by multi-resolution pyramids and visualized using the Deep Zoom Javascript library extended to support viewing Giga-pixel images over time and extracting data on individual colonies. We present a methodology that enables quantification of variations in nominally-identical preparations and between colonies, correlation of colony characteristics with Oct4 expression, and identification of rare events.

Survey statistics of automated segmentations applied to optical imaging of mammalian cells.

BACKGROUND: The goal of this survey paper is to overview cellular measurements using optical microscopy imaging followed by automated image segmentation. The cellular measurements of primary interest are taken from mammalian cells and their components. They are denoted as two- or three-dimensional (2D or 3D) image objects of biological interest. In our applications, such cellular measurements are important for understanding cell phenomena, such as cell counts, cell-scaffold interactions, cell colony growth rates, or cell pluripotency stability, as well as for establishing quality metrics for stem cell therapies. In this context, this survey paper is focused on automated segmentation as a software-based measurement leading to quantitative cellular measurements. METHODS: We define the scope of this survey and a classification schema first. Next, all found and manually filteredpublications are classified according to the main categories: (1) objects of interests (or objects to be segmented), (2) imaging modalities, (3) digital data axes, (4) segmentation algorithms, (5) segmentation evaluations, (6) computational hardware platforms used for segmentation acceleration, and (7) object (cellular) measurements. Finally, all classified papers are converted programmatically into a set of hyperlinked web pages with occurrence and co-occurrence statistics of assigned categories. RESULTS: The survey paper presents to a reader: (a) the state-of-the-art overview of published papers about automated segmentation applied to optical microscopy imaging of mammalian cells, (b) a classification of segmentation aspects in the context of cell optical imaging, (c) histogram and co-occurrence summary statistics about cellular measurements, segmentations, segmented objects, segmentation evaluations, and the use of computational platforms for accelerating segmentation execution, and (d) open research problems to pursue. CONCLUSIONS: The novel contributions of this survey paper are: (1) a new type of classification of cellular measurements and automated segmentation, (2) statistics about the published literature, and (3) a web hyperlinked interface to classification statistics of the surveyed papers at https://isg.nist.gov/deepzoomweb/resources/survey/index.html.

High resolution surface plasmon resonance imaging for single cells.

BACKGROUND: Surface plasmon resonance imaging (SPRI) is a label-free technique that can image refractive index changes at an interface. We have previously used SPRI to study the dynamics of cell-substratum interactions. However, characterization of spatial resolution in 3 dimensions is necessary to quantitatively interpret SPR images. Spatial resolution is complicated by the asymmetric propagation length of surface plasmons in the x and y dimensions leading to image degradation in one direction. Inferring the distance of intracellular organelles and other subcellular features from the interface by SPRI is complicated by uncertainties regarding the detection of the evanescent wave decay into cells. This study provides an experimental basis for characterizing the resolution of an SPR imaging system in the lateral and distal dimensions and demonstrates a novel approach for resolving sub-micrometer cellular structures by SPRI. The SPRI resolution here is distinct in its ability to visualize subcellular structures that are in proximity to a surface, which is comparable with that of total internal reflection fluorescence (TIRF) microscopy but has the advantage of no fluorescent labels. RESULTS: An SPR imaging system was designed that uses a high numerical aperture objective lens to image cells and a digital light projector to pattern the angle of the incident excitation on the sample. Cellular components such as focal adhesions, nucleus, and cellular secretions are visualized. The point spread function of polymeric nanoparticle beads indicates near-diffraction limited spatial resolution. To characterize the z-axis response, we used micrometer scale polymeric beads with a refractive index similar to cells as reference materials to determine the detection limit of the SPR field as a function of distance from the substrate. Multi-wavelength measurements of these microspheres show that it is possible to tailor the effective depth of penetration of the evanescent wave into the cellular environment. CONCLUSION: We describe how the use of patterned incident light provides SPRI at high spatial resolution, and we characterize a finite limit of detection for penetration depth. We demonstrate the application of a novel technique that allows unprecedented subcellular detail for SPRI, and enables a quantitative interpretation of SPRI for subcellular imaging.

Nonparametric estimates of drift and diffusion profiles via Fokker-Planck algebra.

Diffusion processes superimposed upon deterministic motion play a key role in understanding and controlling the transport of matter, energy, momentum, and even information in physics, chemistry, material science, biology, and communications technology. Given functions defining these random and deterministic components, the Fokker-Planck (FP) equation is often used to model these diffusive systems. Many methods exist for estimating the drift and diffusion profiles from one or more identifiable diffusive trajectories; however, when many identical entities diffuse simultaneously, it may not be possible to identify individual trajectories. Here we present a method capable of simultaneously providing nonparametric estimates for both drift and diffusion profiles from evolving density profiles, requiring only the validity of Langevin/FP dynamics. This algebraic FP manipulation provides a flexible and robust framework for estimating stationary drift and diffusion coefficient profiles, is not based on fluctuation theory or solved diffusion equations, and may facilitate predictions for many experimental systems. We illustrate this approach on experimental data obtained from a model lipid bilayer system exhibiting free diffusion and electric field induced drift. The wide range over which this approach provides accurate estimates for drift and diffusion profiles is demonstrated through simulation.

An automated protocol for performance benchmarking a widefield fluorescence microscope.

Widefield fluorescence microscopy is a highly used tool for visually assessing biological samples and for quantifying cell responses. Despite its widespread use in high content analysis and other imaging applications, few published methods exist for evaluating and benchmarking the analytical performance of a microscope. Easy-to-use benchmarking methods would facilitate the use of fluorescence imaging as a quantitative analytical tool in research applications, and would aid the determination of instrumental method validation for commercial product development applications. We describe and evaluate an automated method to characterize a fluorescence imaging system's performance by benchmarking the detection threshold, saturation, and linear dynamic range to a reference material. The benchmarking procedure is demonstrated using two different materials as the reference material, uranyl-ion-doped glass and Schott 475 GG filter glass. Both are suitable candidate reference materials that are homogeneously fluorescent and highly photostable, and the Schott 475 GG filter glass is currently commercially available. In addition to benchmarking the analytical performance, we also demonstrate that the reference materials provide for accurate day to day intensity calibration. Published 2014 Wiley Periodicals Inc.

Testing Equality of Cell Populations Based on Shape and Geodesic Distance.

Image cytometry has emerged as a valuable in vitro screening tool and advances in automated microscopy have made it possible to readily analyze large cellular populations of image data. The purpose of this paper is to illustrate the viability of using cell shape to test equality of cell populations based on image data. Shape space theory is reviewed, from which differences between shapes can be quantified in terms of geodesic distance. Several multivariate nonparametric statistical hypothesis tests are adapted to test equality of cell populations. It is illustrated that geodesic distance can be a better feature than cell spread area and roundness in distinguishing between cell populations. Tests based on geodesic distance are able to detect natural perturbations of cells, whereas Kolmogorov-Smirnov tests based on area and roundness are not.

Boltzmann's H-function and diffusion processes.

There exists a generalization of Boltzmann's H-function that allows for nonuniformly populated stationary states, which may exist far from thermodynamic equilibrium. Here we describe a method for obtaining a generalized or collective diffusion coefficient D directly from this H-function, the only constraints being that the relaxation process is Markov (short memory), continuous in the reaction coordinate, and local in the sense of a flux/force relationship. As an application of this H-function method, we simulate the self-consistent extraction of D via Langevin/Fokker-Planck (L/FP) dynamics on various potential energy landscapes. We observe that the initial epoch of relaxation, which is far removed from the stationary state, provides the most reliable estimates of D. The construction of an H-function that guarantees conformity with the second law of thermodynamics has been generalized to allow for diffusion coefficients that may depend on both the reaction coordinate and time, and the extension to an arbitrary number of reaction coordinates is straightforward. For this multidimensional case, the diffusion tensor must be positive definite in the sense that its eigenvalues must be real and positive. To illustrate the behavior of the proposed collective diffusion coefficient, we simulate the H-function for a variety of Langevin systems. In particular, the impacts on H and D of landscape shape, sample size, selection of an initial distribution, finite dynamic observation range, stochastic correlations, and short/long-term memory effects are examined.