EVATOM: an optical, label-free, machine learning assisted embryo health assessment tool.

The combination of a good quality embryo and proper maternal health factors promise higher chances of a successful in vitro fertilization (IVF) procedure leading to clinical pregnancy and live birth. Of these two factors, selection of a good embryo is a controllable aspect. The current gold standard in clinical practice is visual assessment of an embryo based on its morphological appearance by trained embryologists. More recently, machine learning has been incorporated into embryo selection "packages". Here, we report EVATOM: a machine-learning assisted embryo health assessment tool utilizing an optical quantitative phase imaging technique called artificial confocal microscopy (ACM). We present a label-free nucleus detection method with, to the best of our knowledge, novel quantitative embryo health biomarkers. Two viability assessment models are presented for grading embryos into two classes: healthy/intermediate (H/I) or sick (S) class. The models achieve a weighted F1 score of 1.0 and 0.99 respectively on the in-distribution test set of 72 fixed embryos and a weighted F1 score of 0.9 and 0.95 respectively on the out-of-distribution test dataset of 19 time-instances from 8 live embryos.

Error estimates and physics informed augmentation of neural networks for thermally coupled incompressible Navier Stokes equations.

Physics Informed Neural Networks (PINNs) are shown to be a promising method for the approximation of partial differential equations (PDEs). PINNs approximate the PDE solution by minimizing physics-based loss functions over a given domain. Despite substantial progress in the application of PINNs to a range of problem classes, investigation of error estimation and convergence properties of PINNs, which is important for establishing the rationale behind their good empirical performance, has been lacking. This paper presents convergence analysis and error estimates of PINNs for a multi-physics problem of thermally coupled incompressible Navier-Stokes equations. Through a model problem of Beltrami flow it is shown that a small training error implies a small generalization error. Posteriori convergence rates of total error with respect to the training residual and collocation points are presented. This is of practical significance in determining appropriate number of training parameters and training residual thresholds to get good PINNs prediction of thermally coupled steady state laminar flows. These convergence rates are then generalized to different spatial geometries as well as to different flow parameters that lie in the laminar regime. A pressure stabilization term in the form of pressure Poisson equation is added to the PDE residuals for PINNs. This physics informed augmentation is shown to improve accuracy of the pressure field by an order of magnitude as compared to the case without augmentation. Results from PINNs are compared to the ones obtained from stabilized finite element method and good properties of PINNs are highlighted.

Machine learning assisted health viability assay for mouse embryos with artificial confocal microscopy (ACM).

The combination of a good quality embryo and proper maternal health factors promise higher chances of a successful in vitro fertilization (IVF) procedure leading to clinical pregnancy and live birth. Of these two factors, selection of a good embryo is a controllable aspect. The current gold standard in clinical practice is visual assessment of an embryo based on its morphological appearance by trained embryologists. More recently, machine learning has been incorporated into embryo selection "packages". Here, we report a machine-learning assisted embryo health assessment tool utilizing a quantitative phase imaging technique called artificial confocal microscopy (ACM). We present a label-free nucleus detection method with novel quantitative embryo health biomarkers. Two viability assessment models are presented for grading embryos into two classes: healthy/intermediate (H/I) or sick (S) class. The models achieve a weighted F1 score of 1.0 and 0.99 respectively on the in-distribution test set of 72 fixed embryos and a weighted F1 score of 0.9 and 0.95 respectively on the out-of-distribution test dataset of 19 time-instances from 8 live embryos.

White blood cell detection, classification and analysis using phase imaging with computational specificity (PICS).

Treatment of blood smears with Wright's stain is one of the most helpful tools in detecting white blood cell abnormalities. However, to diagnose leukocyte disorders, a clinical pathologist must perform a tedious, manual process of locating and identifying individual cells. Furthermore, the staining procedure requires considerable preparation time and clinical infrastructure, which is incompatible with point-of-care diagnosis. Thus, rapid and automated evaluations of unlabeled blood smears are highly desirable. In this study, we used color spatial light interference microcopy (cSLIM), a highly sensitive quantitative phase imaging (QPI) technique, coupled with deep learning tools, to localize, classify and segment white blood cells (WBCs) in blood smears. The concept of combining QPI label-free data with AI for the purpose of extracting cellular specificity has recently been introduced in the context of fluorescence imaging as phase imaging with computational specificity (PICS). We employed AI models to first translate SLIM images into brightfield micrographs, then ran parallel tasks of locating and labelling cells using EfficientNet, which is an object detection model. Next, WBC binary masks were created using U-net, a convolutional neural network that performs precise segmentation. After training on digitally stained brightfield images of blood smears with WBCs, we achieved a mean average precision of 75% for localizing and classifying neutrophils, eosinophils, lymphocytes, and monocytes, and an average pixel-wise majority-voting F1 score of 80% for determining the cell class from semantic segmentation maps. Therefore, PICS renders and analyzes synthetically stained blood smears rapidly, at a reduced cost of sample preparation, providing quantitative clinical information.

Cell Cycle Stage Classification Using Phase Imaging with Computational Specificity.

Traditional methods for cell cycle stage classification rely heavily on fluorescence microscopy to monitor nuclear dynamics. These methods inevitably face the typical phototoxicity and photobleaching limitations of fluorescence imaging. Here, we present a cell cycle detection workflow using the principle of phase imaging with computational specificity (PICS). The proposed method uses neural networks to extract cell cycle-dependent features from quantitative phase imaging (QPI) measurements directly. Our results indicate that this approach attains very good accuracy in classifying live cells into G1, S, and G2/M stages, respectively. We also demonstrate that the proposed method can be applied to study single-cell dynamics within the cell cycle as well as cell population distribution across different stages of the cell cycle. We envision that the proposed method can become a nondestructive tool to analyze cell cycle progression in fields ranging from cell biology to biopharma applications.

Automatic Colorectal Cancer Screening Using Deep Learning in Spatial Light Interference Microscopy Data.

The surgical pathology workflow currently adopted by clinics uses staining to reveal tissue architecture within thin sections. A trained pathologist then conducts a visual examination of these slices and, since the investigation is based on an empirical assessment, a certain amount of subjectivity is unavoidable. Furthermore, the reliance on external contrast agents such as hematoxylin and eosin (H&E), albeit being well-established methods, makes it difficult to standardize color balance, staining strength, and imaging conditions, hindering automated computational analysis. In response to these challenges, we applied spatial light interference microscopy (SLIM), a label-free method that generates contrast based on intrinsic tissue refractive index signatures. Thus, we reduce human bias and make imaging data comparable across instruments and clinics. We applied a mask R-CNN deep learning algorithm to the SLIM data to achieve an automated colorectal cancer screening procedure, i.e., classifying normal vs. cancerous specimens. Our results, obtained on a tissue microarray consisting of specimens from 132 patients, resulted in 91% accuracy for gland detection, 99.71% accuracy in gland-level classification, and 97% accuracy in core-level classification. A SLIM tissue scanner accompanied by an application-specific deep learning algorithm may become a valuable clinical tool, enabling faster and more accurate assessments by pathologists.

Label-free SARS-CoV-2 detection and classification using phase imaging with computational specificity.

Efforts to mitigate the COVID-19 crisis revealed that fast, accurate, and scalable testing is crucial for curbing the current impact and that of future pandemics. We propose an optical method for directly imaging unlabeled viral particles and using deep learning for detection and classification. An ultrasensitive interferometric method was used to image four virus types with nanoscale optical path-length sensitivity. Pairing these data with fluorescence images for ground truth, we trained semantic segmentation models based on U-Net, a particular type of convolutional neural network. The trained network was applied to classify the viruses from the interferometric images only, containing simultaneously SARS-CoV-2, H1N1 (influenza-A virus), HAdV (adenovirus), and ZIKV (Zika virus). Remarkably, due to the nanoscale sensitivity in the input data, the neural network was able to identify SARS-CoV-2 vs. the other viruses with 96% accuracy. The inference time for each image is 60 ms, on a common graphic-processing unit. This approach of directly imaging unlabeled viral particles may provide an extremely fast test, of less than a minute per patient. As the imaging instrument operates on regular glass slides, we envision this method as potentially testing on patient breath condensates. The necessary high throughput can be achieved by translating concepts from digital pathology, where a microscope can scan hundreds of slides automatically.

Label-free screening of brain tissue myelin content using phase imaging with computational specificity (PICS).

Inadequate myelination in the central nervous system is associated with neurodevelopmental complications. Thus, quantitative, high spatial resolution measurements of myelin levels are highly desirable. We used spatial light interference microcopy (SLIM), a highly sensitive quantitative phase imaging (QPI) technique, to correlate the dry mass content of myelin in piglet brain tissue with dietary changes and gestational size. We combined SLIM micrographs with an artificial intelligence (AI) classifying model that allows us to discern subtle disparities in myelin distributions with high accuracy. This concept of combining QPI label-free data with AI for the purpose of extracting molecular specificity has recently been introduced by our laboratory as phase imaging with computational specificity. Training on 8000 SLIM images of piglet brain tissue with the 71-layer transfer learning model Xception, we created a two-parameter classification to differentiate gestational size and diet type with an accuracy of 82% and 80%, respectively. To our knowledge, this type of evaluation is impossible to perform by an expert pathologist or other techniques.

Phase imaging with computational specificity (PICS) for measuring dry mass changes in sub-cellular compartments.

Due to its specificity, fluorescence microscopy has become a quintessential imaging tool in cell biology. However, photobleaching, phototoxicity, and related artifacts continue to limit fluorescence microscopy's utility. Recently, it has been shown that artificial intelligence (AI) can transform one form of contrast into another. We present phase imaging with computational specificity (PICS), a combination of quantitative phase imaging and AI, which provides information about unlabeled live cells with high specificity. Our imaging system allows for automatic training, while inference is built into the acquisition software and runs in real-time. Applying the computed fluorescence maps back to the quantitative phase imaging (QPI) data, we measured the growth of both nuclei and cytoplasm independently, over many days, without loss of viability. Using a QPI method that suppresses multiple scattering, we measured the dry mass content of individual cell nuclei within spheroids. In its current implementation, PICS offers a versatile quantitative technique for continuous simultaneous monitoring of individual cellular components in biological applications where long-term label-free imaging is desirable.

ChemNet: A Deep Neural Network for Advanced Composites Manufacturing.

Among advanced manufacturing techniques for fiber-reinforced polymer-matrix composites (FRPCs) which are critical for aerospace, marine, automotive, and energy industries, frontal polymerization (FP) has been recently proposed to save orders of magnitude of time and energy. However, the cure kinetics of the matrix phase, usually a thermosetting polymer, brings difficulty to the design and control of the process. Here, we develop a deep learning model, ChemNet, to solve an inverse problem for predicting and optimizing the cure kinetics parameters of the thermosetting FRPCs for a desired fabrication strategy. ChemNet consists of a fully connected FeedForward 9 layer deep neural network trained on one million examples, and predicts activation energy and reaction enthalpy given the front characteristics such as speed and maximum temperature. ChemNet provides highly accurate predictions measured by the mean squared error (MSE) and by the maximum absolute error (MAE) metrics. ChemNet's performance on the "hidden" test data set had an MSE of 5.58 × 10-6 and a MAE of 1 × 10-3.

Reproductive outcomes predicted by phase imaging with computational specificity of spermatozoon ultrastructure.

The ability to evaluate sperm at the microscopic level, at high-throughput, would be useful for assisted reproductive technologies (ARTs), as it can allow specific selection of sperm cells for in vitro fertilization (IVF). The tradeoff between intrinsic imaging and external contrast agents is particularly acute in reproductive medicine. The use of fluorescence labels has enabled new cell-sorting strategies and given new insights into developmental biology. Nevertheless, using extrinsic contrast agents is often too invasive for routine clinical operation. Raising questions about cell viability, especially for single-cell selection, clinicians prefer intrinsic contrast in the form of phase-contrast, differential-interference contrast, or Hoffman modulation contrast. While such instruments are nondestructive, the resulting image suffers from a lack of specificity. In this work, we provide a template to circumvent the tradeoff between cell viability and specificity by combining high-sensitivity phase imaging with deep learning. In order to introduce specificity to label-free images, we trained a deep-convolutional neural network to perform semantic segmentation on quantitative phase maps. This approach, a form of phase imaging with computational specificity (PICS), allowed us to efficiently analyze thousands of sperm cells and identify correlations between dry-mass content and artificial-reproduction outcomes. Specifically, we found that the dry-mass content ratios between the head, midpiece, and tail of the cells can predict the percentages of success for zygote cleavage and embryo blastocyst formation.

Deep Learning Enabled Strain Mapping of Single-Atom Defects in Two-Dimensional Transition Metal Dichalcogenides with Sub-Picometer Precision.

Two-dimensional (2D) materials offer an ideal platform to study the strain fields induced by individual atomic defects, yet challenges associated with radiation damage have so far limited electron microscopy methods to probe these atomic-scale strain fields. Here, we demonstrate an approach to probe single-atom defects with sub-picometer precision in a monolayer 2D transition metal dichalcogenide, WSe2-2xTe2x. We utilize deep learning to mine large data sets of aberration-corrected scanning transmission electron microscopy images to locate and classify point defects. By combining hundreds of images of nominally identical defects, we generate high signal-to-noise class averages which allow us to measure 2D atomic spacings with up to 0.2 pm precision. Our methods reveal that Se vacancies introduce complex, oscillating strain fields in the WSe2-2xTe2x lattice that correspond to alternating rings of lattice expansion and contraction. These results indicate the potential impact of computer vision for the development of high-precision electron microscopy methods for beam-sensitive materials.

Knowledge-guided analysis of "omics" data using the KnowEnG cloud platform.

We present Knowledge Engine for Genomics (KnowEnG), a free-to-use computational system for analysis of genomics data sets, designed to accelerate biomedical discovery. It includes tools for popular bioinformatics tasks such as gene prioritization, sample clustering, gene set analysis, and expression signature analysis. The system specializes in "knowledge-guided" data mining and machine learning algorithms, in which user-provided data are analyzed in light of prior information about genes, aggregated from numerous knowledge bases and encoded in a massive "Knowledge Network." KnowEnG adheres to "FAIR" principles (findable, accessible, interoperable, and reuseable): its tools are easily portable to diverse computing environments, run on the cloud for scalable and cost-effective execution, and are interoperable with other computing platforms. The analysis tools are made available through multiple access modes, including a web portal with specialized visualization modules. We demonstrate the KnowEnG system's potential value in democratization of advanced tools for the modern genomics era through several case studies that use its tools to recreate and expand upon the published analysis of cancer data sets.

Label-free colorectal cancer screening using deep learning and spatial light interference microscopy (SLIM).

Current pathology workflow involves staining of thin tissue slices, which otherwise would be transparent, followed by manual investigation under the microscope by a trained pathologist. While the hematoxylin and eosin (H&E) stain is well-established and a cost-effective method for visualizing histology slides, its color variability across preparations and subjectivity across clinicians remain unaddressed challenges. To mitigate these challenges, recently we have demonstrated that spatial light interference microscopy (SLIM) can provide a path to intrinsic, objective markers, that are independent of preparation and human bias. Additionally, the sensitivity of SLIM to collagen fibers yields information relevant to patient outcome, which is not available in H&E. Here, we show that deep learning and SLIM can form a powerful combination for screening applications: training on 1,660 SLIM images of colon glands and validating on 144 glands, we obtained a benign vs. cancer classification accuracy of 99%. We envision that the SLIM whole slide scanner presented here paired with artificial intelligence algorithms may prove valuable as a pre-screening method, economizing the clinician's time and effort.

Interstitial magnetic thermotherapy dosimetry based on shear wave magnetomotive optical coherence elastography.

While magnetic thermoseeds are often utilized in interstitial magnetic thermotherapy (iMT) to enable localized tumor ablation, we propose to extend their use as the perturbative source in magnetomotive optical coherence elastography (MM-OCE) so that the heat-induced elasticity alterations can be 'theranostically' probed. MM-OCE measurements were found to agree with indentation results. Tissue stiffening was visualized on iMT-treated porcine liver and canine soft tissue sarcoma specimens, where histology confirmed thermal damages. Additionally, the elasticity was found to increase exponentially and linearly with the conventional thermal dosage metrics and the deposited thermal energy, respectively. Collectively, a physiologically-meaningful, MM-OCE-based iMT dosimetry is feasible.

Indentation Measurements to Validate Dynamic Elasticity Imaging Methods.

We describe macro-indentation techniques for estimating the elastic modulus of soft hydrogels. Our study describes (a) conditions under which quasi-static indentation can validate dynamic shear-wave imaging estimates and (b) how each of these techniques uniquely biases modulus estimates as they couple to the sample geometry. Harmonic shear waves between 25 and 400 Hz were imaged using ultrasonic Doppler and optical coherence tomography methods to estimate shear dispersion. From the shear-wave speed of sound, average elastic moduli of homogeneous samples were estimated. These results are compared directly with macroscopic indentation measurements measured two ways. One set of measurements applied Hertzian theory to the loading phase of the force-displacement curves using samples treated to minimize surface adhesion forces. A second set of measurements applied Johnson-Kendall-Roberts theory to the unloading phase of the force-displacement curve when surface adhesions were significant. All measurements were made using gelatin hydrogel samples of different sizes and concentrations. Agreement within 5% among elastic modulus estimates was achieved for a range of experimental conditions. Consequently, a simple quasi-static indentation measurement using a common gel can provide elastic modulus measurements that help validate dynamic shear-wave imaging estimates.

Mechanical contrast in spectroscopic magnetomotive optical coherence elastography.

The viscoelastic properties of tissues are altered during pathogenesis of numerous diseases and can therefore be a useful indicator of disease status and progression. Several elastography studies have utilized the mechanical frequency response and the resonance frequencies of tissue samples to characterize their mechanical properties. However, using the resonance frequency as a source of mechanical contrast in heterogeneous samples is complicated because it not only depends on the viscoelastic properties but also on the geometry and boundary conditions. In an elastography technique called magnetomotive optical coherence elastography (MM-OCE), the controlled movement of magnetic nanoparticles (MNPs) within the sample is used to obtain the mechanical properties. Previous demonstrations of MM-OCE have typically used point measurements in elastically homogeneous samples assuming a uniform concentration of MNPs. In this study, we evaluate the feasibility of generating MM-OCE elastograms in heterogeneous samples based on a spectroscopic approach which involves measuring the magnetomotive response at different excitation frequencies. Biological tissues and tissue-mimicking phantoms with two elastically distinct regions placed in side-by-side and bilayer configurations were used for the experiments, and finite element method simulations were used to validate the experimental results.

Magnetomotive optical coherence elastography using magnetic particles to induce mechanical waves.

Magnetic particles are versatile imaging agents that have found wide spread applicability in diagnostic, therapeutic, and rheology applications. In this study, we demonstrate that mechanical waves generated by a localized inclusion of magnetic nanoparticles can be used for assessment of the tissue viscoelastic properties using magnetomotive optical coherence elastography. We show these capabilities in tissue mimicking elastic and viscoelastic phantoms and in biological tissues by measuring the shear wave speed under magnetomotive excitation. Furthermore, we demonstrate the extraction of the complex shear modulus by measuring the shear wave speed at different frequencies and fitting to a Kelvin-Voigt model.

InvertNet: a new paradigm for digital access to invertebrate collections.

InvertNet, one of the three Thematic Collection Networks (TCNs) funded in the first round of the U.S. National Science Foundation's Advancing Digitization of Biological Collections (ADBC) program, is tasked with providing digital access to ~60 million specimens housed in 22 arthropod (primarily insect) collections at institutions distributed throughout the upper midwestern USA. The traditional workflow for insect collection digitization involves manually keying information from specimen labels into a database and attaching a unique identifier label to each specimen. This remains the dominant paradigm, despite some recent attempts to automate various steps in the process using more advanced technologies. InvertNet aims to develop improved semi-automated, high-throughput workflows for digitizing and providing access to invertebrate collections that balance the need for speed and cost-effectiveness with long-term preservation of specimens and accuracy of data capture. The proposed workflows build on recent methods for digitizing and providing access to high-quality images of multiple specimens (e.g., entire drawers of pinned insects) simultaneously. Limitations of previous approaches are discussed and possible solutions are proposed that incorporate advanced imaging and 3-D reconstruction technologies. InvertNet couples efficient digitization workflows with a highly robust network infrastructure capable of managing massive amounts of image data and related metadata and delivering high-quality images, including interactive 3-D reconstructions in real time via the Internet.

One-dimensional deterministic transport in neurons measured by dispersion-relation phase spectroscopy.

We studied the active transport of intracellular components along neuron processes using a new method developed in our laboratory: dispersion-relation phase spectroscopy. This method is able to quantitatively map spatially the heterogeneous dynamics of the concentration field of the cargos at submicron resolution without the need for tracking individual components. The results in terms of density correlation function reveal that the decay rate is linear in wavenumber, which is consistent with a narrow Lorentzian distribution of cargo velocity.