CP-Net: Instance-aware part segmentation network for biological cell parsing.

Instance segmentation of biological cells is important in medical image analysis for identifying and segmenting individual cells, and quantitative measurement of subcellular structures requires further cell-level subcellular part segmentation. Subcellular structure measurements are critical for cell phenotyping and quality analysis. For these purposes, instance-aware part segmentation network is first introduced to distinguish individual cells and segment subcellular structures for each detected cell. This approach is demonstrated on human sperm cells since the World Health Organization has established quantitative standards for sperm quality assessment. Specifically, a novel Cell Parsing Net (CP-Net) is proposed for accurate instance-level cell parsing. An attention-based feature fusion module is designed to alleviate contour misalignments for cells with an irregular shape by using instance masks as spatial cues instead of as strict constraints to differentiate various instances. A coarse-to-fine segmentation module is developed to effectively segment tiny subcellular structures within a cell through hierarchical segmentation from whole to part instead of directly segmenting each cell part. Moreover, a sperm parsing dataset is built including 320 annotated sperm images with five semantic subcellular part labels. Extensive experiments on the collected dataset demonstrate that the proposed CP-Net outperforms state-of-the-art instance-aware part segmentation networks.

Testing the generalizability and effectiveness of deep learning models among clinics: sperm detection as a pilot study.

BACKGROUND: Deep learning has been increasingly investigated for assisting clinical in vitro fertilization (IVF). The first technical step in many tasks is to visually detect and locate sperm, oocytes, and embryos in images. For clinical deployment of such deep learning models, different clinics use different image acquisition hardware and different sample preprocessing protocols, raising the concern over whether the reported accuracy of a deep learning model by one clinic could be reproduced in another clinic. Here we aim to investigate the effect of each imaging factor on the generalizability of object detection models, using sperm analysis as a pilot example. METHODS: Ablation studies were performed using state-of-the-art models for detecting human sperm to quantitatively assess how model precision (false-positive detection) and recall (missed detection) were affected by imaging magnification, imaging mode, and sample preprocessing protocols. The results led to the hypothesis that the richness of image acquisition conditions in a training dataset deterministically affects model generalizability. The hypothesis was tested by first enriching the training dataset with a wide range of imaging conditions, then validated through internal blind tests on new samples and external multi-center clinical validations. RESULTS: Ablation experiments revealed that removing subsets of data from the training dataset significantly reduced model precision. Removing raw sample images from the training dataset caused the largest drop in model precision, whereas removing 20x images caused the largest drop in model recall. by incorporating different imaging and sample preprocessing conditions into a rich training dataset, the model achieved an intraclass correlation coefficient (ICC) of 0.97 (95% CI: 0.94-0.99) for precision, and an ICC of 0.97 (95% CI: 0.93-0.99) for recall. Multi-center clinical validation showed no significant differences in model precision or recall across different clinics and applications. CONCLUSIONS: The results validated the hypothesis that the richness of data in the training dataset is a key factor impacting model generalizability. These findings highlight the importance of diversity in a training dataset for model evaluation and suggest that future deep learning models in andrology and reproductive medicine should incorporate comprehensive feature sets for enhanced generalizability across clinics.

Indentation induces instantaneous nuclear stiffening and unfolding of nuclear envelope wrinkles.

The nuclear envelope (NE) separates genomic DNA from the cytoplasm and regulates transport between the cytosol and the nucleus in eukaryotes. Nuclear stiffening enables the cell nucleus to protect itself from extensive deformation, loss of NE integrity, and genome instability. It is known that the reorganization of actin, lamin, and chromatin can contribute to nuclear stiffening. In this work, we show that structural alteration of NE also contributes to instantaneous nuclear stiffening under indentation. In situ mechanical characterization of cell nuclei in intact cells shows that nuclear stiffening and unfolding of NE wrinkles occur simultaneously at the indentation site. A positive correlation between the initial state of NE wrinkles, the unfolding of NE wrinkles, and the stiffening ratio (stiffness fold-change) is found. Additionally, NE wrinkles unfold throughout the nucleus outside the indentation site. Finite element simulation, which involves the purely passive process of structural unfolding, shows that unfolding of NE wrinkles alone can lead to an increase in nuclear stiffness and a reduction in stress and strain levels. Together, these results provide a perspective on how cell nucleus adapts to mechanical stimuli through structural alteration of the NE.

3D Morphology Measurement for Blastocyst Evaluation From "All Angles".

Measuring the 3D morphology of spherical cell aggregates is required in both biology and medicine. Traditional methods either use fluorescent labeling, which cause cell toxicity and are unsuitable for clinical treatment, or use 2D images to roughly estimate 3D morphology. To overcome these limitations, this paper presents a quantitative label-free 3D morphology measurement technique using multi-view images. This technique, for the first time, enables the morphological evaluation of a blastocyst (Day 5 embryo) from "all angles" for IVF treatment. In this technique, a spherical rotation scale invariant feature transform (SR-SIFT) is proposed to address feature distortions for the rotation matrix calculation of the multi-view images. U-Net with generalized Dice loss is used to segment individual trophectoderm (TE) cells and the inner cell mass (ICM) of the blastocyst. Based on the rotation matrices and the segmentation results, the 3D morphological parameters of the entire blastocyst were quantified. Experimental results showed that the error of rotation angle was less than 1 °, the Dice was 95.6% for TE segmentation and 92.3% for ICM segmentation, and the overall measurement error of clinically defined blastocyst parameters was less than 6.7%.

Ultrathin and Flexible Bioelectronic Arrays for Functional Measurement of iPSC-Cardiomyocytes under Cardiotropic Drug Administration and Controlled Microenvironments.

Emerging heart-on-a-chip technology is a promising tool to establish in vitro cardiac models for therapeutic testing and disease modeling. However, due to the technical complexity of integrating cell culture chambers, biosensors, and bioreactors into a single entity, a microphysiological system capable of reproducing controlled microenvironmental cues to regulate cell phenotypes, promote iPS-cardiomyocyte maturity, and simultaneously measure the dynamic changes of cardiomyocyte function in situ is not available. This paper reports an ultrathin and flexible bioelectronic array platform in 24-well format for higher-throughput contractility measurement under candidate drug administration or defined microenvironmental conditions. In the array, carbon black (CB)-PDMS flexible strain sensors were embedded for detecting iPSC-CM contractility signals. Carbon fiber electrodes and pneumatic air channels were integrated to provide electrical and mechanical stimulation to improve iPSC-CM maturation. Performed experiments validate that the bioelectronic array accurately reveals the effects of cardiotropic drugs and identifies mechanical/electrical stimulation strategies for promoting iPSC-CM maturation.

Development and evaluation of a live birth prediction model for evaluating human blastocysts from a retrospective study.

Background: In infertility treatment, blastocyst morphological grading is commonly used in clinical practice for blastocyst evaluation and selection, but has shown limited predictive power on live birth outcomes of blastocysts. To improve live birth prediction, a number of artificial intelligence (AI) models have been established. Most existing AI models for blastocyst evaluation only used images for live birth prediction, and the area under the receiver operating characteristic (ROC) curve (AUC) achieved by these models has plateaued at ~0.65. Methods: This study proposed a multimodal blastocyst evaluation method using both blastocyst images and patient couple's clinical features (e.g., maternal age, hormone profiles, endometrium thickness, and semen quality) to predict live birth outcomes of human blastocysts. To utilize the multimodal data, we developed a new AI model consisting of a convolutional neural network (CNN) to process blastocyst images and a multilayer perceptron to process patient couple's clinical features. The data set used in this study consists of 17,580 blastocysts with known live birth outcomes, blastocyst images, and patient couple's clinical features. Results: This study achieved an AUC of 0.77 for live birth prediction, which significantly outperforms related works in the literature. Sixteen out of 103 clinical features were identified to be predictors of live birth outcomes and helped improve live birth prediction. Among these features, maternal age, the day of blastocyst transfer, antral follicle count, retrieved oocyte number, and endometrium thickness measured before transfer are the top five features contributing to live birth prediction. Heatmaps showed that the CNN in the AI model mainly focuses on image regions of inner cell mass and trophectoderm (TE) for live birth prediction, and the contribution of TE-related features was greater in the CNN trained with the inclusion of patient couple's clinical features compared with the CNN trained with blastocyst images alone. Conclusions: The results suggest that the inclusion of patient couple's clinical features along with blastocyst images increases live birth prediction accuracy. Funding: Natural Sciences and Engineering Research Council of Canada and the Canada Research Chairs Program.

More than 50 million couples worldwide experience infertility. The most common treatment is in vitro fertilization (IVF). Fertility specialists collect eggs and sperm from the prospective parents. They combine the egg and sperm in a laboratory and allow the fertilized eggs to develop for five days into a multi-celled blastocyst. Then, the specialists select the healthiest blastocysts and return them to the patient's uterus. Since 1978, more than 8 million children have been conceived through IVF. Yet, only about 30% of IVF attempts result in a successful birth. As a result, fertility patients often undergo multiple rounds of IVF, which can be expensive and emotionally draining. Several factors determine IVF success, one of which is the health of the blastocysts selected for transfer to the uterus. Specialists select the blastocysts using several criteria. But these human assessments are subjective and inconsistent in predicting which ones are most likely to result in a successful birth. Recent studies suggest artificial intelligence technology may help select blastocysts. Liu et al. show that using artificial intelligence to assess blastocysts and fertility patient characteristics leads to more accurate predictions about which blastocysts are likely to result in a successful birth. In the experiments, the researchers trained an artificial intelligence computer program using pictures of 17,580 blastocysts with known birth outcomes and the parents' clinical characteristics. The model identified 16 parental factors associated with birth outcomes. The top 5 most predictive parental factors were maternal age, the day of blastocyst transfer to the uterus, how many eggs were present in the ovaries, the number of eggs retrieved and the thickness of the uterus lining. The program achieved the highest prediction of healthy births so far, compared to success rates listed in other studies. Artificial intelligence-aided blastocyte selection using patient and blastocyst characteristics may improve IVF success rates and reduce the number of treatment cycles patient couples undergo. Before specialists can use artificial intelligence in their clinics, they must conduct confirmatory clinical studies that enroll patient couples to compare conventional methods and artificial intelligence.

Staining-free, Automated Sperm Analysis for In Vitro Fertilization Lab Use.

PURPOSE: Computer-aided sperm analysis is typically used in andrology labs, not in in vitro fertilization labs, which requires staining for sperm morphology measurement. In in vitro fertilization labs, sperm analysis still relies on manual observation and suffers from subjectivity and inconsistency. We developed a system for automated measurement of sperm concentration, motility, and morphology without the need for sperm staining. The reproducibility and reliability of the system were evaluated. MATERIALS AND METHODS: Thirty-five fresh semen and 25 washed samples were obtained from male partners attending for fertility investigations. Sperm concentration, motility, and morphology were automatically measured simultaneously, leveraging robust sperm tracking for concentration and motility measurement and low contrast image segmentation for morphology measurement of live sperm. Reproducibility of sperm measurements was evaluated by intraclass correlation coefficients. Reliability of sperm measurement was evaluated by Passing and Bablok regression analysis and Bland-Altman analysis. RESULTS: Automated measurement of concentration, motility, and morphology had intraclass correlation coefficients higher than 0.97. The regression and Bland-Altman analysis indicated that automated measurement and off-line manual benchmarking with zoomed-in images were interchangeable. Further analysis on semen and washed samples and the measurement on progressive and nonprogressive motility also showed high reproducibility and reliability. CONCLUSIONS: Automated sperm analysis revealed high reproducibility and reliability. The system is designed for routine use in in vitro fertilization labs to perform quantitative sperm analysis on live samples.

Automated Denudation of Oocytes.

Denudation is a technique for removal of the cumulus cell mass from oocytes in clinical intracytoplasmic sperm injection (ICSI). Manual oocyte denudation requires long training hours and stringent skills, but still suffers from low yield rate and denudation efficiency due to human fatigue and skill variations across operators. To address these limitations, this paper reports a robotic system for automated oocyte denudation. In this system, several key techniques are proposed, including a vision-based contact detection method for measuring the relative z position between the micropipette tip and the dish substrate, recognition of oocytes and the surrounding cumulus cells, automated calibration algorithm for eliminating the misalignment angle, and automated control of the flow rate based on the model of oocyte dynamics during micropipette aspiration and deposition. Experiments on mouse oocytes demonstrated that the robotic denudation system achieved a high yield rate of 97.0 ± 2.8% and denudation efficiency of 95.0 ± 0.8%. Additionally, oocytes denuded by the robotic system showed comparable fertilization rate and developmental competence compared with manual denudation. Our robotic denudation system represents one step towards the automation and standardization of ICSI procedures.

Microrobotic Swarms for Intracellular Measurement with Enhanced Signal-to-Noise Ratio.

In cell biology, fluorescent dyes are routinely used for biochemical measurements. The traditional global dye treatment method suffers from low signal-to-noise ratios (SNR), especially when used for detecting a low concentration of ions, and increasing the concentration of fluorescent dyes causes more severe cytotoxicity. Here, we report a robotic technique that controls how a low amount of fluorescent-dye-coated magnetic nanoparticles accurately forms a swarm and increases the fluorescent dye concentration in a local region inside a cell for intracellular measurement. Different from existing magnetic micromanipulation systems that generate large swarms (several microns and above) or that cannot move the generated swarm to an arbitrary position, our system is capable of generating a small swarm (e.g., 1 µm) and accurately positioning the swarm inside a single cell (position control accuracy: 0.76 µm). In experiments, the generated swarm inside the cell showed an SNR 10 times higher than the traditional global dye treatment method. The high-SNR robotic swarm enabled intracellular measurements that had not been possible to achieve with traditional global dye treatment. The robotic swarm technique revealed an apparent pH gradient in a migrating cell and was used to measure the intracellular apparent pH in a single oocyte of living C. elegans. With the position control capability, the swarm was also applied to measure calcium changes at the perinuclear region of a cell before and after mechanical stimulation. The results showed a significant calcium increase after mechanical stimulation, and the calcium increase was regulated by the mechanically sensitive ion channel, PIEZO1.

Microrobotic swarms for selective embolization.

Inspired by the collective intelligence in natural swarms, microrobotic agents have been controlled to form artificial swarms for targeted drug delivery, enhanced imaging, and hyperthermia. Different from these well-investigated tasks, this work aims to develop microrobotic swarms for embolization, which is a clinical technique used to block blood vessels for treating tumors, fistulas, and arteriovenous malformations. Magnetic particle swarms were formed for selective embolization to address the low selectivity of the present embolization technique that is prone to cause complications such as stroke and blindness. We established an analytical model that describes the relationships between fluid viscosity, flow rate, branching angle, magnetic field strength, and swarm integrity, based on which an actuation strategy was developed to maintain the swarm integrity inside a targeted region under fluidic flow conditions. Experiments in microfluidic channels, ex vivo tissues, and in vivo porcine kidneys validated the efficacy of the proposed strategy for selective embolization.

A Carbon-Based Biosensing Platform for Simultaneously Measuring the Contraction and Electrophysiology of iPSC-Cardiomyocyte Monolayers.

Heart beating is triggered by the generation and propagation of action potentials through the myocardium, resulting in the synchronous contraction of cardiomyocytes. This process highlights the importance of electrical and mechanical coordination in organ function. Investigating the pathogenesis of heart diseases and potential therapeutic actions in vitro requires biosensing technologies which allow for long-term and simultaneous measurement of the contractility and electrophysiology of cardiomyocytes. However, the adoption of current biosensing approaches for functional measurement of in vitro cardiac models is hampered by low sensitivity, difficulties in achieving multifunctional detection, and costly manufacturing processes. Leveraging carbon-based nanomaterials, we developed a biosensing platform that is capable of performing on-chip and simultaneous measurement of contractility and electrophysiology of human induced pluripotent stem-cell-derived cardiomyocyte (iPSC-CM) monolayers. This platform integrates with a flexible thin-film cantilever embedded with a carbon black (CB)-PDMS strain sensor for high-sensitivity contraction measurement and four pure carbon nanotube (CNT) electrodes for the detection of extracellular field potentials with low electrode impedance. Cardiac functional properties including contractile stress, beating rate, beating rhythm, and extracellular field potential were evaluated to quantify iPSC-CM responses to common cardiotropic agents. In addition, an in vitro model of drug-induced cardiac arrhythmia was established to further validate the platform for disease modeling and drug testing.

Quantitative selection of single human sperm with high DNA integrity for intracytoplasmic sperm injection.

OBJECTIVE: To study at the single-cell level whether a sperm's motility and morphology parameters reflect its DNA integrity, and to establish a set of quantitative criteria for selecting single sperm with high DNA integrity. DESIGN: Prospective study. SETTING: In vitro fertilization center and university laboratories. PATIENT(S): Male patients undergoing infertility treatments. INTERVENTION(S): None. MAIN OUTCOME MEASURE(S): The motility and morphology parameters of each sperm were measured with the use of computer vision algorithms. The sperm was then aspirated and transferred for DNA fragmentation measurement by single-cell gel electrophoresis (comet assay). RESULT(S): We adapted the World Health Organization criteria, which were originally defined for semen analysis, and established a set of quantitative criteria for single-sperm selection in intracytoplasmic sperm injection. Sperm satisfying the criteria had significantly lower DNA fragmentation levels than the sample population. Both normal motility and normal morphology were required for a sperm to have low DNA fragmentation. The quantitative criteria were integrated into a software program for sperm selection. In blind tests in which our software and three embryologists selected sperm from the same patient samples, our software outperformed the embryologists and selected sperm with the highest DNA integrity. CONCLUSION(S): At the single-cell level, a sperm's motility and morphology parameters reflect its DNA integrity. The developed technique and criteria hold the potential to mitigate the risk factor of sperm DNA fragmentation in intracytoplasmic sperm injection.

Advances in sperm analysis: techniques, discoveries and applications.

Infertility affects one in six couples worldwide, and fertility continues to deteriorate globally, partly owing to a decline in semen quality. Sperm analysis has a central role in diagnosing and treating male factor infertility. Many emerging techniques, such as digital holography, super-resolution microscopy and next-generation sequencing, have been developed that enable improved analysis of sperm motility, morphology and genetics to help overcome limitations in accuracy and consistency, and improve sperm selection for infertility treatment. These techniques have also improved our understanding of fundamental sperm physiology by enabling discoveries in sperm behaviour and molecular structures. Further progress in sperm analysis and integrating these techniques into laboratories and clinics requires multidisciplinary collaboration, which will increase discovery and improve clinical outcomes.

Automated motility and morphology measurement of live spermatozoa.

BACKGROUND: Automated sperm analysis has wide applications in infertility diagnosis. Existing systems are not able to measure sperm count and both motility and morphology of individual live spermatozoa. Morphology measurement requires invasive staining, making the spermatozoa after morphology measurement not applicable to infertility treatment. OBJECTIVE: To evaluate the reproducibility and reliability of automated measurement of individual live sperm's motility and morphology. MATERIALS AND METHODS: Fresh semen samples were obtained from twenty male partners attending for fertility investigations. The system firstly measured motility for all the spermatozoa within the field of view under a low magnification (20×), then a spermatozoa of interest is selected by the user and automatically relocated by the system after switching to a high magnification (100×) for morphology measurement. Reproducibility of sperm measurements was evaluated by intraclass correlation coefficients on consecutive measurement. Reliability of motility and morphology measurement was evaluated by tracking error rate and limits of agreement, respectively, with manual measurement as benchmark. RESULTS: Measurement of all motility and morphology parameters had intraclass correlation coefficients higher than 0.94. Sperm motility measurement had a tracking error rate of 2.1%. Limit of agreement analysis indicated that automated measurement and manual measurement of sperm morphology were interchangeable. Automated measurement of all morphology parameters was not statistically different from manual measurement, as confirmed by the paired sample t test. DISCUSSION: Automated motility and morphology measurement of single sperm revealed high reproducibility and reliability. The system also achieved a high efficiency for motility and morphology measurement. In addition to the intracytoplasmic sperm injection (ICSI) samples with polyvinylpyrrolidone (PVP), the developed sperm measurement technique is also effective for analyzing semen and washed samples. The system provides a valuable tool for quantitative measurement and selection of single spermatozoa for ICSI. It can also be used for sperm motility and morphology analysis in andrology laboratories.

Time-frequency analysis of the tip motion in liquids using the wavelet transform in dynamic atomic force microscopy.

The tip motion of the dynamic atomic force microscope in liquids shows complex transient behaviors when using a low stiffness cantilever. The second flexural mode of the cantilever is momentarily excited. Multiple impacts between the tip and the sample might occur in one oscillation cycle. However, the commonly used Fourier transform method cannot provide time-related information about these transient features. To overcome this limitation, we apply the wavelet transform to perform the time-frequency analysis of the tip motion in liquids. The momentary excitation of the second mode and the phenomenon of multiple impacts are clearly shown in the time-frequency plane of the wavelet scalogram. The instantaneous frequencies and magnitudes of the second mode are extracted by the wavelet ridge analysis, which can provide quantitative estimations of the tip motion in the second mode. Moreover, the relations of the maximum instantaneous magnitude (MIM) to the amplitude setpoint and the Young's modulus of the sample surface are investigated. The results suggest that the MIM can be used to characterize the nanomechanical property of the sample surface at high amplitude setpoints.

Real-time scan speed control of the atomic force microscopy for reducing imaging time based on sample topography.

Here, a novel method, real-time scan speed control for raster scan amplitude modulation atomic force microscopes (AM-AFMs), is proposed. In general, the imaging rate is set to a fixed value before the experiment, which is determined by the feedback control calculations on each imaging point. Many efforts have been made to increase the AFM imaging rate, including using the cantilever with high eigenfrequency, employing new scan methods, and optimizing other mechanical components. The proposed real-time control method adjusts the scan speed linearly according to the error of every imaging point, which is mainly determined by the sample topography. Through setting residence time on each imaging point reasonably, the performance of AM-AFMs can be fully exploited while the scanner vibration is avoided when scan speed changes. Experiments and simulations are performed to demonstrate this control algorithm. This method would increase the imaging rate for samples with strongly fluctuant topography up to about 3 times without sacrificing any image quality, especially in large-scale and high-resolution imaging, in the meanwhile, it reduces the professional requirements for AM-AFM operators. Since the control strategy employs a linear algorithm to calculate the scanning speed based on the error signal, the proposed method avoids the frequent switching of the scanning speed between the high speed and the low speed. And it is easier to implement because there is no need to modify the original hardware of the AFM for its application.

Contributed Review: Application of voice coil motors in high-precision positioning stages with large travel ranges.

Recent interest in high-precision positioning stages with large travel ranges has sparked renewed attention to the development of voice coil motors (VCMs). Due to their large output force, VCMs can actuate more complicated flexure structures, eliminate rail friction, and improve positioning speed. The VCM structure is both compact and flexible; hence, it is convenient to design VCMs for a variety of stage structures. Furthermore, VCMs combined with other actuators are able to achieve large travel ranges with high precision. In this paper, we summarize the principles and control methods of a typical VCM, and we analyze its properties, including thrust force, acceleration, and response time. We then present recent research on high-precision VCM positioning stages with large travel ranges.