Multi-scale window transformer for cervical cytopathology image recognition.

Cervical cancer is a major global health issue, particularly in developing countries where access to healthcare is limited. Early detection of pre-cancerous lesions is crucial for successful treatment and reducing mortality rates. However, traditional screening and diagnostic processes require cytopathology doctors to manually interpret a huge number of cells, which is time-consuming, costly, and prone to human experiences. In this paper, we propose a Multi-scale Window Transformer (MWT) for cervical cytopathology image recognition. We design multi-scale window multi-head self-attention (MW-MSA) to simultaneously integrate cell features of different scales. Small window self-attention is used to extract local cell detail features, and large window self-attention aims to integrate features from smaller-scale window attention to achieve window-to-window information interaction. Our design enables long-range feature integration but avoids whole image self-attention (SA) in ViT or twice local window SA in Swin Transformer. We find convolutional feed-forward networks (CFFN) are more efficient than original MLP-based FFN for representing cytopathology images. Our overall model adopts a pyramid architecture. We establish two multi-center cervical cell classification datasets of two-category 192,123 images and four-category 174,138 images. Extensive experiments demonstrate that our MWT outperforms state-of-the-art general classification networks and specialized classifiers for cytopathology images in the internal and external test sets. The results on large-scale datasets prove the effectiveness and generalization of our proposed model. Our work provides a reliable cytopathology image recognition method and helps establish computer-aided screening for cervical cancer. Our code is available at https://github.com/nmyz669/MWT, and our web service tool can be accessed at https://huggingface.co/spaces/nmyz/MWTdemo.

Non-Gaussian anomalous diffusion of optical vortices.

Anomalous diffusion of different particlelike entities, the deviation from typical Brownian motion, is ubiquitous in complex physical and biological systems. While optical vortices move randomly in evolving speckle fields, optical vortices have only been observed to exhibit pure Brownian motion in random speckle fields. Here we present direct experimental evidence of the anomalous diffusion of optical vortices in temporally varying speckle patterns from multiple-scattering viscoelastic media. Moreover, we observe two characteristic features, i.e., the self-similarity and the antipersistent correlation of the optical vortex motion, indicating that the mechanism of the observed subdiffusion of optical vortices can only be attributed to fractional Brownian motion (FBM). We further demonstrate that the vortex displacements exhibit a non-Gaussian heavy-tailed distribution. Additionally, we modulate the extent of subdiffusion, such as diffusive scaling exponents, and the non-Gaussianity of optical vortices by altering the viscoelasticity of samples. The discovery of the complex FBM but non-Gaussian subdiffusion of optical vortices may not only offer insight into certain fundamental physics, including the anomalous diffusion of vortices in fluids and the decoupling between Brownianity and Gaussianity, but also suggest a strong potential for utilizing optical vortices as tracers in microrheology instead of the introduced exogenous probe particles in particle tracking microrheology.

Ten-kilohertz two-photon microscopy imaging of single-cell dendritic activity and hemodynamics in vivo.

Significance: The studying of rapid neuronal signaling across large spatial scales in intact, living brains requires both high temporal resolution and versatility of the measurement device. Aim: We introduce a high-speed two-photon microscope based on a custom-built acousto-optic deflector (AOD). This microscope has a maximum line scan frequency of 400 kHz and a maximum frame rate of 10,000 frames per second (fps) at 250 × 40 pixels . For stepwise magnification from population view to subcellular view with high spatial and temporal resolution, we combined the AOD with resonance-galvo (RS) scanning. Approach: With this combinatorial device that supports both large-view navigation and small-view high-speed imaging, we measured dendritic calcium propagation velocity and the velocity of single red blood cells (RBCs). Results: We measured dendritic calcium propagation velocity ( 80 / 62.5 - 116.7 µ m / ms ) in OGB-1-labeled single cortical neurons in mice in vivo. To benchmark the spatial precision and detection sensitivity of measurement in vivo, we also visualized the trajectories of single RBCs and found that their movement speed follows Poiseuille's law of laminar flow. Conclusions: This proof-of-concept methodological development shows that the combination of AOD and RS scanning two-photon microscopy provides both versatility and precision for quantitative analysis of single neuronal activities and hemodynamics in vivo.

Multiparameter mobile blood analysis for complete blood count using contrast-enhanced defocusing imaging and machine vision.

Blood analysis through complete blood count is the most basic medical test for disease diagnosis. Conventional blood analysis requires bulky and expensive laboratory facilities and skilled technicians, limiting the universal medical use of blood analysis outside well-equipped laboratory environments. Here, we propose a multiparameter mobile blood analyzer combined with label-free contrast-enhanced defocusing imaging (CEDI) and machine vision for instant and on-site diagnostic applications. We designed a low-cost and high-resolution miniature microscope (size: 105 mm × 77 mm × 64 mm, weight: 314 g) that comprises a pair of miniature aspheric lenses and a 415 nm LED for blood image acquisition. The analyzer, adopting CEDI, can obtain both the refractive index distributions of the white blood cell (WBC) and hemoglobin spectrophotometric information, enabling the analyzer to supply rich blood parameters, including the five-part WBC differential count, red blood cell (RBC) count, and mean corpuscular hemoglobin (MCH) quantification with machine vision algorithms and the Lambert-Beer law. We have shown that our assay can analyze a blood sample within 10 minutes without complex staining, and measurements (30 samples) from the analyzer have a strong linear correlation with clinical reference values (significance level of 0.0001). This study provides a miniature, light weight, low-cost, and easy-to-use blood analysis technique that overcomes the challenge of simultaneously realizing FWD count, RBC count, and MCH analysis using a mobile device and has great potential for integrated surveillance of various epidemic diseases, including coronavirus infection, invermination, and anemia, especially in low- and middle-income countries.

Anatomical Evidence for Parasympathetic Innervation of the Renal Vasculature and Pelvis.

BACKGROUND: The kidneys critically contribute to body homeostasis under the control of the autonomic nerves, which enter the kidney along the renal vasculature. Although the renal sympathetic and sensory nerves have long been confirmed, no significant anatomic evidence exists for renal parasympathetic innervation. METHODS: We identified cholinergic nerve varicosities associated with the renal vasculature and pelvis using various anatomic research methods, including a genetically modified mouse model and immunostaining. Single-cell RNA sequencing (scRNA-Seq) was used to analyze the expression of AChRs in the renal artery and its segmental branches. To assess the origins of parasympathetic projecting nerves of the kidney, we performed retrograde tracing using recombinant adeno-associated virus (AAV) and pseudorabies virus (PRV), followed by imaging of whole brains, spinal cords, and ganglia. RESULTS: We found that cholinergic axons supply the main renal artery, segmental renal artery, and renal pelvis. On the renal artery, the newly discovered cholinergic nerve fibers are separated not only from the sympathetic nerves but also from the sensory nerves. We also found cholinergic ganglion cells within the renal nerve plexus. Moreover, the scRNA-Seq analysis suggested that acetylcholine receptors (AChRs) are expressed in the renal artery and its segmental branches. In addition, retrograde tracing suggested vagus afferents conduct the renal sensory pathway to the nucleus of the solitary tract (NTS), and vagus efferents project to the kidney. CONCLUSIONS: Cholinergic nerves supply renal vasculature and renal pelvis, and a vagal brain-kidney axis is involved in renal innervation.

Label-free hematology analysis method based on defocusing phase-contrast imaging under illumination of 415†nm light.

Label-free imaging technology is a trending way to simplify and improve conventional hematology analysis by bypassing lengthy and laborious staining procedures. However, the existing methods do not well balance system complexity, data acquisition efficiency, and data analysis accuracy, which severely impedes their clinical translation. Here, we propose defocusing phase-contrast imaging under the illumination of 415 nm light to realize label-free hematology analysis. We have verified that the subcellular morphology of blood components can be visualized without complex staining due to the factor that defocusing can convert the second-order derivative distribution of samples' optical phase into intensity and the illumination of 415 nm light can significantly enhance the contrast. It is demonstrated that the defocusing phase-contrast images for the five leucocyte subtypes can be automatically discriminated by a trained deep-learning program with high accuracy (the mean F1 score: 0.986 and mean average precision: 0.980). Since this technique is based on a regular microscope, it simultaneously realizes low system complexity and high data acquisition efficiency with remarkable quantitative analysis ability. It supplies a label-free, reliable, easy-to-use, fast approach to simplifying and reforming the conventional way of hematology analysis.

Cervical cytopathology image refocusing via multi-scale attention features and domain normalization.

Cervical cytopathology image refocusing is important for addressing the problem of defocus blur in whole slide images. However, most of current deblurring methods are developed for global motion blur instead of local defocus blur and need a lot of supervised re-training for unseen domains. In this paper, we propose a refocusing method for cervical cytopathology images via multi-scale attention features and domain normalization. Our method consists of a domain normalization net (DNN) and a refocusing net (RFN). In DNN, we adopt registration-free cycle scheme for normalizing the unseen unsupervised domains into the seen supervised domain and introduce gray mask loss and hue-encoding mask loss to ensure the consistency of cell structure and basic hue. In RFN, combining the locality and sparseness characteristics of defocus blur, we design a multi-scale refocusing network to enhance the reconstruction of cell nucleus and cytoplasm, and introduce defocus intensity estimation mask to strengthen the reconstruction of local blur. We integrate hybrid learning strategy on the supervised and unsupervised domains to make RFN achieving well refocusing on the unsupervised domain. We build a cervical cytopathology image refocusing dataset and conduct extensive experiments to demonstrate the superiority of our method compared with current deblurring state-of-the-art models. Furthermore, we prove that the refocused images help improve the performance of subsequent high-level analysis tasks. We release the refocusing dataset and source codes to promote the development of this field.

Super-resolution Segmentation Network for Reconstruction of Packed Neurites.

Neuron reconstruction can provide the quantitative data required for measuring the neuronal morphology and is crucial in brain research. However, the difficulty in reconstructing dense neurites, wherein massive labor is required for accurate reconstruction in most cases, has not been well resolved. In this work, we provide a new pathway for solving this challenge by proposing the super-resolution segmentation network (SRSNet), which builds the mapping of the neurites in the original neuronal images and their segmentation in a higher-resolution (HR) space. During the segmentation process, the distances between the boundaries of the packed neurites are enlarged, and only the central parts of the neurites are segmented. Owing to this strategy, the super-resolution segmented images are produced for subsequent reconstruction. We carried out experiments on neuronal images with a voxel size of 0.2 µm × 0.2 µm × 1 µm produced by fMOST. SRSNet achieves an average F1 score of 0.88 for automatic packed neurites reconstruction, which takes both the precision and recall values into account, while the average F1 scores of other state-of-the-art automatic tracing methods are less than 0.70.

Brain-wide projection reconstruction of single functionally defined neurons.

Reconstructing axonal projections of single neurons at the whole-brain level is currently a converging goal of the neuroscience community that is fundamental for understanding the logic of information flow in the brain. Thousands of single neurons from different brain regions have recently been morphologically reconstructed, but the corresponding physiological functional features of these reconstructed neurons are unclear. By combining two-photon Ca2+ imaging with targeted single-cell plasmid electroporation, we reconstruct the brain-wide morphologies of single neurons that are defined by a sound-evoked response map in the auditory cortices (AUDs) of awake mice. Long-range interhemispheric projections can be reliably labelled via co-injection with an adeno-associated virus, which enables enhanced expression of indicator protein in the targeted neurons. Here we show that this method avoids the randomness and ambiguity of conventional methods of neuronal morphological reconstruction, offering an avenue for developing a precise one-to-one map of neuronal projection patterns and physiological functional features.

Large depth-of-field fluorescence microscopy based on deep learning supported by Fresnel incoherent correlation holography.

Fluorescence microscopy plays an irreplaceable role in biomedicine. However, limited depth of field (DoF) of fluorescence microscopy is always an obstacle of image quality, especially when the sample is with an uneven surface or distributed in different depths. In this manuscript, we combine deep learning with Fresnel incoherent correlation holography to describe a method to obtain significant large DoF fluorescence microscopy. Firstly, the hologram is restored by the Auto-ASP method from out-of-focus to in-focus in double-spherical wave Fresnel incoherent correlation holography. Then, we use a generative adversarial network to eliminate the artifacts introduced by Auto-ASP and output the high-quality image as a result. We use fluorescent beads, USAF target and mouse brain as samples to demonstrate the large DoF of more than 400µm, which is 13 times better than that of traditional wide-field microscopy. Moreover, our method is with a simple structure, which can be easily combined with many existing fluorescence microscopic imaging technology.

Minimizing Probability Graph Connectivity Cost for Discontinuous Filamentary Structures Tracing in Neuron Image.

Neuron tracing from optical image is critical in understanding brain function in diseases. A key problem is to trace discontinuous filamentary structures from noisy background, which is commonly encountered in neuronal and some medical images. Broken traces lead to cumulative topological errors, and current methods were hard to assemble various fragmentary traces for correct connection. In this paper, we propose a graph connectivity theoretical method for precise filamentary structure tracing in neuron image. First, we build the initial subgraphs of signals via a region-to-region based tracing method on CNN predicted probability. CNN technique removes noise interference, whereas its prediction for some elongated fragments is still incomplete. Second, we reformulate the global connection problem of individual or fragmented subgraphs under heuristic graph restrictions as a dynamic linear programming function via minimizing graph connectivity cost, where the connected cost of breakpoints are calculated using their probability strength via minimum cost path. Experimental results on challenging neuronal images proved that the proposed method outperformed existing methods and achieved similar results of manual tracing, even in some complex discontinuous issues. Performances on vessel images indicate the potential of the method for some other tubular objects tracing.

STSRNet: Self-Texture Transfer Super-Resolution and Refocusing Network.

Biomedical microscopy images with high-resolution (HR) and axial information can help analysis and diagnosis. However, obtaining such images usually takes more time and economic costs, which makes it impractical in most scenarios. In this paper, we first propose a novel Self-texture Transfer Super-resolution and Refocusing Network (STSRNet) to reconstruct HR multi-focal plane (MFP) images from a single 2D low-resolution (LR) wide field image without relying on scanning or any special devices. The proposed STSRNet consists of three parts: the backbone module for extracting features, the self-texture transfer module for transferring and fusing features, and the flexible reconstruction module for SR and refocusing. Specifically, the self-texture transfer module is designed for images with self-similarity such as cytological images and it searches for similar textures within the image and transfers to help MFP reconstruction. As for reconstruction module, it is composed of multiple pluggable components, each of which is responsible for a specific focal plane, so as to performs SR and refocusing all focal planes at one time to reduce computation. We conduct extensive experiments on cytological images and the experiments show that MFP images reconstructed by STSRNet have richer details in the axial and horizontal directions than input images. At the same time, the reconstructed MFP images also perform better than single 2D wide field images on high-level tasks. The proposed method provides relatively high-quality MFP images when real MFP images cannot be obtained, which greatly expands the application potential of LR wide-field images. To further promote the development of this field, we released our cytology dataset named RSDC for more researchers to use.

Cross-Streams Through the Ventral Posteromedial Thalamic Nucleus to Convey Vibrissal Information.

Whisker detection is crucial to adapt to the environment for some animals, but how the nervous system processes and integrates whisker information is still an open question. It is well-known that two main parallel pathways through Ventral posteromedial thalamic nucleus (VPM) ascend to the barrel cortex, and classical theory suggests that the cross-talk from trigeminal nucleus interpolaris (Sp5i) to principal nucleus (Pr5) between the main parallel pathways contributes to the multi-whisker integration in barrel columns. Moreover, some studies suggest there are other cross-streams between the parallel pathways. To confirm their existence, in this study we used a dual-viral labeling strategy and high-resolution, large-volume light imaging to get the complete morphology of individual VPM neurons and trace their projections. We found some new thalamocortical projections from the ventral lateral part of VPM (VPMvl) to barrel columns. In addition, the retrograde-viral labeling and imaging results showed there were the large trigeminothalamic projections from Sp5i to the dorsomedial section of VPM (VPMdm). Our results reveal new cross-streams between the parallel pathways through VPM, which may involve the execution of multi-whisker integration in barrel columns.

Foreground Estimation in Neuronal Images With a Sparse-Smooth Model for Robust Quantification.

3D volume imaging has been regarded as a basic tool to explore the organization and function of the neuronal system. Foreground estimation from neuronal image is essential in the quantification and analysis of neuronal image such as soma counting, neurite tracing and neuron reconstruction. However, the complexity of neuronal structure itself and differences in the imaging procedure, including different optical systems and biological labeling methods, result in various and complex neuronal images, which greatly challenge foreground estimation from neuronal image. In this study, we propose a robust sparse-smooth model (RSSM) to separate the foreground and the background of neuronal image. The model combines the different smoothness levels of the foreground and the background, and the sparsity of the foreground. These prior constraints together contribute to the robustness of foreground estimation from a variety of neuronal images. We demonstrate the proposed RSSM method could promote some best available tools to trace neurites or locate somas from neuronal images with their default parameters, and the quantified results are similar or superior to the results that generated from the original images. The proposed method is proved to be robust in the foreground estimation from different neuronal images, and helps to improve the usability of current quantitative tools on various neuronal images with several applications.

StainNet: A Fast and Robust Stain Normalization Network.

Stain normalization often refers to transferring the color distribution to the target image and has been widely used in biomedical image analysis. The conventional stain normalization usually achieves through a pixel-by-pixel color mapping model, which depends on one reference image, and it is hard to achieve accurately the style transformation between image datasets. In principle, this difficulty can be well-solved by deep learning-based methods, whereas, its complicated structure results in low computational efficiency and artifacts in the style transformation, which has restricted the practical application. Here, we use distillation learning to reduce the complexity of deep learning methods and a fast and robust network called StainNet to learn the color mapping between the source image and the target image. StainNet can learn the color mapping relationship from a whole dataset and adjust the color value in a pixel-to-pixel manner. The pixel-to-pixel manner restricts the network size and avoids artifacts in the style transformation. The results on the cytopathology and histopathology datasets show that StainNet can achieve comparable performance to the deep learning-based methods. Computation results demonstrate StainNet is more than 40 times faster than StainGAN and can normalize a 100,000 × 100,000 whole slide image in 40 s.

Expansion tomography for large volume tissue imaging with nanoscale resolution.

Expansion microscopy enables conventional diffraction limit microscopy to achieve super-resolution imaging. However, the enlarged tissue lacks an objective lens with sufficient working distance that can image tissues with whole-brain-scale coverage. Here, we present expansion tomography (ExT) to solve this problem. We have established a modified super-absorbent hydrogel (ExT gel) that possesses high mechanical strength and enables serial sectioning. ExT gel enables tissue and cell imaging and is compatible with various fluorescent labeling strategies. Combining with the high-throughput light-sheet tomography (HLTP) system, we have shown the capability of large volume imaging with nanoscale resolution of mouse brain intact neuronal circuits. The ExT method would allow image samples to support super-resolution imaging of intact tissues with virtually unlimited axial extensions.

Robust whole slide image analysis for cervical cancer screening using deep learning.

Computer-assisted diagnosis is key for scaling up cervical cancer screening. However, current recognition algorithms perform poorly on whole slide image (WSI) analysis, fail to generalize for diverse staining and imaging, and show sub-optimal clinical-level verification. Here, we develop a progressive lesion cell recognition method combining low- and high-resolution WSIs to recommend lesion cells and a recurrent neural network-based WSI classification model to evaluate the lesion degree of WSIs. We train and validate our WSI analysis system on 3,545 patient-wise WSIs with 79,911 annotations from multiple hospitals and several imaging instruments. On multi-center independent test sets of 1,170 patient-wise WSIs, we achieve 93.5% Specificity and 95.1% Sensitivity for classifying slides, comparing favourably to the average performance of three independent cytopathologists, and obtain 88.5% true positive rate for highlighting the top 10 lesion cells on 447 positive slides. After deployment, our system recognizes a one giga-pixel WSI in about 1.5 min.

Automated Neuron Tracing Using Content-Aware Adaptive Voxel Scooping on CNN Predicted Probability Map.

Neuron tracing, as the essential step for neural circuit building and brain information flow analyzing, plays an important role in the understanding of brain organization and function. Though lots of methods have been proposed, automatic and accurate neuron tracing from optical images remains challenging. Current methods often had trouble in tracing the complex tree-like distorted structures and broken parts of neurite from a noisy background. To address these issues, we propose a method for accurate neuron tracing using content-aware adaptive voxel scooping on a convolutional neural network (CNN) predicted probability map. First, a 3D residual CNN was applied as preprocessing to predict the object probability and suppress high noise. Then, instead of tracing on the binary image produced by maximum classification, an adaptive voxel scooping method was presented for successive neurite tracing on the probability map, based on the internal content properties (distance, connectivity, and probability continuity along direction) of the neurite. Last, the neuron tree graph was built using the length first criterion. The proposed method was evaluated on the public BigNeuron datasets and fluorescence micro-optical sectioning tomography (fMOST) datasets and outperformed current state-of-art methods on images with neurites that had broken parts and complex structures. The high accuracy tracing proved the potential of the proposed method for neuron tracing on large-scale.