MSGM: An Advanced Deep Multi-Size Guiding Matching Network for Whole Slide Histopathology Images Addressing Staining Variation and Low Visibility Challenges.

Matching whole slide histopathology images to provide comprehensive information on homologous tissues is beneficial for cancer diagnosis. However, the challenge arises with the Giga-pixel whole slide images (WSIs) when aiming for high-accuracy matching. Learning-based methods are difficult to generalize well with large-size WSIs, necessitating the integration of traditional matching methods to enhance accuracy as the size increases. In this paper, we propose a multi-size guiding matching method applicable high-accuracy requirements. Specifically, we design learning multiscale texture to train deep descriptors, called TDescNet, that trains 64 ×64×256 and 256 ×256×128 size convolution layer as C64 and C256 descriptors to overcome staining variation and low visibility challenges. Furthermore, we develop the 3D-ring descriptor using sparse keypoints to support the description of large-size WSIs. Finally, we employ C64, C256, and 3D-ring descriptors to progressively guide refined local matching, utilizing geometric consistency to identify correct matching results. Experiments show that when matching WSIs of size 4096×4096 pixels, our average matching error is 123.48 [Formula: see text] and the success rate is 93.02 % in 43 cases. Notably, our method achieves an average improvement of 65.52 [Formula: see text] in matching accuracy compared to recent state-of-the-art methods, with enhancements ranging from 36.27 [Formula: see text] to 131.66 [Formula: see text]. Therefore, we achieve high-fidelity whole-slice image matching, and overcome staining variation and low visibility challenges, enabling assistance in comprehensive cancer diagnosis through matched WSIs.

Global Semantic-Sense Aggregation Network for Salient Object Detection in Remote Sensing Images.

Salient object detection (SOD) aims to accurately identify significant geographical objects in remote sensing images (RSI), providing reliable support and guidance for extensive geographical information analyses and decisions. However, SOD in RSI faces numerous challenges, including shadow interference, inter-class feature confusion, as well as unclear target edge contours. Therefore, we designed an effective Global Semantic-aware Aggregation Network (GSANet) to aggregate salient information in RSI. GSANet computes the information entropy of different regions, prioritizing areas with high information entropy as potential target regions, thereby achieving precise localization and semantic understanding of salient objects in remote sensing imagery. Specifically, we proposed a Semantic Detail Embedding Module (SDEM), which explores the potential connections among multi-level features, adaptively fusing shallow texture details with deep semantic features, efficiently aggregating the information entropy of salient regions, enhancing information content of salient targets. Additionally, we proposed a Semantic Perception Fusion Module (SPFM) to analyze map relationships between contextual information and local details, enhancing the perceptual capability for salient objects while suppressing irrelevant information entropy, thereby addressing the semantic dilution issue of salient objects during the up-sampling process. The experimental results on two publicly available datasets, ORSSD and EORSSD, demonstrated the outstanding performance of our method. The method achieved 93.91% Sα, 98.36% Eξ, and 89.37% Fß on the EORSSD dataset.

Monitoring response to neoadjuvant therapy for breast cancer in all treatment phases using an ultrasound deep learning model.

Purpose: The aim of this study was to investigate the value of a deep learning model (DLM) based on breast tumor ultrasound image segmentation in predicting pathological response to neoadjuvant chemotherapy (NAC) in breast cancer. Methods: The dataset contains a total of 1393 ultrasound images of 913 patients from Renmin Hospital of Wuhan University, of which 956 ultrasound images of 856 patients were used as the training set, and 437 ultrasound images of 57 patients underwent NAC were used as the test set. A U-Net-based end-to-end DLM was developed for automatically tumor segmentation and area calculation. The predictive abilities of the DLM, manual segmentation model (MSM), and two traditional ultrasound measurement methods (longest axis model [LAM] and dual-axis model [DAM]) for pathological complete response (pCR) were compared using changes in tumor size ratios to develop receiver operating characteristic curves. Results: The average intersection over union value of the DLM was 0.856. The early-stage ultrasound-predicted area under curve (AUC) values of pCR were not significantly different from those of the intermediate and late stages (p< 0.05). The AUCs for MSM, DLM, LAM and DAM were 0.840, 0.756, 0.778 and 0.796, respectively. There was no significant difference in AUC values of the predictive ability of the four models. Conclusion: Ultrasonography was predictive of pCR in the early stages of NAC. DLM have a similar predictive value to conventional ultrasound for pCR, with an add benefit in effectively improving workflow.

An orientation-free ring feature descriptor with stain-variability normalization for pathology image matching.

Comprehensively analyzing the corresponding regions in the images of serial slices stained using different methods is a common but important operation in pathological diagnosis. To help increase the efficiency of the analysis, various image registration methods are proposed to match the corresponding regions in different images, but their performance is highly influenced by the rotations, deformations, and variations of staining between the serial pathology images. In this work, we propose an orientation-free ring feature descriptor with stain-variability normalization for pathology image matching. Specifically, we normalize image staining to similar levels to minimize the impact of staining differences on pathology image matching. To overcome the rotation and deformation issues, we propose a rotation-invariance orientation-free ring feature descriptor that generates novel adaptive bins from ring features to build feature vectors. We measure the Euclidean distance of the feature vectors to evaluate keypoint similarity to achieve pathology image matching. A total of 46 pairs of clinical pathology images in hematoxylin-eosin and immunohistochemistry straining to verify the performance of our method. Experimental results indicate that our method meets the pathology image matching accuracy requirements (error ¡ 300µm), especially competent for large-angle rotation cases common in clinical practice.

Optical time-stretch imaging flow cytometry in the compressed domain.

Imaging flow cytometry based on optical time-stretch (OTS) imaging combined with a microfluidic chip attracts much attention in the large-scale single-cell analysis due to its high throughput, high precision, and label-free operation. Compressive sensing has been integrated into OTS imaging to relieve the pressure on the sampling and transmission of massive data. However, image decompression brings an extra overhead of computing power to the system, but does not generate additional information. In this work, we propose and demonstrate OTS imaging flow cytometry in the compressed domain. Specifically, we constructed a machine-learning network to analyze the cells without decompressing the images. The results show that our system enables high-quality imaging and high-accurate cell classification with an accuracy of over 99% at a compression ratio of 10%. This work provides a viable solution to the big data problem in OTS imaging flow cytometry, boosting its application in practice.

KRT13-expressing epithelial cell population predicts better response to chemotherapy and immunotherapy in bladder cancer: Comprehensive evidences based on BCa database.

Neoadjuvant chemotherapy (NAC) prior to surgery and immune checkpoint therapy (ICT) has revolutionized bladder cancer (BCa) treatment. Patients likely to benefit from these therapies need to be accurately stratified; however, this remains a major clinical challenge. In the present study, single-cell RNA sequencing was used to evaluate the predictive ability of an epithelial cell population highly expressing keratin 13 (KRT13) to assess therapeutic response in BCa. The presence of KRT13-enriched tumors indicated favorable outcomes after NAC and superior response to ICT in patients with BCa. Furthermore, KRT13 population characteristics appeared to be closely related to changes in the immune microenvironment in the vicinity of this cell population. We constructed a prognostic model using an artificial neural network based on the gene signatures in the KRT13 population; the model demonstrated strong robustness and superiority. Additionally, a user-friendly and open-access web application named BCa database was developed for researchers to study BCa by mining the connective map database.

Cell damage evaluation by intelligent imaging flow cytometry.

Essential thrombocythemia (ET) is an uncommon situation in which the body produces too many platelets. This can cause blood clots anywhere in the body and results in various symptoms and even strokes or heart attacks. Removing excessive platelets using acoustofluidic methods receives extensive attention due to their high efficiency and high yield. While the damage to the remaining cells, such as erythrocytes and leukocytes is yet evaluated. Existing cell damage evaluation methods usually require cell staining, which are time-consuming and labor-intensive. In this paper, we investigate cell damage by optical time-stretch (OTS) imaging flow cytometry with high throughput and in a label-free manner. Specifically, we first image the erythrocytes and leukocytes sorted by acoustofluidic sorting chip with different acoustic wave powers and flowing speed using OTS imaging flow cytometry at a flowing speed up to 1 m/s. Then, we employ machine learning algorithms to extract biophysical phenotypic features from the cellular images, as well as to cluster and identify images. The results show that both the errors of the biophysical phenotypic features and the proportion of abnormal cells are within 10% in the undamaged cell groups, while the errors are much greater than 10% in the damaged cell groups, indicating that acoustofluidic sorting causes little damage to the cells within the appropriate acoustic power, agreeing well with clinical assays. Our method provides a novel approach for high-throughput and label-free cell damage evaluation in scientific research and clinical settings.

Typing of acute leukemia by intelligent optical time-stretch imaging flow cytometry on a chip.

Acute leukemia (AL) is one of the top life-threatening diseases. Accurate typing of AL can significantly improve its prognosis. However, conventional methods for AL typing often require cell staining, which is time-consuming and labor-intensive. Furthermore, their performance is highly limited by the specificity and availability of fluorescent labels, which can hardly meet the requirements of AL typing in clinical settings. Here, we demonstrate AL typing by intelligent optical time-stretch (OTS) imaging flow cytometry on a microfluidic chip. Specifically, we employ OTS microscopy to capture the images of cells in clinical bone marrow samples with a spatial resolution of 780 nm at a high flowing speed of 1 m s-1 in a label-free manner. Then, to show the clinical utility of our method for which the features of clinical samples are diverse, we design and construct a deep convolutional neural network (CNN) to analyze the cellular images and determine the AL type of each sample. We measure 30 clinical samples composed of 7 acute lymphoblastic leukemia (ALL) samples, 17 acute myelogenous leukemia (AML) samples, and 6 samples from healthy donors, resulting in a total of 227 620 images acquired. Results show that our method can distinguish ALL and AML with an accuracy of 95.03%, which, to the best of our knowledge, is a record in label-free AL typing. In addition to AL typing, we believe that the high throughput, high accuracy, and label-free operation of our method make it a potential solution for cell analysis in scientific research and clinical settings.

BM-Net: CNN-Based MobileNet-V3 and Bilinear Structure for Breast Cancer Detection in Whole Slide Images.

Breast cancer is one of the most common types of cancer and is the leading cause of cancer-related death. Diagnosis of breast cancer is based on the evaluation of pathology slides. In the era of digital pathology, these slides can be converted into digital whole slide images (WSIs) for further analysis. However, due to their sheer size, digital WSIs diagnoses are time consuming and challenging. In this study, we present a lightweight architecture that consists of a bilinear structure and MobileNet-V3 network, bilinear MobileNet-V3 (BM-Net), to analyze breast cancer WSIs. We utilized the WSI dataset from the ICIAR2018 Grand Challenge on Breast Cancer Histology Images (BACH) competition, which contains four classes: normal, benign, in situ carcinoma, and invasive carcinoma. We adopted data augmentation techniques to increase diversity and utilized focal loss to remove class imbalance. We achieved high performance, with 0.88 accuracy in patch classification and an average 0.71 score, which surpassed state-of-the-art models. Our BM-Net shows great potential in detecting cancer in WSIs and is a promising clinical tool.

Adversarial Multiscale Feature Learning Framework for Overlapping Chromosome Segmentation.

Chromosome karyotype analysis is of great clinical importance in the diagnosis and treatment of diseases. Since manual analysis is highly time and effort consuming, computer-assisted automatic chromosome karyotype analysis based on images is routinely used to improve the efficiency and accuracy of the analysis. However, the strip-shaped chromosomes easily overlap each other when imaged, significantly affecting the accuracy of the subsequent analysis and hindering the development of chromosome analysis instruments. In this paper, we present an adversarial, multiscale feature learning framework to improve the accuracy and adaptability of overlapping chromosome segmentation. We first adopt the nested U-shaped network with dense skip connections as the generator to explore the optimal representation of the chromosome images by exploiting multiscale features. Then we use the conditional generative adversarial network (cGAN) to generate images similar to the original ones; the training stability of the network is enhanced by applying the least-square GAN objective. Finally, we replace the common cross-entropy loss with the advanced Lovász-Softmax loss to improve the model's optimization and accelerate the model's convergence. Comparing with the established algorithms, the performance of our framework is proven superior by using public datasets in eight evaluation criteria, showing its great potential in overlapping chromosome segmentation.

High-throughput and high-accuracy diagnosis of multiple myeloma with multi-object detection.

Multiple myeloma (MM) is a type of blood cancer where plasma cells abnormally multiply and crowd out regular blood cells in the bones. Automated analysis of bone marrow smear examination is considered promising to improve the performance and reduce the labor cost in MM diagnosis. To address the drawbacks in established methods, which mainly aim at identifying monoclonal plasma cells (monoclonal PCs) via binary classification, in this work, considering that monoclonal PCs is not the only basis in MM diagnosis, for the first we construct a multi-object detection model for MM diagnosis. The experimental results show that our model can handle the images at a throughput of 80 slides/s and identify six lineages of bone marrow cells with an average accuracy of 90.8%. This work makes a step further toward full-automatic and high-efficiency MM diagnosis.

Analysis of signal detection configurations in optical time-stretch imaging.

Optical time-stretch (OTS) imaging is effective for observing ultra-fast dynamic events in real time by virtue of its capability of acquiring images with high spatial resolution at high speed. In different implementations of OTS imaging, different configurations of its signal detection, i.e. fiber-coupled and free-space detection schemes, are employed. In this research, we quantitatively analyze and compare the two detection configurations of OTS imaging in terms of sensitivity and image quality with the USAF-1951 resolution chart and diamond films, respectively, providing a valuable guidance for the system design of OTS imaging in diverse fields.

Temporally interleaved optical time-stretch imaging.

Optical time-stretch imaging has shown potential in diverse fields for its capability of acquiring images at high speed and high resolution. However, its wide application is hindered by the stringent requirement on the instrumentation hardware caused by the high-speed serial data stream. Here we demonstrate temporally interleaved optical time-stretch imaging that lowers the requirement without sacrificing the frame rate or spatial resolution by interleaving the high-speed data stream into multiple channels in the time domain. Its performance is validated with both a United States Air Force (USAF)-1951 resolution chart and a single-crystal diamond film. We achieve a 101 Mfps 1D scanning rate and 3 µm spatial resolution with only a 2.5 GS/s sampling rate by using a two-channel-interleaved system.