A survey on cell nuclei instance segmentation and classification: Leveraging context and attention.

Nuclear-derived morphological features and biomarkers provide relevant insights regarding the tumour microenvironment, while also allowing diagnosis and prognosis in specific cancer types. However, manually annotating nuclei from the gigapixel Haematoxylin and Eosin (H&E)-stained Whole Slide Images (WSIs) is a laborious and costly task, meaning automated algorithms for cell nuclei instance segmentation and classification could alleviate the workload of pathologists and clinical researchers and at the same time facilitate the automatic extraction of clinically interpretable features for artificial intelligence (AI) tools. But due to high intra- and inter-class variability of nuclei morphological and chromatic features, as well as H&E-stains susceptibility to artefacts, state-of-the-art algorithms cannot correctly detect and classify instances with the necessary performance. In this work, we hypothesize context and attention inductive biases in artificial neural networks (ANNs) could increase the performance and generalization of algorithms for cell nuclei instance segmentation and classification. To understand the advantages, use-cases, and limitations of context and attention-based mechanisms in instance segmentation and classification, we start by reviewing works in computer vision and medical imaging. We then conduct a thorough survey on context and attention methods for cell nuclei instance segmentation and classification from H&E-stained microscopy imaging, while providing a comprehensive discussion of the challenges being tackled with context and attention. Besides, we illustrate some limitations of current approaches and present ideas for future research. As a case study, we extend both a general (Mask-RCNN) and a customized (HoVer-Net) instance segmentation and classification methods with context- and attention-based mechanisms and perform a comparative analysis on a multicentre dataset for colon nuclei identification and counting. Although pathologists rely on context at multiple levels while paying attention to specific Regions of Interest (RoIs) when analysing and annotating WSIs, our findings suggest translating that domain knowledge into algorithm design is no trivial task, but to fully exploit these mechanisms in ANNs, the scientific understanding of these methods should first be addressed.

CHiMP: deep-learning tools trained on protein crystallization micrographs to enable automation of experiments.

A group of three deep-learning tools, referred to collectively as CHiMP (Crystal Hits in My Plate), were created for analysis of micrographs of protein crystallization experiments at the Diamond Light Source (DLS) synchrotron, UK. The first tool, a classification network, assigns images into categories relating to experimental outcomes. The other two tools are networks that perform both object detection and instance segmentation, resulting in masks of individual crystals in the first case and masks of crystallization droplets in addition to crystals in the second case, allowing the positions and sizes of these entities to be recorded. The creation of these tools used transfer learning, where weights from a pre-trained deep-learning network were used as a starting point and repurposed by further training on a relatively small set of data. Two of the tools are now integrated at the VMXi macromolecular crystallography beamline at DLS, where they have the potential to absolve the need for any user input, both for monitoring crystallization experiments and for triggering in situ data collections. The third is being integrated into the XChem fragment-based drug-discovery screening platform, also at DLS, to allow the automatic targeting of acoustic compound dispensing into crystallization droplets.

A dual-labeled dataset and fusion model for automatic teeth segmentation, numbering, and state assessment on panoramic radiographs.

BACKGROUND: Recently, deep learning has been increasingly applied in the field of dentistry. The aim of this study is to develop a model for the automatic segmentation, numbering, and state assessment of teeth on panoramic radiographs. METHODS: We created a dual-labeled dataset on panoramic radiographs for training, incorporating both numbering and state labels. We then developed a fusion model that combines a YOLOv9-e instance segmentation model with an EfficientNetv2-l classification model. The instance segmentation model is used for tooth segmentation and numbering, whereas the classification model is used for state evaluation. The final prediction results integrate tooth position, numbering, and state information. The model's output includes result visualization and automatic report generation. RESULTS: Precision, Recall, mAP50 (mean Average Precision), and mAP50-95 for the tooth instance segmentation task are 0.989, 0.955, 0.975, and 0.840, respectively. Precision, Recall, Specificity, and F1 Score for the tooth classification task are 0.943, 0.933, 0.985, and 0.936, respectively. CONCLUSIONS: This fusion model is the first to integrate automatic dental segmentation, numbering, and state assessment. It provides highly accurate results, including detailed visualizations and automated report generation.

SMLS-YOLO: an extremely lightweight pathological myopia instance segmentation method.

Pathological myopia is a major cause of blindness among people under 50 years old and can result in severe vision loss in extreme cases. Currently, its detection primarily relies on manual methods, which are slow and heavily dependent on the expertise of physicians, making them impractical for large-scale screening. To tackle these challenges, we propose SMLS-YOLO, an instance segmentation method based on YOLOv8n-seg. Designed for efficiency in large-scale screenings, SMLS-YOLO employs an extremely lightweight model. First, StarNet is introduced as the backbone of SMLS-YOLO to extract image features. Subsequently, the StarBlock from StarNet is utilized to enhance the C2f, resulting in the creation of the C2f-Star feature extraction module. Furthermore, shared convolution and scale reduction strategies are employed to optimize the segmentation head for a more lightweight design. Lastly, the model incorporates the Multi-Head Self-Attention (MHSA) mechanism following the backbone to further refine the feature extraction process. Experimental results on the pathological myopia dataset demonstrate that SMLS-YOLO outperforms the baseline YOLOv8n-seg by reducing model parameters by 46.9%, increasing Box mAP@0.5 by 2.4%, and enhancing Mask mAP@0.5 by 4%. Furthermore, when compared to other advanced instance segmentation and semantic segmentation algorithms, SMLS-YOLO also maintains a leading position, suggesting that SMLS-YOLO has promising applications in the segmentation of pathological myopia images.

High-resolution image dataset for the automatic classification of phenological stage and identification of racemes in Urochloa spp. hybrids.

Urochloa grasses are widely used forages in the Neotropics and are gaining importance in other regions due to their role in meeting the increasing global demand for sustainable agricultural practices. High-throughput phenotyping (HTP) is important for accelerating Urochloa breeding programs focused on improving forage and seed yield. While RGB imaging has been used for HTP of vegetative traits, the assessment of phenological stages and seed yield using image analysis remains unexplored in this genus. This work presents a dataset of 2,400 high-resolution RGB images of 200 Urochloa hybrid genotypes, captured over seven months and covering both vegetative and reproductive stages. Images were manually labelled as vegetative or reproductive, and a subset of 255 reproductive stage images were annotated to identify 22,340 individual racemes. This dataset enables the development of machine learning and deep learning models for automated phenological stage classification and raceme identification, facilitating HTP and accelerated breeding of Urochloa spp. hybrids with high seed yield potential.

Accurate detection and instance segmentation of unstained living adherent cells in differential interference contrast images.

Detecting and segmenting unstained living adherent cells in differential interference contrast (DIC) images is crucial in biomedical research, such as cell microinjection, cell tracking, cell activity characterization, and revealing cell phenotypic transition dynamics. We present a robust approach, starting with dataset transformation. We curated 520 pairs of DIC images, containing 12,198 HepG2 cells, with ground truth annotations. The original dataset was randomly split into training, validation, and test sets. Rotations were applied to images in the training set, creating an interim "α set." Similar transformations formed "ß" and "γ sets" for validation and test data. The α set trained a Mask R-CNN, while the ß set produced predictions, subsequently filtered and categorized. A residual network (ResNet) classifier determined mask retention. The γ set underwent iterative processing, yielding final segmentation. Our method achieved a weighted average of 0.567 in average precision (AP)0.75bbox and 0.673 in AP0.75segm, both outperforming major algorithms for cell detection and segmentation. Visualization also revealed that our method excels in practicality, accurately capturing nearly every cell, a marked improvement over alternatives.

The Development of a Yolov8-Based Model for the Measurement of Critical Shoulder Angle (CSA), Lateral Acromion Angle (LAA), and Acromion Index (AI) from Shoulder X-ray Images.

Background: The accurate and effective evaluation of parameters such as critical shoulder angle, lateral acromion angle, and acromion index from shoulder X-ray images is crucial for identifying pathological changes and assessing disease risk in the shoulder joint. Methods: In this study, a YOLOv8-based model was developed to automatically measure these three parameters together, contributing to the existing literature. Initially, YOLOv8 was used to segment the acromion, glenoid, and humerus regions, after which the CSA, LAA angles, and AI between these regions were calculated. The MURA dataset was employed in this study. Results: Segmentation performance was evaluated with the Dice and Jaccard similarity indices, both exceeding 0.9. Statistical analyses of the measurement performance, including Pearson correlation coefficient, RMSE, and ICC values demonstrated that the proposed model exhibits high consistency and similarity with manual measurements. Conclusions: The results indicate that automatic measurement methods align with manual measurements with high accuracy and offer an effective alternative for clinical applications. This study provides valuable insights for the early diagnosis and management of shoulder diseases and makes a significant contribution to existing measurement methods.

DIO-SLAM: A Dynamic RGB-D SLAM Method Combining Instance Segmentation and Optical Flow.

Feature points from moving objects can negatively impact the accuracy of Visual Simultaneous Localization and Mapping (VSLAM) algorithms, while detection or semantic segmentation-based VSLAM approaches often fail to accurately determine the true motion state of objects. To address this challenge, this paper introduces DIO-SLAM: Dynamic Instance Optical Flow SLAM, a VSLAM system specifically designed for dynamic environments. Initially, the detection thread employs YOLACT (You Only Look At CoefficienTs) to distinguish between rigid and non-rigid objects within the scene. Subsequently, the optical flow thread estimates optical flow and introduces a novel approach to capture the optical flow of moving objects by leveraging optical flow residuals. Following this, an optical flow consistency method is implemented to assess the dynamic nature of rigid object mask regions, classifying them as either moving or stationary rigid objects. To mitigate errors caused by missed detections or motion blur, a motion frame propagation method is employed. Lastly, a dense mapping thread is incorporated to filter out non-rigid objects using semantic information, track the point clouds of rigid objects, reconstruct the static background, and store the resulting map in an octree format. Experimental results demonstrate that the proposed method surpasses current mainstream dynamic VSLAM techniques in both localization accuracy and real-time performance.

An Improved Instance Segmentation Method for Complex Elements of Farm UAV Aerial Survey Images.

Farm aerial survey layers can assist in unmanned farm operations, such as planning paths and early warnings. To address the inefficiencies and high costs associated with traditional layer construction, this study proposes a high-precision instance segmentation algorithm based on SparseInst. Considering the structural characteristics of farm elements, this study introduces a multi-scale attention module (MSA) that leverages the properties of atrous convolution to expand the sensory field. It enhances spatial and channel feature weights, effectively improving segmentation accuracy for large-scale and complex targets in the farm through three parallel dense connections. A bottom-up aggregation path is added to the feature pyramid fusion network, enhancing the model's ability to perceive complex targets such as mechanized trails in farms. Coordinate attention blocks (CAs) are incorporated into the neck to capture richer contextual semantic information, enhancing farm aerial imagery scene recognition accuracy. To assess the proposed method, we compare it against existing mainstream object segmentation models, including the Mask R-CNN, Cascade-Mask, SOLOv2, and Condinst algorithms. The experimental results show that the improved model proposed in this study can be adapted to segment various complex targets in farms. The accuracy of the improved SparseInst model greatly exceeds that of Mask R-CNN and Cascade-Mask and is 10.8 and 12.8 percentage points better than the average accuracy of SOLOv2 and Condinst, respectively, with the smallest number of model parameters. The results show that the model can be used for real-time segmentation of targets under complex farm conditions.

DAMM for the detection and tracking of multiple animals within complex social and environmental settings.

Accurate detection and tracking of animals across diverse environments are crucial for studying brain and behavior. Recently, computer vision techniques have become essential for high-throughput behavioral studies; however, localizing animals in complex conditions remains challenging due to intra-class visual variability and environmental diversity. These challenges hinder studies in naturalistic settings, such as when animals are partially concealed within nests. Moreover, current tools are laborious and time-consuming, requiring extensive, setup-specific annotation and training procedures. To address these challenges, we introduce the 'Detect-Any-Mouse-Model' (DAMM), an object detector for localizing mice in complex environments with minimal training. Our approach involved collecting and annotating a diverse dataset of single- and multi-housed mice in complex setups. We trained a Mask R-CNN, a popular object detector in animal studies, to perform instance segmentation and validated DAMM's performance on a collection of downstream datasets using zero-shot and few-shot inference. DAMM excels in zero-shot inference, detecting mice and even rats, in entirely unseen scenarios and further improves with minimal training. Using the SORT algorithm, we demonstrate robust tracking, competitive with keypoint-estimation-based methods. Notably, to advance and simplify behavioral studies, we release our code, model weights, and data, along with a user-friendly Python API and a Google Colab implementation.

A robust image segmentation and synthesis pipeline for histopathology.

Significant diagnostic variability between and within observers persists in pathology, despite the fact that digital slide images provide the ability to measure and quantify features much more precisely compared to conventional methods. Automated and accurate segmentation of cancerous cell and tissue regions can streamline the diagnostic process, providing insights into the cancer progression, and helping experts decide on the most effective treatment. Here, we evaluate the performance of the proposed PathoSeg model, with an architecture comprising of a modified HRNet encoder and a UNet++ decoder integrated with a CBAM block to utilize attention mechanism for an improved segmentation capability. We demonstrate that PathoSeg outperforms the current state-of-the-art (SOTA) networks in both quantitative and qualitative assessment of instance and semantic segmentation. Notably, we leverage the use of synthetic data generated by PathopixGAN, which effectively addresses the data imbalance problem commonly encountered in histopathology datasets, further improving the performance of PathoSeg. It utilizes spatially adaptive normalization within a generative and discriminative mechanism to synthesize diverse histopathological environments dictated through semantic information passed through pixel-level annotated Ground Truth semantic masks.Besides, we contribute to the research community by providing an in-house dataset that includes semantically segmented masks for breast carcinoma tubules (BCT), micro/macrovesicular steatosis of the liver (MSL), and prostate carcinoma glands (PCG). In the first part of the dataset, we have a total of 14 whole slide images from 13 patients' liver, with fat cell segmented masks, totaling 951 masks of size 512 × 512 pixels. In the second part, it includes 17 whole slide images from 13 patients with prostate carcinoma gland segmentation masks, amounting to 30,000 masks of size 512 × 512 pixels. In the third part, the dataset contains 51 whole slides from 36 patients, with breast carcinoma tubule masks totaling 30,000 masks of size 512 × 512 pixels. To ensure transparency and encourage further research, we will make this dataset publicly available for non-commercial and academic purposes. To facilitate reproducibility and encourage further research, we will also make our code and pre-trained models publicly available at https://github.com/DeepMIALab/PathoSeg.

Lesion Localization and Pathological Diagnosis of Ovine Pulmonary Adenocarcinoma Based on MASK R-CNN.

Ovine pulmonary adenocarcinoma (OPA) is a contagious lung tumour caused by the Jaagsiekte Sheep Retrovirus (JSRV). Histopathological diagnosis is the gold standard for OPA diagnosis. However, interpretation of traditional pathology images is complex and operator dependent. The mask regional convolutional neural network (Mask R-CNN) has emerged as a valuable tool in pathological diagnosis. This study utilized 54 typical OPA whole slide images (WSI) to extract 7167 typical lesion images containing OPA to construct a Common Objects in Context (COCO) dataset for OPA pathological images. The dataset was categorized into training and test sets (8:2 ratio) for model training and validation. Mean average specificity (mASp) and average sensitivity (ASe) were used to evaluate model performance. Six WSI-level pathological images (three OPA and three non-OPA images), not included in the dataset, were used for anti-peeking model validation. A random selection of 500 images, not included in the dataset establishment, was used to compare the performance of the model with assessment by pathologists. Accuracy, sensitivity, specificity, and concordance rate were evaluated. The model achieved a mASp of 0.573 and an ASe of 0.745, demonstrating effective lesion detection and alignment with expert annotation. In Anti-Peeking verification, the model showed good performance in locating OPA lesions and distinguished OPA from non-OPA pathological images. In the random 500-image diagnosis, the model achieved 92.8% accuracy, 100% sensitivity, and 88% specificity. The agreement rates between junior and senior pathologists were 100% and 96.5%, respectively. In conclusion, the Mask R-CNN-based OPA diagnostic model developed for OPA facilitates rapid and accurate diagnosis in practical applications.

A Method for Sorting High-Quality Fresh Sichuan Pepper Based on a Multi-Domain Multi-Scale Feature Fusion Algorithm.

Post-harvest selection of high-quality Sichuan pepper is a critical step in the production process. To achieve this, a visual system needs to analyze Sichuan pepper with varying postures and maturity levels. To quickly and accurately sort high-quality fresh Sichuan pepper, this study proposes a multi-scale frequency domain feature fusion module (MSF3M) and a multi-scale dual-domain feature fusion module (MS-DFFM) to construct a multi-scale, multi-domain fusion algorithm for feature fusion of Sichuan pepper images. The MultiDomain YOLOv8 Model network is then built to segment and classify the target Sichuan pepper, distinguishing the maturity level of individual Sichuan peppercorns. A selection method based on the average local pixel value difference is proposed for sorting high-quality fresh Sichuan pepper. Experimental results show that the MultiDomain YOLOv8-seg achieves an mAP50 of 88.8% for the segmentation of fresh Sichuan pepper, with a model size of only 5.84 MB. The MultiDomain YOLOv8-cls excels in Sichuan pepper maturity classification, with an accuracy of 98.34%. Compared to the YOLOv8 baseline model, the MultiDomain YOLOv8 model offers higher accuracy and a more lightweight structure, making it highly effective in reducing misjudgments and enhancing post-harvest processing efficiency in agricultural applications, ultimately increasing producer profits.

High-Precision Automated Soybean Phenotypic Feature Extraction Based on Deep Learning and Computer Vision.

The automated collection of plant phenotypic information has become a trend in breeding and smart agriculture. Four YOLOv8-based models were used to segment mature soybean plants placed in a simple background in a laboratory environment, identify pods, distinguish the number of soybeans in each pod, and obtain soybean phenotypes. The YOLOv8-Repvit model yielded the most optimal recognition results, with an R2 coefficient value of 0.96 for both pods and beans, and the RMSE values were 2.89 and 6.90, respectively. Moreover, a novel algorithm was devised to efficiently differentiate between the main stem and branches of soybean plants, called the midpoint coordinate algorithm (MCA). This was accomplished by linking the white pixels representing the stems in each column of the binary image to draw curves that represent the plant structure. The proposed method reduces computational time and spatial complexity in comparison to the A* algorithm, thereby providing an efficient and accurate approach for measuring the phenotypic characteristics of soybean plants. This research lays a technical foundation for obtaining the phenotypic data of densely overlapped and partitioned mature soybean plants under field conditions at harvest.

Detecting the interaction between microparticles and biomass in biological wastewater treatment process with Deep Learning method.

Investigating the interaction between influent particles and biomass is basic and important for the biological wastewater treatment. The micro-level methods allow for this, such as the microscope image analysis method with the conventional ImageJ processing software. However, these methods are cost and time-consuming, and require a large amount of work on manual parameter tuning. To deal with this problem, we proposed a deep learning (DL) method to automatically detect and quantify microparticles free from biomass and entrapped in biomass from microscope images. Firstly, we introduced a "TU Delft-Interaction between Particles and Biomass" dataset containing labeled microscope images. Then, we built DL models using this dataset with seven state-of-the-art model architectures for a instance segmentation task, such as Mask R-CNN, Cascade Mask R-CNN, Yolact and YOLOv8. The results show that the Cascade Mask R-CNN with ResNet50 backbone achieves promising detection accuracy, with a mAP50box and mAP50mask of 90.6 % on the test set. Then, we benchmarked our results against the conventional ImageJ processing method. The results show that the DL method significantly outperforms the ImageJ processing method in terms of detection accuracy and processing cost. The DL method shows a 13.8 % improvement in micro-average precision, and a 21.7 % improvement in micro-average recall, compared to the ImageJ method. Moreover, the DL method can process 70 images within 1 min, while the ImageJ method costs at least 6 h. The promising performance of our method allows it to offer a potential alternative to examine the interaction between microparticles and biomass in biological wastewater treatment process in an affordable manner. This approach offers more useful insights into the treatment process, enabling further reveal the microparticles transfer in biological treatment systems.

TP-Transfiner: high-quality segmentation network for tea pest.

Detecting and controlling tea pests promptly are crucial for safeguarding tea production quality. Due to the insufficient feature extraction ability of traditional CNN-based methods, they face challenges such as inaccuracy and inefficiency of detecting pests in dense and mimicry scenarios. This study proposes an end-to-end tea pest detection and segmentation framework, TeaPest-Transfiner (TP-Transfiner), based on Mask Transfiner to address the challenge of detecting and segmenting pests in mimicry and dense scenarios. In order to improve the feature extraction inability and weak accuracy of traditional convolution modules, this study proposes three strategies. Firstly, a deformable attention block is integrated into the model, which consists of deformable convolution and self-attention using the key content only term. Secondly, the FPN architecture in the backbone network is improved with a more effective feature-aligned pyramid network (FaPN). Lastly, focal loss is employed to balance positive and negative samples during the training period, and parameters are adapted to the dataset distribution. Furthermore, to address the lack of tea pest images, a dataset called TeaPestDataset is constructed, which contains 1,752 images and 29 species of tea pests. Experimental results on the TeaPestDataset show that the proposed TP-Transfiner model achieves state-of-the-art performance compared with other models, attaining a detection precision (AP50) of 87.211% and segmentation performance of 87.381%. Notably, the model shows a significant improvement in segmentation average precision (mAP) by 9.4% and a reduction in model size by 30% compared to the state-of-the-art CNN-based model Mask R-CNN. Simultaneously, TP-Transfiner's lightweight module fusion maintains fast inference speeds and a compact model size, demonstrating practical potential for pest control in tea gardens, especially in dense and mimicry scenarios.

Segmentation and characterization of macerated fibers and vessels using deep learning.

PURPOSE: Wood comprises different cell types, such as fibers, tracheids and vessels, defining its properties. Studying cells' shape, size, and arrangement in microscopy images is crucial for understanding wood characteristics. Typically, this involves macerating (soaking) samples in a solution to separate cells, then spreading them on slides for imaging with a microscope that covers a wide area, capturing thousands of cells. However, these cells often cluster and overlap in images, making the segmentation difficult and time-consuming using standard image-processing methods. RESULTS: In this work, we developed an automatic deep learning segmentation approach that utilizes the one-stage YOLOv8 model for fast and accurate segmentation and characterization of macerated fiber and vessel form aspen trees in microscopy images. The model can analyze 32,640 x 25,920 pixels images and demonstrate effective cell detection and segmentation, achieving a mAP 0.5 - 0.95 of 78 %. To assess the model's robustness, we examined fibers from a genetically modified tree line known for longer fibers. The outcomes were comparable to previous manual measurements. Additionally, we created a user-friendly web application for image analysis and provided the code for use on Google Colab. CONCLUSION: By leveraging YOLOv8's advances, this work provides a deep learning solution to enable efficient quantification and analysis of wood cells suitable for practical applications.

CP-Net: Instance-aware part segmentation network for biological cell parsing.

Instance segmentation of biological cells is important in medical image analysis for identifying and segmenting individual cells, and quantitative measurement of subcellular structures requires further cell-level subcellular part segmentation. Subcellular structure measurements are critical for cell phenotyping and quality analysis. For these purposes, instance-aware part segmentation network is first introduced to distinguish individual cells and segment subcellular structures for each detected cell. This approach is demonstrated on human sperm cells since the World Health Organization has established quantitative standards for sperm quality assessment. Specifically, a novel Cell Parsing Net (CP-Net) is proposed for accurate instance-level cell parsing. An attention-based feature fusion module is designed to alleviate contour misalignments for cells with an irregular shape by using instance masks as spatial cues instead of as strict constraints to differentiate various instances. A coarse-to-fine segmentation module is developed to effectively segment tiny subcellular structures within a cell through hierarchical segmentation from whole to part instead of directly segmenting each cell part. Moreover, a sperm parsing dataset is built including 320 annotated sperm images with five semantic subcellular part labels. Extensive experiments on the collected dataset demonstrate that the proposed CP-Net outperforms state-of-the-art instance-aware part segmentation networks.

Analysis of pig posture detection in group-housed pigs using deep learning-based mask scoring instance segmentation.

Pig posture is closely linked with livestock health and welfare. There has been significant interest among researchers in using deep learning techniques for pig posture detection. However, this task is challenging due to variations in image angles and times, as well as the presence of multiple pigs in a single image. In this study, we explore an object detection and segmentation algorithm based on instance segmentation scoring to detect different pig postures (sternal lying, lateral lying, walking, and sitting) and segment pig areas in group images, thereby enabling the identification of individual pig postures within a group. The algorithm combines a residual network with 50 layers and a feature pyramid network to extract feature maps from input images. These feature maps are then used to generate regions of interest (RoI) using a region candidate network. For each RoI, the algorithm performs regression to determine the location, classification, and segmentation of each pig posture. To address challenges such as missing targets and error detections among overlapping pigs in group housing, non-maximum suppression (NMS) is used with a threshold of 0.7. Through extensive hyperparameter analysis, a learning rate of 0.01, a batch size of 512, and 4 images per batch offer superior performance, with accuracy surpassing 96%. Similarly, the mean average precision (mAP) exceeds 83% for object detection and instance segmentation under these settings. Additionally, we compare the method with the faster R-CNN object detection model. Further, execution times on different processing units considering various hyperparameters and iterations have been analyzed.

[Tooth segmentation and identification on cone-beam computed tomography with convolutional neural network based on spatial embedding information].

OBJECTIVE: To propose a novel neural network to achieve tooth instance segmentation and recognition based on cone-beam computed tomography (CBCT) voxel data. METHODS: The proposed methods included three different convolutional neural network models. The architecture was based on the Resnet module and built according to the structure of "Encoder-Decoder" and U-Net. The CBCT image was de-sampled and a fixed-size region of interest (ROI) containing all the teeth was determined. ROI would first through a two-branch "encoder and decoder" structure of the network, the network could predict each voxel in the input data of the spatial embedding. The post-processing algorithm would cluster the prediction results of the relevant spatial location information according to the two-branch network to realize the tooth instance segmentation. The tooth position identification was realized by another U-Net model based on the multi-classification segmentation task. According to the predicted results of the network, the post-processing algorithm would classify the tooth position according to the voting results of each tooth instance segmentation. At the original spatial resolution, a U-Net network model for the fine-tooth segmentation was trained using the region corresponding to each tooth as the input. According to the results of instance segmentation and tooth position identification, the model would process the correspon-ding positions on the high-resolution CBCT images to obtain the high-resolution tooth segmentation results. In this study, CBCT data of 59 cases with simple crown prostheses and implants were collected for manual labeling as the database, and statistical indicators were evaluated for the prediction results of the algorithm. To assess the performance of tooth segmentation and classification, instance Dice similarity coefficient (IDSC) and the average Dice similarity coefficient (ADSC) were calculated. RESULTS: The experimental results showed that the IDSC was 89.35%, and the ADSC was 84. 74%. After eliminating the data with prostheses artifacts, the database of 43 samples was generated, and the performance of the training network was better, with 90.34% for IDSC and 87.88% for ADSC. The framework achieved excellent performance on tooth segmentation and identification. Voxels near intercuspation surfaces and fuzzy boundaries could be separated into correct instances by this framework. CONCLUSIONS: The results show that this method can not only successfully achieve 3D tooth instance segmentation but also identify all teeth notation numbers accurately, which has clinical practicability.