Development of a modified 3D region proposal network for lung nodule detection in computed tomography scans: a secondary analysis of lung nodule datasets.

BACKGROUND: Low-dose computed tomography (LDCT) has been shown useful in early lung cancer detection. This study aimed to develop a novel deep learning model for detecting pulmonary nodules on chest LDCT images. METHODS: In this secondary analysis, three lung nodule datasets, including Lung Nodule Analysis 2016 (LUNA16), Lung Nodule Received Operation (LNOP), and Lung Nodule in Health Examination (LNHE), were used to train and test deep learning models. The 3D region proposal network (RPN) was modified via a series of pruning experiments for better predictive performance. The performance of each modified deep leaning model was evaluated based on sensitivity and competition performance metric (CPM). Furthermore, the performance of the modified 3D RPN trained on three datasets was evaluated by 10-fold cross validation. Temporal validation was conducted to assess the reliability of the modified 3D RPN for detecting lung nodules. RESULTS: The results of pruning experiments indicated that the modified 3D RPN composed of the Cross Stage Partial Network (CSPNet) approach to Residual Network (ResNet) Xt (CSP-ResNeXt) module, feature pyramid network (FPN), nearest anchor method, and post-processing masking, had the optimal predictive performance with a CPM of 92.2%. The modified 3D RPN trained on the LUNA16 dataset had the highest CPM (90.1%), followed by the LNOP dataset (CPM: 74.1%) and the LNHE dataset (CPM: 70.2%). When the modified 3D RPN trained and tested on the same datasets, the sensitivities were 94.6%, 84.8%, and 79.7% for LUNA16, LNOP, and LNHE, respectively. The temporal validation analysis revealed that the modified 3D RPN tested on LNOP test set achieved a CPM of 71.6% and a sensitivity of 85.7%, and the modified 3D RPN tested on LNHE test set had a CPM of 71.7% and a sensitivity of 83.5%. CONCLUSION: A modified 3D RPN for detecting lung nodules on LDCT scans was designed and validated, which may serve as a computer-aided diagnosis system to facilitate lung nodule detection and lung cancer diagnosis.

A modified 3D RPN for detecting lung nodules on CT images that exhibited greater sensitivity and CPM than did several previously reported CAD detection models was established.

Development of a real-time RGB-D visual feedback-assisted pulmonary rehabilitation system.

Background: Following surgery, perioperative pulmonary rehabilitation (PR) is important for patients with early-stage lung cancer. However, current inpatient programs are often limited in time and space, and outpatient settings have access barriers. Therefore, we aimed to develop a background-free, zero-contact thoracoabdominal movement-tracking model that is easily set up and incorporated into a pre-existing PR program or extended to home-based rehabilitation and remote monitoring. We validated its effectiveness in providing preclinical real-time RGB-D (colour-depth camera) visual feedback. Methods: Twelve healthy volunteers performed deep breathing exercises following audio instruction for three cycles, followed by audio instruction and real-time visual feedback for another three cycles. In the visual feedback system, we used a RealSense™ D415 camera to capture RGB and depth images for human pose-estimation with Google MediaPipe. Target-tracking regions were defined based on the relative position of detected joints. The processed depth information of the tracking regions was visualised on a screen as a motion bar to provide real-time visual feedback of breathing intensity. Pulmonary function was simultaneously recorded using spirometric measurements, and changes in pulmonary volume were derived from respiratory airflow signals. Results: Our movement-tracking model showed a very strong correlation (r = 0.90 ± 0.05) between thoracic motion signals and spirometric volume, and a strong correlation (r = 0.73 ± 0.22) between abdominal signals and spirometric volume. Displacement of the chest wall was enhanced by RGB-D visual feedback (23 vs 20 mm, P = 0.034), and accompanied by an increased lung volume (2.58 vs 2.30 L, P = 0.003). Conclusion: We developed an easily implemented thoracoabdominal movement-tracking model and reported the positive impact of real-time RGB-D visual feedback on self-promoted external chest wall expansion, accompanied by increased internal lung volumes. This system can be extended to home-based PR.

Combined model integrating deep learning, radiomics, and clinical data to classify lung nodules at chest CT.

OBJECTIVES: The study aimed to develop a combined model that integrates deep learning (DL), radiomics, and clinical data to classify lung nodules into benign or malignant categories, and to further classify lung nodules into different pathological subtypes and Lung Imaging Reporting and Data System (Lung-RADS) scores. MATERIALS AND METHODS: The proposed model was trained, validated, and tested using three datasets: one public dataset, the Lung Nodule Analysis 2016 (LUNA16) Grand challenge dataset (n = 1004), and two private datasets, the Lung Nodule Received Operation (LNOP) dataset (n = 1027) and the Lung Nodule in Health Examination (LNHE) dataset (n = 1525). The proposed model used a stacked ensemble model by employing a machine learning (ML) approach with an AutoGluon-Tabular classifier. The input variables were modified 3D convolutional neural network (CNN) features, radiomics features, and clinical features. Three classification tasks were performed: Task 1: Classification of lung nodules into benign or malignant in the LUNA16 dataset; Task 2: Classification of lung nodules into different pathological subtypes; and Task 3: Classification of Lung-RADS score. Classification performance was determined based on accuracy, recall, precision, and F1-score. Ten-fold cross-validation was applied to each task. RESULTS: The proposed model achieved high accuracy in classifying lung nodules into benign or malignant categories in LUNA 16 with an accuracy of 92.8%, as well as in classifying lung nodules into different pathological subtypes with an F1-score of 75.5% and Lung-RADS scores with an F1-score of 80.4%. CONCLUSION: Our proposed model provides an accurate classification of lung nodules based on the benign/malignant, different pathological subtypes, and Lung-RADS system.

Intelligent Tapping Machine: Tap Geometry Inspection.

Currently, the majority of industrial metal processing involves the use of taps for cutting. However, existing tap machines require relocation to specialized inspection stations and only assess the condition of the cutting edges for defects. They do not evaluate the quality of the cutting angles and the amount of removed material. Machine vision, a key component of smart manufacturing, is commonly used for visual inspection. Taps are employed for processing various materials. Traditional tap replacement relies on the technician's accumulated empirical experience to determine the service life of the tap. Therefore, we propose the use of visual inspection of the tap's external features to determine whether replacement or regrinding is needed. We examined the bearing surface of the tap and utilized single images to identify the cutting angle, clearance angle, and cone angles. By inspecting the side of the tap, we calculated the wear of each cusp. This inspection process can facilitate the development of a tap life system, allowing for the estimation of the durability and wear of taps and nuts made of different materials. Statistical analysis can be employed to predict the lifespan of taps in production lines. Experimental error is 16 µm. Wear from tapping 60 times is equivalent to 8 s of electric grinding. We have introduced a parameter, thread removal quantity, which has not been proposed by anyone else.

Defect Inspection Using Modified YoloV4 on a Stitched Image of a Spinning Tool.

In Industry 4.0, automation is a critical requirement for mechanical production. This study proposes a computer vision-based method to capture images of rotating tools and detect defects without the need to stop the machine in question. The study uses frontal lighting to capture images of the rotating tools and employs scale-invariant feature transform (SIFT) to identify features of the tool images. Random sample consensus (RANSAC) is then used to obtain homography information, allowing us to stitch the images together. The modified YOLOv4 algorithm is then applied to the stitched image to detect any surface defects on the tool. The entire tool image is divided into multiple patch images, and each patch image is detected separately. The results show that the modified YOLOv4 algorithm has a recall rate of 98.7% and a precision rate of 97.3%, and the defect detection process takes approximately 7.6 s to complete for each stitched image.

Nut Geometry Inspection Using Improved Hough Line and Circle Methods.

Nuts are the cornerstone of human industrial construction, especially A-grade nuts that can only be used in power plants, precision instruments, aircraft, and rockets. However, the traditional nuts inspection method is to manually operate the measuring instrument for conducting an inspection, so the quality of the A-grade nut cannot be guaranteed. In this work, a machine vision-based inspection system was proposed, which performs a real-time geometric inspection of the nuts before and after tapping on the production line. In order to automatically screen out A-Grade nuts on the production line, there are 7 inspections within this proposed nut inspection system. The measurements of parallel, opposite side length, straightness, radius, roundness, concentricity, and eccentricity were proposed. To shorten the overall detection time regarding nut production, the program needed to be accurate and uncomplicated. By modifying the Hough line and Hough circle, the algorithm became faster and more suitable for nut detection. The optimized Hough line and Hough circle can be used for all measures in the testing process.

Improving puncture accuracy in percutaneous CT-guided needle insertion with wireless inertial measurement unit: a phantom study.

OBJECTIVES: A novel method applying inertial measurement units (IMUs) was developed to assist CT-guided puncture, which enables real-time displays of planned and actual needle trajectories. The method was compared with freehand and laser protractor-assisted methods. METHODS: The phantom study was performed by three operators with 8, 2, and 0 years of experience in CT-guided procedure conducted five consecutive needle placements for three target groups using three methods (freehand, laser protractor-assisted, or IMU-assisted method). The endpoints included mediolateral angle error and caudocranial angle error of the first pass, the procedure time, the total number of needle passes, and the radiation dose. RESULTS: There was a significant difference in the number of needle passes (IMU 1.2 ± 0.42, laser protractor 2.9 ± 1.6, freehand 3.6 ± 2.0 time, p < 0.001), the procedure time (IMU 3.0 ± 1.2, laser protractor 6.4 ± 2.9, freehand 6.2 ± 3.1 min, p < 0.001), the mediolateral angle error of the first pass (IMU 1.4 ± 1.2, laser protractor 1.6 ± 1.3, freehand 3.7 ± 2.5 degree, p < 0.001), the caudocranial angle error of the first pass (IMU 1.2 ± 1.2, laser protractor 5.3 ± 4.7, freehand 3.9 ± 3.1 degree, p < 0.001), and the radiation dose (IMU 250.5 ± 74.1, laser protractor 484.6 ± 260.2, freehand 561.4 ± 339.8 mGy-cm, p < 0.001) among three CT-guided needle insertion methods. CONCLUSION: The wireless IMU improves the angle accuracy and speed of CT-guided needle punctures as compared with laser protractor guidance and freehand techniques. KEY POINTS: • The IMU-assisted method showed a significant decrease in the number of needle passes (IMU 1.2 ± 0.42, laser protractor 2.9 ± 1.6, freehand 3.6 ± 2.0 time, p < 0.001). • The IMU-assisted method showed a significant decrease in the procedure time (IMU 3.0 ± 1.2, laser protractor 6.4 ± 2.9, freehand 6.2 ± 3.1 min, p < 0.001). • The IMU-assisted method showed a significant decrease in the mediolateral angle error of the first pass and the caudocranial angle error of the first pass.

Cascaded Segmentation U-Net for Quality Evaluation of Scraping Workpiece.

In the terms of industry, the hand-scraping method is a key technology for achieving high precision in machine tools, and the quality of scraping workpieces directly affects the accuracy and service life of the machine tool. However, most of the quality evaluation of the scraping workpieces is carried out by the scraping worker's subjective judgment, which results in differences in the quality of the scraping workpieces and is time-consuming. Hence, in this research, an edge-cloud computing system was developed to obtain the relevant parameters, which are the percentage of point (POP) and the peak point per square inch (PPI), for evaluating the quality of scraping workpieces. On the cloud computing server-side, a novel network called cascaded segmentation U-Net is proposed to high-quality segment the height of points (HOP) (around 40 µm height) in favor of small datasets training and then carries out a post-processing algorithm that automatically calculates POP and PPI. This research emphasizes the architecture of the network itself instead. The design of the components of our network is based on the basic idea of identity function, which not only solves the problem of the misjudgment of the oil ditch and the residual pigment but also allows the network to be end-to-end trained effectively. At the head of the network, a cascaded multi-stage pixel-wise classification is designed for obtaining more accurate HOP borders. Furthermore, the "Cross-dimension Compression" stage is used to fuse high-dimensional semantic feature maps across the depth of the feature maps into low-dimensional feature maps, producing decipherable content for final pixel-wise classification. Our system can achieve an error rate of 3.7% and 0.9 points for POP and PPI. The novel network achieves an Intersection over Union (IoU) of 90.2%.

A 10-item Fugl-Meyer Motor Scale Based on Machine Learning.

OBJECTIVE: The Fugl-Meyer motor scale (FM) is a well-validated measure for assessing upper extremity and lower extremity motor functions in people with stroke. The FM contains numerous items (50), which reduces its clinical usability. The purpose of this study was to develop a short form of the FM for people with stroke using a machine-learning methodology (FM-ML) and compare the efficiency (ie, number of items) and psychometric properties of the FM-ML with those of other FM versions, including the original FM, the 37-item FM, and the 12-item FM. METHODS: This observational study with follow-up used secondary data analysis. For developing the FM-ML, the random lasso method of ML was used to select the 10 most informative items (in terms of index of importance). Next, the scores of the FM-ML were calculated using an artificial neural network. Finally, the concurrent validity, predictive validity, responsiveness, and test-retest reliability of all FM versions were examined. RESULTS: The FM-ML used fewer items (80% fewer than the FM, 73% fewer than the 37-item FM, and 17% fewer than the 12-item FM) to achieve psychometric properties comparable with those of the other FM versions (concurrent validity: Pearson r = 0.95-0.99 vs 0.91-0.97; responsiveness: Pearson r = 0.78-0.91 vs 0.33-0.72; and test-retest reliability: intraclass correlation coefficient = 0.88-0.92 vs 0.93-0.98). CONCLUSION: The findings preliminarily support the efficiency and psychometric properties of the 10-item FM-ML. IMPACT: The FM-ML has potential to substantially improve the efficiency of motor function assessments in patients with stroke.

A PCB Alignment System Using RST Template Matching with CUDA on Embedded GPU Board.

The fiducial-marks-based alignment process is one of the most critical steps in printed circuit board (PCB) manufacturing. In the alignment process, a machine vision technique is used to detect the fiducial marks and then adjust the position of the vision system in such a way that it is aligned with the PCB. The present study proposed an embedded PCB alignment system, in which a rotation, scale and translation (RST) template-matching algorithm was employed to locate the marks on the PCB surface. The coordinates and angles of the detected marks were then compared with the reference values which were set by users, and the difference between them was used to adjust the position of the vision system accordingly. To improve the positioning accuracy, the angle and location matching process was performed in refinement processes. To overcome the matching time, in the present study we accelerated the rotation matching by eliminating the weak features in the scanning process and converting the normalized cross correlation (NCC) formula to a sum of products. Moreover, the scanning time was reduced by implementing the entire RST process in parallel on threads of a graphics processing unit (GPU) by applying hash functions to find refined positions in the refinement matching process. The experimental results showed that the resulting matching time was around 32× faster than that achieved on a conventional central processing unit (CPU) for a test image size of 1280 × 960 pixels. Furthermore, the precision of the alignment process achieved a considerable result with a tolerance of 36.4µm.

Photography enhancement based on the fusion of tone and color mappings in adaptive local region.

Existing tone reproduction schemes are generally based on a single image and are, therefore, unable to accurately recover the local details and colors of scene since the limited available information. Accordingly, the proposed tone reproduction system utilizes two images with different exposures (one low and one high) to capture the local detail and color information of low- and high-luminance regions of scene, respectively. The adaptive local region of each pixel is developed in order to appropriately reveal the details and maintain the overall impression of scene. Our system implements the local tone mapping and color mapping based on the adaptive local region by taking the lowly-exposed image as the basis and referencing the information of highly-exposed image. The local tone mapping compresses the luminance range in the image and enhances the local contrast to reveal the details, while the local color mapping maps the precise color information from the highly-exposed image to the lowly-exposed image. Finally, a fusion process is proposed to mix the local tone mapping and local color mapping results to produce the output image. A multiresolution approach is also developed to reduce time cost. The experimental results confirm that the system generates realistic reproductions of HDR scenes.

Automatic location of facial feature points and synthesis of facial sketches using direct combined model.

Automatically locating multiple feature points (i.e., the shape) in a facial image and then synthesizing the corresponding facial sketch are highly challenging since facial images typically exhibit a wide range of poses, expressions, and scales, and have differing degrees of illumination and/or occlusion. When the facial sketches are to be synthesized in the unique sketching style of a particular artist, the problem becomes even more complex. To resolve these problems, this paper develops an automatic facial sketch synthesis system based on a novel direct combined model (DCM) algorithm. The proposed system executes three cascaded procedures, namely, 1) synthesis of the facial shape from the input texture information (i.e., the facial image); 2) synthesis of the exaggerated facial shape from the synthesized facial shape; and 3) synthesis of a sketch from the original input image and the synthesized exaggerated shape. Previous proposals for reconstructing facial shapes and synthesizing the corresponding facial sketches are heavily reliant on the quality of the texture reconstruction results, which, in turn, are highly sensitive to occlusion and lighting effects in the input image. However, the DCM approach proposed in this paper accurately reconstructs the facial shape and then produces lifelike synthesized facial sketches without the need to recover occluded feature points or to restore the texture information lost as a result of unfavorable lighting conditions. Moreover, the DCM approach is capable of synthesizing facial sketches from input images with a wide variety of facial poses, gaze directions, and facial expressions even when such images are not included within the original training data set.

Rapid image completion system using multiresolution patch-based directional and nondirectional approaches.

This study presents a rapid image completion system comprising a training (or analysis) process and an image completion (or synthesis) process. The proposed system adopts a multiresolution approach, which not only improves the convergence rate of the synthesis process, but also provides the ability to deal with large replaced regions. In the training process, a down-sampling approach is applied to create a patch-based texture eigenspace based on multiresolution background region information. In the image completion process, an up-sampling approach is applied to synthesize the replaced foreground regions. To ensure the continuity of the geometric texture structure between the original background scene regions and the replaced foreground regions, directional and nondirectional image completion approaches are developed to reconstruct the global geometric structure and to enhance the local detailed features of the replaced foreground regions in the lower and higher resolution level images, respectively. Moreover, the synthesis priority order of the individual patches and the appropriate choice of completion scheme (i.e., directional or nondirectional) are both determined in accordance with a Hessian matrix decision value (HMDV) parameter. Finally, a texture refinement process is performed to optimize the resolution of the synthesized result.