A novel complementation method of an acoustic shadow region utilizing a convolutional neural network for ultrasound-guided therapy.

PURPOSE: Noise-free ultrasound images are essential for organ monitoring during regional ultrasound-guided therapy. When the affected area is located under the ribs, however, acoustic shadow is caused by the reflection of sound from hard tissues such as bone, and the image is output with missing information in this region. Therefore, in the present study, we attempt to complement the image in the missing area. METHODS: The overall flow of the complementation method to generate a shadow-free composite image is as follows. First, we constructed a binary classification method for the presence or absence of acoustic shadow on a phantom kidney based on a convolutional neural network. Second, we created a composite shadow-free image by searching for a suitable image from a time-series database and superimposing the corresponding area without shadow onto the missing area of the target image. In addition, we constructed and verified an automatic kidney mask generation method utilizing U-Net. RESULTS: The complementation accuracy for kidney tracking could be enhanced by template matching. Zero-mean normalized cross-correlation (ZNCC) values after complementation were higher than that of before complementation under four different data generation conditions: (i) changing the position of the bed of the robotic ultrasound diagnostic system in the translational direction, (ii) changing the probe angle in the translational direction, (iii) with the addition of rotational motion of the probe to condition (ii). Although there was large variation in the shape of the kidney contour in condition (iii), the proposed method improved the ZNCC value from 0.5437 to 0.5807. CONCLUSIONS: The effectiveness of the proposed method was demonstrated in phantom experiments. Verification of its effectiveness in real organs is necessary in future study.

Evaluation of ultrasonic fibrosis diagnostic system using convolutional network for ordinal regression.

PURPOSE: Diagnosis of liver fibrosis is important for establishing treatment and assessing the risk of carcinogenesis. Ultrasound imaging is an excellent diagnostic method as a screening test in terms of non-invasiveness and simplicity. The purpose of this study was to automatically diagnose liver fibrosis using ultrasound images to reduce the burden on physicians. METHODS: We proposed and implemented a system for extracting regions of liver parenchyma utilizing U-Net. Using regions of interest, the stage of fibrosis was classified as F0, F1, F2, F3, or F4 utilizing CORALNet, an ordinal regression model based on ResNet18. The effectiveness of the proposed system was verified. RESULTS: The system implemented using U-Net had a maximum mean Dice coefficient of 0.929. The results of classification of liver fibrosis utilizing CORALNet had a mean absolute error (MAE) of 1.22 and root mean square error (RMSE) of 1.60. The per-case results had a MAE of 1.55 and RMSE of 1.34. CONCLUSION: U-Net extracted regions of liver parenchyma from the images with high accuracy, and CORALNet showed effectiveness using ordinal information to classify fibrosis in the images. As a future task, we will study a model that is less dependent on teaching data.

Estimating 3-dimensional liver motion using deep learning and 2-dimensional ultrasound images.

PURPOSE: The main purpose of this study is to construct a system to track the tumor position during radiofrequency ablation (RFA) treatment. Existing tumor tracking systems are designed to track a tumor in a two-dimensional (2D) ultrasound (US) image. As a result, the three-dimensional (3D) motion of the organs cannot be accommodated and the ablation area may be lost. In this study, we propose a method for estimating the 3D movement of the liver as a preliminary system for tumor tracking. Additionally, in current 3D movement estimation systems, the motion of different structures during RFA could reduce the tumor visibility in US images. Therefore, we also aim to improve the estimation of the 3D movement of the liver by improving the liver segmentation. We propose a novel approach to estimate the relative 6-axial movement (x, y, z, roll, pitch, and yaw) between the liver and the US probe in order to estimate the overall movement of the liver. METHOD: We used a convolutional neural network (CNN) to estimate the 3D displacement from two-dimensional US images. In addition, to improve the accuracy of the estimation, we introduced a segmentation map of the liver region as the input for the regression network. Specifically, we improved the extraction accuracy of the liver region by using a bi-directional convolutional LSTM U-Net with densely connected convolutions (BCDU-Net). RESULTS: By using BCDU-Net, the accuracy of the segmentation was dramatically improved, and as a result, the accuracy of the movement estimation was also improved. The mean absolute error for the out-of-plane direction was 0.0645 mm/frame. CONCLUSION: The experimental results show the effectiveness of our novel method to identify the movement of the liver by BCDU-Net and CNN. Precise segmentation of the liver by BCDU-Net also contributes to enhancing the performance of the liver movement estimation.