Spatiotemporal analysis of speckle dynamics to track invisible needle in ultrasound sequences using convolutional neural networks: a phantom study.

PURPOSE: Accurate needle placement into the target point is critical for ultrasound interventions like biopsies and epidural injections. However, aligning the needle to the thin plane of the transducer is a challenging issue as it leads to the decay of visibility by the naked eye. Therefore, we have developed a CNN-based framework to track the needle using the spatiotemporal features of the speckle dynamics. METHODS: There are three key techniques to optimize the network for our application. First, we used Gunnar-Farneback (GF) as a traditional motion field estimation technique to augment the model input with the spatiotemporal features extracted from the stack of consecutive frames. We also designed an efficient network based on the state-of-the-art Yolo framework (nYolo). Lastly, the Assisted Excitation (AE) module was added at the neck of the network to handle the imbalance problem. RESULTS: Fourteen freehand ultrasound sequences were collected by inserting an injection needle steeply into the Ultrasound Compatible Lumbar Epidural Simulator and Femoral Vascular Access Ezono test phantoms. We divided the dataset into two sub-categories. In the second category, in which the situation is more challenging and the needle is totally invisible, the angle and tip localization error were 2.43 ± 1.14° and 2.3 ± 1.76 mm using Yolov3+GF+AE and 2.08 ± 1.18° and 2.12 ± 1.43 mm using nYolo+GF+AE. CONCLUSION: The proposed method has the potential to track the needle in a more reliable operation compared to other state-of-the-art methods and can accurately localize it in 2D B-mode US images in real time, allowing it to be used in current ultrasound intervention procedures.

Coordinate-based fast lightweight path search algorithm for electromagnetic navigation bronchoscopy.

Electromagnetic navigation bronchoscopy (ENB) uses electromagnetic positioning technology to guide the bronchoscope to accurately and quickly reach the lesion along the planned path. However, enormous data in high-resolution lung computed tomography (CT) and the complex structure of multilevel branching bronchial tree make fast path search challenging for path planning. We propose a coordinate-based fast lightweight path search (CPS) algorithm for ENB. First, the centerline is extracted from the bronchial tree by applying topological thinning. Then, Euclidean-distance-based coordinate search is applied. The centerline points are represented by their coordinates, and adjacent points along the navigation path are selected considering the shortest Euclidean distance to the target on the centerline nearest the lesion. From the top of the trachea centerline, search is repeated until reaching the target. In 50 high-resolution lung CT images acquired from five scanners, the CPS algorithm achieves accuracy, average search time, and average memory consumption of 100%, 88.5 ms, and 166.0 MB, respectively, reducing search time by 74.3% and 73.1% and memory consumption by 83.3% and 83.0% compared with Dijkstra and A* algorithms, respectively. CPS algorithm is suitable for path search in multilevel branching bronchial tree navigation based on high-resolution lung CT images.

Differentiation between Pancreatic Ductal Adenocarcinoma and Normal Pancreatic Tissue for Treatment Response Assessment using Multi-Scale Texture Analysis of CT Images.

Background: Pancreatic ductal adenocarcinoma (PDAC) is the most prevalent type of pancreas cancer with a high mortality rate and its staging is highly dependent on the extent of involvement between the tumor and surrounding vessels, facilitating treatment response assessment in PDAC. Objective: This study aims at detecting and visualizing the tumor region and the surrounding vessels in PDAC CT scan since, despite the tumors in other abdominal organs, clear detection of PDAC is highly difficult. Material and Methods: This retrospective study consists of three stages: 1) a patch-based algorithm for differentiation between tumor region and healthy tissue using multi-scale texture analysis along with L1-SVM (Support Vector Machine) classifier, 2) a voting-based approach, developed on a standard logistic function, to mitigate false detections, and 3) 3D visualization of the tumor and the surrounding vessels using ITK-SNAP software. Results: The results demonstrate that multi-scale texture analysis strikes a balance between recall and precision in tumor and healthy tissue differentiation with an overall accuracy of 0.78±0.12 and a sensitivity of 0.90±0.09 in PDAC. Conclusion: Multi-scale texture analysis using statistical and wavelet-based features along with L1-SVM can be employed to differentiate between healthy and pancreatic tissues. Besides, 3D visualization of the tumor region and surrounding vessels can facilitate the assessment of treatment response in PDAC. However, the 3D visualization software must be further developed for integrating with clinical applications.

An improved capsule network for glioma segmentation on MRI images: A curriculum learning approach.

Glioma segmentation is an essential step in tumor identification and treatment planning. Glioma segmentation is a challenging task because it appears with blurred and irregular boundaries in a variety of shapes. In this paper, we propose an efficient and novel model for automatic glioma segmentation based on capsule neural networks. We improved the architecture and training of the SegCaps model, the first capsule-based segmentation network. The proposed architecture is improved by introducing dilation blocks in the primary capsule block to get deeper features while avoiding resolution reduction. The prediction layer of the network is also modified using one-dimensional convolution filters, enabling the network to not only maximize tumor existence likelihood but also regularize the capsule orientations within the tumor. Our main contribution, however, is to introduce an enhanced curriculum-based training algorithm into the proposed SegCaps model. We adapt the curriculum learning for the model by suggesting a new pacing mechanism based on a roulette-wheel selection algorithm that enriches randomness in the network and prevents bias. A hybrid dice loss function is also employed, which is better adapted to the introduced curriculum-based training procedure. We evaluated the performance of improved SegCaps on the BraTS2020, a multimodal benchmark dataset for brain tumor segmentation. The experimental results confirmed that the improvements yield a top-performing yet memory-efficient deep capsule architecture. The proposed model outperformed the best-reported accuracies on the BraTS2020, achieving improved dice scores of 85.16% and 81.88% for tumor core and enhancing tumor segmentation, respectively. Using 90%, fewer parameters than the popular U-Net also confirmed the high memory efficiency of our proposed model.

Photoacoustic-MR Image Registration Based on a Co-Sparse Analysis Model to Compensate for Brain Shift.

Brain shift is an important obstacle to the application of image guidance during neurosurgical interventions. There has been a growing interest in intra-operative imaging to update the image-guided surgery systems. However, due to the innate limitations of the current imaging modalities, accurate brain shift compensation continues to be a challenging task. In this study, the application of intra-operative photoacoustic imaging and registration of the intra-operative photoacoustic with pre-operative MR images are proposed to compensate for brain deformation. Finding a satisfactory registration method is challenging due to the unpredictable nature of brain deformation. In this study, the co-sparse analysis model is proposed for photoacoustic-MR image registration, which can capture the interdependency of the two modalities. The proposed algorithm works based on the minimization of mapping transform via a pair of analysis operators that are learned by the alternating direction method of multipliers. The method was evaluated using an experimental phantom and ex vivo data obtained from a mouse brain. The results of the phantom data show about 63% improvement in target registration error in comparison with the commonly used normalized mutual information method. The results proved that intra-operative photoacoustic images could become a promising tool when the brain shift invalidates pre-operative MRI.

Segmentation of pancreatic ductal adenocarcinoma (PDAC) and surrounding vessels in CT images using deep convolutional neural networks and texture descriptors.

Fully automated and volumetric segmentation of critical tumors may play a crucial role in diagnosis and surgical planning. One of the most challenging tumor segmentation tasks is localization of pancreatic ductal adenocarcinoma (PDAC). Exclusive application of conventional methods does not appear promising. Deep learning approaches has achieved great success in the computer aided diagnosis, especially in biomedical image segmentation. This paper introduces a framework based on convolutional neural network (CNN) for segmentation of PDAC mass and surrounding vessels in CT images by incorporating powerful classic features, as well. First, a 3D-CNN architecture is used to localize the pancreas region from the whole CT volume using 3D Local Binary Pattern (LBP) map of the original image. Segmentation of PDAC mass is subsequently performed using 2D attention U-Net and Texture Attention U-Net (TAU-Net). TAU-Net is introduced by fusion of dense Scale-Invariant Feature Transform (SIFT) and LBP descriptors into the attention U-Net. An ensemble model is then used to cumulate the advantages of both networks using a 3D-CNN. In addition, to reduce the effects of imbalanced data, a multi-objective loss function is proposed as a weighted combination of three classic losses including Generalized Dice Loss (GDL), Weighted Pixel-Wise Cross Entropy loss (WPCE) and boundary loss. Due to insufficient sample size for vessel segmentation, we used the above-mentioned pre-trained networks and fine-tuned them. Experimental results show that the proposed method improves the Dice score for PDAC mass segmentation in portal-venous phase by 7.52% compared to state-of-the-art methods in term of DSC. Besides, three dimensional visualization of the tumor and surrounding vessels can facilitate the evaluation of PDAC treatment response.

Cascaded Learning with Generative Adversarial Networks for Low Dose CT Denoising.

CT machines can be tuned in order to reduce the radiation dose used for imaging, yet reducing the radiation dose results in noisy images which are not suitable in clinical practice. In order for low dose CT to be used effectively in practice this issue must be addressed. Generative Adversarial Networks (GAN) have been used widely in computer vision research and have proven themselves as a powerful tool for producing images with high perceptual quality. In this work we use a cascade of two neural networks, the first is a Generative Adversarial Network and the second is a Deep Convolutional Neural Network. The first network generates a denoised sample which is then fine-tuned by the second network via residue learning. We show that our cascaded method outperforms related works and more effectively reconstructs fine structural details in low contrast regions of the image.

Bronchoscope motion tracking using centerline-guided Gaussian mixture model in navigated bronchoscopy.

Electromagnetic-based navigation bronchoscopy requires accurate and robust estimation of the bronchoscope position inside the bronchial tree. However, respiratory motion, coughing, patient movement, and airway deformation inflicted by bronchoscope significantly hinder the accuracy of intraoperative bronchoscopic localization. In this study, a real-time and automatic registration procedure was proposed to superimpose the current location of the bronchoscope to corresponding locations on a centerline extracted from bronchial computed tomography (CT) images. A centerline-guided Gaussian mixture model (CG-GMM) was introduced to register a bronchoscope's position concerning extracted centerlines. A GMM was fitted to bronchoscope positions where the orientation likelihood was chosen to assign the membership probabilities of the mixture model, which led to preserving the global and local structures. The problem was formulated and solved under the expectation maximization framework, where the feature correspondence and spatial transformation are estimated iteratively. Validation was performed on a dynamic phantom with four different respiratory motions and four human real bronchoscopy (RB) datasets. Results of the experiments conducted on the bronchial phantom showed that the average positional tracking error using the proposed approach was equal to 1.98 [Formula: see text] 0.98 mm that was reduced in comparison with independent electromagnetic tracking (EMT), iterative closest point (ICP), and coherent point drift (CPD) methods by 64%, 58%, and 53%, respectively. In the patient assessment part of the study, the average positional tracking error was 4.73 [Formula: see text] 4.76 mm and compared to ICP, and CPD methods showed 31.4% improvement of successfully tracked frames. Our approach introduces a novel method for real-time respiratory motion compensation that provides reliable guidance during bronchoscopic interventions and, thus could increase the diagnostic yield of transbronchial biopsy.

Photoacoustic image improvement based on a combination of sparse coding and filtering.

SIGNIFICANCE: Photoacoustic imaging (PAI) has been greatly developed in a broad range of diagnostic applications. The efficiency of light to sound conversion in PAI is limited by the ubiquitous noise arising from the tissue background, leading to a low signal-to-noise ratio (SNR), and thus a poor quality of images. Frame averaging has been widely used to reduce the noise; however, it compromises the temporal resolution of PAI. AIM: We propose an approach for photoacoustic (PA) signal denoising based on a combination of low-pass filtering and sparse coding (LPFSC). APPROACH: LPFSC method is based on the fact that PA signal can be modeled as the sum of low frequency and sparse components, which allows for the reduction of noise levels using a hybrid alternating direction method of multipliers in an optimization process. RESULTS: LPFSC method was evaluated using in-silico and experimental phantoms. The results show a 26% improvement in the peak SNR of PA signal compared to the averaging method for in-silico data. On average, LPFSC method offers a 63% improvement in the image contrast-to-noise ratio and a 33% improvement in the structural similarity index compared to the averaging method for objects located at three different depths, ranging from 10 to 20 mm, in a porcine tissue phantom. CONCLUSIONS: The proposed method is an effective tool for PA signal denoising, whereas it ultimately improves the quality of reconstructed images, especially at higher depths, without limiting the image acquisition speed.

Automatic detection of symmetry plane for computer-aided surgical simulation in craniomaxillofacial surgery.

Symmetry plane calculation is used in fracture reduction or reconstruction in the midface. Estimating a reliable symmetry plane without advanced anatomic knowledge is the most critical challenge. In this work, we developed a new automated method to find the mid-plane in CT images of an intact skull and a skull with a unilateral midface fracture. By use of a 3D point-cloud of a skull, we demonstrate that the proposed algorithm could find a mid-plane that meets clinical criteria. There is no need for advanced anatomical knowledge through the use of this algorithm. The algorithm used principal component analysis to find the initial plane. Then the rotation matrix, derived from an iterative closest point (ICP) registration method, is used to update the normal vector of the plane and find the optimum symmetry plane. A mathematical index, Hausdorff distance (HD), is used to evaluate the similarity of one mid-plane side in comparison to the contralateral side. HD decreased by 66% in the intact skull and 65% in a fractured skull and converged in just six iterations. High convergence speed, low computational load, and high accuracy suggest the use of the algorithm in the planning procedure. This easy-to-use algorithm with its advantages, as mentioned above, could be used as an operator in craniomaxillofacial software.

Erratum: Retinal image mosaicking using scale-invariant feature transformation feature descriptors and Voronoi diagram (Erratum).

[This corrects the article DOI: 10.1117/1.JMI.7.4.044001.].

Retinal image mosaicking using scale-invariant feature transformation feature descriptors and Voronoi diagram.

Purpose: Peripheral retinal lesions substantially increase the risk of diabetic retinopathy and retinopathy of prematurity. The peripheral changes can be visualized in wide field imaging, which is obtained by combining multiple images with an overlapping field of view using mosaicking methods. However, a robust and accurate registration of mosaicking techniques for normal angle fundus cameras is still a challenge due to the random selection of matching points and execution time. We propose a method of retinal image mosaicking based on scale-invariant feature transformation (SIFT) feature descriptor and Voronoi diagram. Approach: In our method, the SIFT algorithm is used to describe local features in the input images. Then the input images are subdivided into regions based on the Voronoi method. Each pair of Voronoi regions is matched by the method zero mean normalized cross correlation. After matching, the retinal images are mapped into the same coordinate system to form a mosaic image. The success rate and the mean registration error (RE) of our method were compared with those of other state-of-the-art methods for the P category of the fundus image registration database. Results: Experimental results show that the proposed method accurately registered 42% of retinal image pairs with a mean RE of 3.040 pixels, while a lower success rate was observed in the other four state-of-the-art retinal image registration methods GDB-ICP (33%), Harris-PIIFD (0%), HM-2016 (0%), and HM-2017 (2%). Conclusions: The proposed method outperforms state-of-the-art methods in terms of quality and running time and reduces the computational complexity.

Dictionary learning technique enhances signal in LED-based photoacoustic imaging.

There has been growing interest in low-cost light sources such as light-emitting diodes (LEDs) as an excitation source in photoacoustic imaging. However, LED-based photoacoustic imaging is limited by low signal due to low energy per pulse-the signal is easily buried in noise leading to low quality images. Here, we describe a signal de-noising approach for LED-based photoacoustic signals based on dictionary learning with an alternating direction method of multipliers. This signal enhancement method is then followed by a simple reconstruction approach delay and sum. This approach leads to sparse representation of the main components of the signal. The main improvements of this approach are a 38% higher contrast ratio and a 43% higher axial resolution versus the averaging method but with only 4% of the frames and consequently 49.5% less computational time. This makes it an appropriate option for real-time LED-based photoacoustic imaging.

Spinal pain relief procedures with the assistance of the MRI-updated statistical shape model.

BACKGROUND: Updating the statistical shape model (SSM) used in image guidance systems for the treatment of back pain using pre-op computed tomography (CT) and intra-op ultrasound (US) is challenging due to the scarce availability of pre-op images and the low resolution of the two imaging modalities. METHODS: A new approach is proposed here to update SSMs based on the sparse representation of the preoperative MRI images of patients as well as CT images, followed by displaying the injection needle and 3D tracking view of the patients' spine. RESULTS: The statistical analysis shows that updating the SSM using the patients' available MRI images (in more than 95% of the cases) instead of CT images (in less than 5%) will help maintain the required accuracy of needle injection based on the evaluation of injection in different parts of the phantom. CONCLUSION: The results show that using the proposed model helps reduce the dosage and processing time significantly while maintaining the precision required for the pain procedures.

High-quality photoacoustic image reconstruction based on deep convolutional neural network: towards intra-operative photoacoustic imaging.

The use of intra-operative imaging system as an intervention solution to provide more accurate localization of complicated structures has become a necessity during the neurosurgery. However, due to the limitations of conventional imaging systems, high-quality real-time intra-operative imaging remains as a challenging problem. Meanwhile, photoacoustic imaging has appeared so promising to provide images of crucial structures such as blood vessels and microvasculature of tumors. To achieve high-quality photoacoustic images of vessels regarding the artifacts caused by the incomplete data, we proposed an approach based on the combination of time-reversal (TR) and deep learning methods. The proposed method applies a TR method in the first layer of the network which is followed by the convolutional neural network with weights adjusted to a set of simulated training data for the other layers to estimate artifact-free photoacoustic images. It was evaluated using a generated synthetic database of vessels. The mean of signal to noise ratio (SNR), peak SNR, structural similarity index, and edge preservation index for the test data were reached 14.6 dB, 35.3 dB, 0.97 and 0.90, respectively. As our results proved, by using the lower number of detectors and consequently the lower data acquisition time, our approach outperforms the TR algorithm in all criteria in a computational time compatible with clinical use.

Compensation of dynamic electromagnetic field distortion using simultaneous localization and mapping method with application in endobronchial ultrasound-transbronchial needle aspiration (EBUS-TBNA) guidance.

BACKGROUND: Electromagnetic (EM)-based navigation methods without line-of-sight restrictions may improve lymph node sampling precision in transbronchial needle aspiration (TBNA) procedure. However, EM tracking susceptibility to metallic objects severely declines its precision. METHOD: We proposed to track the location of a tool in a dynamic bronchial phantom and compensate field distortion in a real-time procedure. Extended Kalman filter simultaneous localization and mapping (EKF-SLAM) algorithm employ the bronchial motion and observations of a redundant sensor. The proposed approach was applied to the phantom with four different amplitudes of breathing motion in the presence of two types of field-distorting objects. RESULTS: The proposed approach improved the accuracy of EM tracking on average from 18.94 ±1.17 mm to 4.59 ±0.29 mm and from 14.2 ±0.69 mm to 4.31 ±0.18mm in the presence of steel and aluminum, respectively. CONCLUSIONS: With EM tracking position error reduction based on the EKF-SLAM technique, the approach is appeared promising for a navigated ultrasound TBNA procedure.

Application of multi-wavelength technique for photoacoustic imaging to delineate tumor margins during maximum-safe resection of glioma: A preliminary simulation study.

Accurate margin delineation and safe maximal resection of glioma is one of the most challenging problems of neurosurgery, due to its close resemblance to normal brain parenchyma. However, different intraoperative visualization methods have been used for real-time intraoperative investigation of the borders of the resection cavity, each having advantages and limitations. This preliminary study was designed to simulate multi-wavelength photoacoustic imaging for brain tumor margin delineation for maximum safe resection of glioma. Since the photoacoustic signal is directly related to the amount of optical energy absorption by the endogenous tissue chromophores such as hemoglobin; it may be able to illustrate the critical structures such as tumor vessels during surgery. The simulation of the optical and acoustic part was done by using Monte-Carlo and k-wave toolbox, respectively. As our simulation results proved, at different wavelengths and depths, the amount of optical absorption for the blood layer is significantly different from others such as normal and tumoral tissues. Furthermore, experimental validation of our approach confirms that, by using multi-wavelengths proportional to the depth of the tumor margin during surgery, tumor margin can be differented using photoacoustic imaging at various depths. Photoacoustic imaging may be considered as a promising imaging modality which combines the spectral contrast of optical imaging as well as the spatial resolution of ultrasound imaging, and may be able to delineate the vascular-rich glioma margins at different depths of the resection cavity during surgery.

Artificially enriching the training dataset of statistical shape models via constrained cage-based deformation.

The construction of a powerful statistical shape model (SSM) requires a rich training dataset that includes the large variety of complex anatomical topologies. The lack of real data causes most SSMs unable to generalize possible unseen instances. Artificial enrichment of training data is one of the methods proposed to address this issue. In this paper, we introduce a novel technique called constrained cage-based deformation (CCBD), which has the ability to produce unlimited artificial data that promises to enrich variability within the training dataset. The proposed method is a two-step algorithm: in the first step, it moves a few handles together, and in the second step transfers the displacements of these handles to the base mesh vertices to generate a real new instance. The evaluation of statistical characteristics of the CCBD confirms that our proposed technique outperforms notable data-generating methods quantitatively, in terms of the generalization ability, and with respect to specificity.

A novel approach to the extraction of fetal electrocardiogram based on empirical mode decomposition and correlation analysis.

Fetal heart rate monitoring is the process of checking the condition of the fetus during pregnancy and it would allow doctors and nurses to detect early signs of trouble during labor and delivery. The fetal ECG (FECG) signal is so weak and also is corrupted by other signals and noises, mainly by maternal ECG signal. It is so hard to acquire a noise-free, precise and reliable FECG using the conventional methods. In this study, a combination of empirical mode decomposition (EMD) algorithms, correlation and match filtering is used for extracting FECG from maternal abdominal ECG signals. The proposed method benefits from match filtering ability to detect fetal signal and QRS complex to detect weak QRS peaks. The combined method, has been applied successfully on different signal qualities, even for signals that their analysis was hard and complicated for other methods. This method is able to detect R-R intervals with high accuracy. It was proved that the complete ensemble empirical mode decomposition method provides a better frequency resolution of modes and also requires less iterations that leads to a considerably less computational cost than EMD and ensemble empirical mode decomposition and can reconstruct the FECG completely from the calculated modes. We believe that this method opens a new field in non-invasive maternal abdominal signal processing so the FECG signal could be extracted with high speed and accuracy.

Quantification of 1 H-MRS signals based on sparse metabolite profiles in the time-frequency domain.

MRS is an analytical approach used for both quantitative and qualitative analysis of human body metabolites. The accurate and robust quantification capability of proton MRS (1 H-MRS) enables the accurate estimation of living tissue metabolite concentrations. However, such methods can be efficiently employed for quantification of metabolite concentrations only if the overlapping nature of metabolites, existing static field inhomogeneity and low signal-to-noise ratio (SNR) are taken into consideration. Representation of 1 H-MRS signals in the time-frequency domain enables us to handle the baseline and noise better. This is possible because the MRS signal of each metabolite is sparsely represented, with only a few peaks, in the frequency domain, but still along with specific time-domain features such as distinct decay constant associated with T2 relaxation rate. The baseline, however, has a smooth behavior in the frequency domain. In this study, we proposed a quantification method using continuous wavelet transformation of 1 H-MRS signals in combination with sparse representation of features in the time-frequency domain. Estimation of the sparse representations of MR spectra is performed according to the dictionaries constructed from metabolite profiles. Results on simulated and phantom data show that the proposed method is able to quantify the concentration of metabolites in 1 H-MRS signals with high accuracy and robustness. This is achieved for both low SNR (5 dB) and low signal-to-baseline ratio (-5 dB) regimes.