Real-time FPGA-based implementation of the AKAZE algorithm with nonlinear scale space generation using image partitioning.

The first step in a scale invariant image matching system is scale space generation. Nonlinear scale space generation algorithms such as AKAZE, reduce noise and distortion in different scales while retaining the borders and key-points of the image. An FPGA-based hardware architecture for AKAZE nonlinear scale space generation is proposed to speed up this algorithm for real-time applications. The three contributions of this work are (1) mapping the two passes of the AKAZE algorithm onto a hardware architecture that realizes parallel processing of multiple sections, (2) multi-scale line buffers which can be used for different scales, and (3) a time-sharing mechanism in the memory management unit to process multiple sections of the image in parallel. We propose a time-sharing mechanism for memory management to prevent artifacts as a result of separating the process of image partitioning. We also use approximations in the algorithm to make hardware implementation more efficient while maintaining the repeatability of the detection. A frame rate of 304 frames per second for a 1280 × 768 image resolution is achieved which is favorably faster in comparison with other work.

A Novel Hardware-Software Co-Design and Implementation of the HOG Algorithm.

The histogram of oriented gradients is a commonly used feature extraction algorithm in many applications. Hardware acceleration can boost the speed of this algorithm due to its large number of computations. We propose a hardware-software co-design of the histogram of oriented gradients and the subsequent support vector machine classifier, which can be used to process data from digital image sensors. Our main focus is to minimize the resource usage of the algorithm while maintaining its accuracy and speed. This design and implementation make four contributions. First, we allocate the computationally expensive steps of the algorithm, including gradient calculation, magnitude computation, bin assignment, normalization and classification, to hardware, and the less complex windowing step to software. Second, we introduce a logarithm-based bin assignment. Third, we use parallel computation and a time-sharing protocol to create a histogram in order to achieve the processing of one pixel per clock cycle after the initialization (setup time) of the pipeline, and produce valid results at each clock cycle afterwards. Finally, we use a simplified block normalization logic to reduce hardware resource usage while maintaining accuracy. Our design attains a frame rate of 115 frames per second on a Xilinx® Kintex® Ultrascale™ FPGA while using less hardware resources, and only losing accuracy marginally, in comparison with other existing work.

Elastic registration of prostate MR images based on estimation of deformation states.

Magnetic resonance imaging (MRI) is being used increasingly for image-guided targeted biopsy and focal therapy of prostate cancer. In this paper, a combined rigid and deformable registration technique is proposed to register pre-treatment diagnostic 3T magnetic resonance (MR) images of the prostate, with the identified target tumor(s), to intra-treatment 1.5T MR images. The pre-treatment T2-weighted MR images were acquired with patients in a supine position using an endorectal coil in a 3T scanner, while the intra-treatment T2-weighted MR images were acquired in a 1.5T scanner before insertion of the needle with patients in the semi-lithotomy position. Both the rigid and deformable registration algorithms employ an intensity-based distance metric defined based on the modality independent neighborhood descriptors (MIND) between images. The optimization routine for estimating the rigid transformation parameters is initialized using four pairs of manually selected approximate corresponding points on the boundaries of the prostate. In this paper, the problem of deformable image registration is approached from the perspective of state estimation for dynamical systems. The registration algorithm employs a rather generic dynamic linear elastic model of the tissue deformation discretized by the finite element method (FEM). We use the model in a classical state estimation framework to estimate the deformation of the prostate based on the distance metric between pre- and intra-treatment images. Our deformable registration results using 17 sets of prostate MR images showed that the proposed method yielded a target registration error (TRE) of 1.87 ± 0.94 mm,2.03 ± 0.94 mm, and 1.70 ± 0.93 mm for the whole gland (WG), central gland (CG), and peripheral zone (PZ), respectively, using 76 manually-identified fiducial points. This was an improvement over the 2.67 ± 1.31 mm, 2.95 ± 1.43 mm, and 2.34 ± 1.11 mm, respectively for the WG, CG, and PZ after rigid registration alone. Dice similarity coefficients (DSC) in the WG, CG and PZ were 88.2 ± 5.3, 85.6 ± 7.6 and 68.7 ± 6.9 percent, respectively. Furthermore, the mean absolute distances (MAD) between surfaces was 1.26 ± 0.56 mm and 1.27 ± 0.55 mm in the WG and CG, after deformable registration. These results indicate that the proposed registration technique has sufficient accuracy for localizing prostate tumors in MRI-guided targeted biopsy or focal therapy of clinically localized prostate cancer.

Non-rigid registration of medical images based on estimation of deformation states.

A unified framework for automatic non-rigid 3D-3D and 3D-2D registration of medical images with static and dynamic deformations is proposed in this paper. The problem of non-rigid image registration is approached as a classical state estimation problem using a generic deformation model for the soft tissue. The registration technique employs a dynamic linear elastic continuum mechanics model of the tissue deformation, which is discretized using the finite element method. In the proposed method, the registration is achieved through a Kalman-like filtering process, which incorporates information from the deformation model and a vector of observation prediction errors computed from an intensity-based similarity/distance metric between images. With this formulation, single and multiple-modality, 3D-3D and 3D-2D image registration problems can all be treated within the same framework. The performance of the proposed registration technique was evaluated in a number of different registration scenarios. First, 3D magnetic resonance (MR) images of uncompressed and compressed breast tissue were co-registered. 3D MR images of the uncompressed breast tissue were also registered to a sequence of simulated 2D interventional MR images of the compressed breast. Finally, the registration algorithm was employed to dynamically track a target sub-volume inside the breast tissue during the process of the biopsy needle insertion based on registering pre-insertion 3D MR images to a sequence of real-time simulated 2D interventional MR images. Registration results indicate that the proposed method can be effectively employed for the registration of medical images in image-guided procedures, such as breast biopsy in which the tissue undergoes static and dynamic deformations.

Model-Based Deformable Registration of Preoperative 3D to Intraoperative Low-Resolution 3D and 2D Sequences of MR Images

We have developed an automatic model-based deformable registration method applicable to MR soft-tissue imaging. The registration algorithm uses a dynamic finite element (FE) continuum mechanics model of the tissue deformation to register its 3D preoperative images with intraoperative 1) 3D low-resolution or 2) 2D MR images. The registration is achieved through a filtering process that combines information from the deformation model and observation errors based on correlation ratio, mutual information or sum of square differences between images. Experimental results with a breast phantom show that the proposed method converges in few iterations in the presence of very large deformations, similar to those typically observed in breast biopsy applications.

Model-based 3D/2D deformable registration of MR images.

A method is proposed for automatic registration of 3D preoperative magnetic resonance images of deformable tissue to a sequence of its 2D intraoperative images. The algorithm employs a dynamic continuum mechanics model of the deformation and similarity (distance) measures such as correlation ratio, mutual information or sum of squared differences for registration. The registration is solely based on information present in the 3D preoperative and 2D intraoperative images and does not require fiducial markers, feature extraction or image segmentation. Results of experiments with a biopsy training breast phantom show that the proposed method can perform well in the presence of large deformations. This is particularly useful for clinical applications such as MR-based breast biopsy where large tissue deformations occur.