Detecting and quantifying spatial misalignment between longitudinal kilovoltage computed tomography (kVCT) scans of the head and neck by using convolutional neural networks (CNNs).

BACKGROUND: Adaptive radiotherapy (ART) aims to address anatomical modifications appearing during the treatment of patients by modifying the planning treatment according to the daily positioning image. Clinical implementation of ART relies on the quality of the deformable image registration (DIR) algorithms included in the ART workflow. To translate ART into clinical practice, automatic DIR assessment is needed. OBJECTIVE: This article aims to estimate spatial misalignment between two head and neck kilovoltage computed tomography (kVCT) images by using two convolutional neural networks (CNNs). METHODS: The first CNN quantifies misalignments between 0 mm and 15 mm and the second CNN detects and classifies misalignments into two classes (poor alignment and good alignment). Both networks take pairs of patches of 33x33x33 mm3 as inputs and use only the image intensity information. The training dataset was built by deforming kVCT images with basis splines (B-splines) to simulate DIR error maps. The test dataset was built using 2500 landmarks, consisting of hard and soft landmark tissues annotated by 6 clinicians at 10 locations. RESULTS: The quantification CNN reaches a mean error of 1.26 mm (± 1.75 mm) on the landmark set which, depending on the location, has annotation errors between 1 mm and 2 mm. The errors obtained for the quantification network fit the computed interoperator error. The classification network achieves an overall accuracy of 79.32%, and although the classification network overdetects poor alignments, it performs well (i.e., it achieves a rate of 90.4%) in detecting poor alignments when given one. CONCLUSION: The performances of the networks indicate the feasibility of using CNNs for an agnostic and generic approach to misalignment quantification and detection.

Image Processing Pitfalls in Vendor Adaptive Radiotherapy Software with Tomotherapy-like Systems: Feedback from Clinical Case Reports.

BACKGROUND: Adaptive radiotherapy (ART) has the potential to reduce the toxicities of radiotherapy and improve overall survival by considering variations in the patient's anatomy during the course of treatment. ART's first commercial solutions are now implemented in clinical radiotherapy departments. However, before they can be used safely with real patients, these solutions must be rigorously evaluated to precisely determine the limits of their use. METHODS: In this paper, we evaluated an offline ART vendor system in 50 patients treated on tomotherapy- like systems for six months. Illustrated by numerous examples of head and neck, thoracic and abdominopelvic localizations, two limitations of image processing used in the ART workflow have been highlighted: deformable image registration (DIR) accuracy and the way the limited field of view (FOV) is compensated. This feedback from clinical experience makes it possible to identify topics of image processing research with strong clinical interest. RESULTS: Current DIR method accuracy may be too weak for some clinical ART applications, and their improvement remains highly important, especially for multimodality registration. Improvements in contour propagation methods also remain crucial today. We showed that there is a need for the development of automatic DIR accuracy quantification methods to help streamline the ART process. Finally, the limited FOV of the onboard images may induce dose calculation errors, highlighting the need to develop new FOV extension methods. CONCLUSION: We have evaluated a vendor ART system, but some image processing pitfalls, such as DIR accuracy and the limited FOV of the onboard images, make its implementation into clinical practice difficult for the moment.

Survey on deep learning for radiotherapy.

More than 50% of cancer patients are treated with radiotherapy, either exclusively or in combination with other methods. The planning and delivery of radiotherapy treatment is a complex process, but can now be greatly facilitated by artificial intelligence technology. Deep learning is the fastest-growing field in artificial intelligence and has been successfully used in recent years in many domains, including medicine. In this article, we first explain the concept of deep learning, addressing it in the broader context of machine learning. The most common network architectures are presented, with a more specific focus on convolutional neural networks. We then present a review of the published works on deep learning methods that can be applied to radiotherapy, which are classified into seven categories related to the patient workflow, and can provide some insights of potential future applications. We have attempted to make this paper accessible to both radiotherapy and deep learning communities, and hope that it will inspire new collaborations between these two communities to develop dedicated radiotherapy applications.

Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network.

Stereotactic treatments are today the reference techniques for the irradiation of brain metastases in radiotherapy. The dose per fraction is very high, and delivered in small volumes (diameter <1 cm). As part of these treatments, effective detection and precise segmentation of lesions are imperative. Many methods based on deep-learning approaches have been developed for the automatic segmentation of gliomas, but very little for that of brain metastases. We adapted an existing 3D convolutional neural network (DeepMedic) to detect and segment brain metastases on MRI. At first, we sought to adapt the network parameters to brain metastases. We then explored the single or combined use of different MRI modalities, by evaluating network performance in terms of detection and segmentation. We also studied the interest of increasing the database with virtual patients or of using an additional database in which the active parts of the metastases are separated from the necrotic parts. Our results indicated that a deep network approach is promising for the detection and the segmentation of brain metastases on multimodal MRI.

Joint filtering estimation of Stokes vector images based on a nonlocal means approach.

Conventional estimation techniques of Stokes images from observed radiance images through different polarization filters suffer from noise contamination that hampers correct interpretation or even leads to unphysical estimated signatures. This paper presents an efficient restoration technique based on nonlocal means, permitting accurate estimation of smoothly variable polarization signatures in the Stokes image while preserving sharp transitions. The method is assessed on simulated data as well as on real images.

Double-pass self-spectral-calibration of a polarization state analyzer.

We present a new method that allows efficient spectral calibration for a polarization state analyzer. The procedure does not require any additional polarization optical element other than the polarization state analyzer itself. It uses a double-pass technique that can be achieved up to a very good precision. The method is illustrated using real measurements done at several wavelengths with a rotating wave plate polarization state analyzer. Alignment of axis as well as true retardation at a specific wavelength are easily obtained by a standard function fitting.

Optical high-precision three-dimensional vision-based quality control of manufactured parts by use of synthetic images and knowledge for image-data evaluation and interpretation.

Vision-based evaluation of industrial workpieces can make efficient use of knowledge-based approaches, in particular for quality control, inspection, and accurate-measurement tasks. A possible approach is to compare real images with conceptual (synthetic) images generated by use of standard computer-aided design models, which include tolerances and take the application-specific conditions into account (e.g., the measured-calibration data). Integrated in (industrial) real-life environments, our evaluation methods have been successfully applied to on-line inspection of manufactured parts including sculptured surfaces, using structured light techniques for the reconstruction of three-dimensional shapes. Accuracies in the range 15-50 microm are routinely achieved by use of either isolated images or spatially registered image sequences.