Connections Between Numerical Algorithms for PDEs and Neural Networks.

We investigate numerous structural connections between numerical algorithms for partial differential equations (PDEs) and neural architectures. Our goal is to transfer the rich set of mathematical foundations from the world of PDEs to neural networks. Besides structural insights, we provide concrete examples and experimental evaluations of the resulting architectures. Using the example of generalised nonlinear diffusion in 1D, we consider explicit schemes, acceleration strategies thereof, implicit schemes, and multigrid approaches. We connect these concepts to residual networks, recurrent neural networks, and U-net architectures. Our findings inspire a symmetric residual network design with provable stability guarantees and justify the effectiveness of skip connections in neural networks from a numerical perspective. Moreover, we present U-net architectures that implement multigrid techniques for learning efficient solutions of partial differential equation models, and motivate uncommon design choices such as trainable nonmonotone activation functions. Experimental evaluations show that the proposed architectures save half of the trainable parameters and can thus outperform standard ones with the same model complexity. Our considerations serve as a basis for explaining the success of popular neural architectures and provide a blueprint for developing new mathematically well-founded neural building blocks.

Designing rotationally invariant neural networks from PDEs and variational methods.

Partial differential equation models and their associated variational energy formulations are often rotationally invariant by design. This ensures that a rotation of the input results in a corresponding rotation of the output, which is desirable in applications such as image analysis. Convolutional neural networks (CNNs) do not share this property, and existing remedies are often complex. The goal of our paper is to investigate how diffusion and variational models achieve rotation invariance and transfer these ideas to neural networks. As a core novelty, we propose activation functions which couple network channels by combining information from several oriented filters. This guarantees rotation invariance within the basic building blocks of the networks while still allowing for directional filtering. The resulting neural architectures are inherently rotationally invariant. With only a few small filters, they can achieve the same invariance as existing techniques which require a fine-grained sampling of orientations. Our findings help to translate diffusion and variational models into mathematically well-founded network architectures and provide novel concepts for model-based CNN design.

Robustness of brain tumor segmentation.

Purpose: The segmentation of brain tumors is one of the most active areas of medical image analysis. While current methods perform superhuman on benchmark data sets, their applicability in daily clinical practice has not been evaluated. In this work, we investigate the generalization behavior of deep neural networks in this scenario. Approach: We evaluate the performance of three state-of-the-art methods, a basic U-Net architecture, and a cascadic Mumford-Shah approach. We also propose two simple modifications (which do not change the topology) to improve generalization performance. Results: In these experiments, we show that a well-trained U-network shows the best generalization behavior and is sufficient to solve this segmentation problem. We illustrate why extensions of this model in a realistic scenario can be not only pointless but even harmful. Conclusions: We conclude from these experiments that the generalization performance of deep neural networks is severely limited in medical image analysis especially in the area of brain tumor segmentation. In our opinion, current topologies are optimized for the actual benchmark data set but are not directly applicable in daily clinical practice.

Benchmarking Wilms' tumor in multisequence MRI data: why does current clinical practice fail? Which popular segmentation algorithms perform well?

Wilms' tumor is one of the most frequent malignant solid tumors in childhood. Accurate segmentation of tumor tissue is a key step during therapy and treatment planning. Since it is difficult to obtain a comprehensive set of tumor data of children, there is no benchmark so far allowing evaluation of the quality of human or computer-based segmentations. The contributions in our paper are threefold: (i) we present the first heterogeneous Wilms' tumor benchmark data set. It contains multisequence MRI data sets before and after chemotherapy, along with ground truth annotation, approximated based on the consensus of five human experts. (ii) We analyze human expert annotations and interrater variability, finding that the current clinical practice of determining tumor volume is inaccurate and that manual annotations after chemotherapy may differ substantially. (iii) We evaluate six computer-based segmentation methods, ranging from classical approaches to recent deep-learning techniques. We show that the best ones offer a quality comparable to human expert annotations.

Turning Diffusion-Based Image Colorization Into Efficient Color Compression.

The work of Levin et al. (2004) popularized stroke-based methods that add color to gray value images according to a small amount of user-specified color samples. Even though such reconstructions from sparse data suggest a possible use in compression, only few attempts were made so far in this direction. Diffusion-based compression methods pursue a similar idea: they store only few image pixels and inpaint the missing regions. Despite this close relation and a lack of diffusion-based color codecs, colorization ideas were so far only integrated into transform-based approaches such as JPEG. We address this missing link with two contributions. First, we show the relation between the discrete colorization of Levin et al. and continuous diffusion-based inpainting in the YCbCr color space. It decomposes the image into a luma (brightness) channel and two chroma (color) channels. Our luma-guided diffusion framework steers the diffusion inpainting in the chroma channels according to the structure in the luma channel. We show that making the luma-guided colorization anisotropic outperforms the method of Levin et al. significantly. Second, we propose a new luma preference codec that invests a large fraction of the bit budget into an accurate representation of the luma channel. This allows a high-quality reconstruction of color data with our colorization technique. Simultaneously, we exploit the fact that the human visual system is more sensitive to structural than to color information. Our experiments demonstrate that our new codec outperforms the state of the art in diffusion-based image compression and is competitive to transform-based codecs.

On improving the efficiency of tensor voting.

This paper proposes two alternative formulations to reduce the high computational complexity of tensor voting, a robust perceptual grouping technique used to extract salient information from noisy data. The first scheme consists of numerical approximations of the votes, which have been derived from an in-depth analysis of the plate and ball voting processes. The second scheme simplifies the formulation while keeping the same perceptual meaning of the original tensor voting: The stick tensor voting and the stick component of the plate tensor voting must reinforce surfaceness, the plate components of both the plate and ball tensor voting must boost curveness, whereas junctionness must be strengthened by the ball component of the ball tensor voting. Two new parameters have been proposed for the second formulation in order to control the potentially conflictive influence of the stick component of the plate vote and the ball component of the ball vote. Results show that the proposed formulations can be used in applications where efficiency is an issue since they have a complexity of order O(1). Moreover, the second proposed formulation has been shown to be more appropriate than the original tensor voting for estimating saliencies by appropriately setting the two new parameters.

Novel immunogenic antigens increase classification accuracy in meningioma to 93.84%.

There is growing evidence that simultaneous analysis of multiple autoantibody reactions can be utilized for diagnosis of neoplasms. Using a set of 57 meningioma-associated antigens, we recently separated meningioma patients from individuals without known disease with an accuracy of 90.3%. Here, we ask whether a largely increased set of immunogenic antigens can further improve this discrimination. We used an array with 1,827 human recombinant clones and measured reactivity of serum autoantibodies against the clones by a novel automated image analysis procedure. We were able to separate meningioma sera from sera of healthy controls with a specificity of 95.62%, a sensitivity of 91.83% and an accuracy of 93.84%. Of the analyzed clones, 23 in-frame clones were highly informative for the classification of meningioma vs. normal sera as shown by their AUC values. These results demonstrate that the accuracy of a serum-based diagnostic can be readily and considerably improved by screening extended sets of proteins.

Improving seroreactivity-based detection of glioma.

Seroreactivity profiling emerges as valuable technique for minimal invasive cancer detection. Recently, we provided first evidence for the applicability of serum profiling of glioma using a limited number of immunogenic antigens. Here, we screened 57 glioma and 60 healthy sera for autoantibodies against 1827 Escherichia coli expressed clones, including 509 in-frame peptide sequences. By a linear support vector machine approach, we calculated mean specificity, sensitivity, and accuracy of 100 repetitive classifications. We were able to differentiate glioma sera from sera of the healthy controls with a specificity of 90.28%, a sensitivity of 87.31% and an accuracy of 88.84%. We were also able to differentiate World Health Organization grade IV glioma sera from healthy sera with a specificity of 98.45%, a sensitivity of 80.93%, and an accuracy of 92.88%. To rank the antigens according to their information content, we computed the area under the receiver operator characteristic curve value for each clone. Altogether, we found 46 immunogenic clones including 16 in-frame clones that were informative for the classification of glioma sera versus healthy sera. For the separation of glioblastoma versus healthy sera, we found 91 informative clones including 26 in-frame clones. The best-suited in-frame clone for the classification glioma sera versus healthy sera corresponded to the vimentin gene (VIM) that was previously associated with glioma. In the future, autoantibody signatures in glioma not only may prove useful for diagnosis but also offer the prospect for a personalized immune-based therapy.

Moment invariants as shape recognition technique for comparing protein binding sites.

MOTIVATION: An approach for identifying similarities of protein-protein binding sites is presented. The geometric shape of a binding site is described by computing a feature vector based on moment invariants. In order to search for similarities, feature vectors of binding sites are compared. Similar feature vectors indicate binding sites with similar shapes. RESULTS: The approach is validated on a representative set of protein-protein binding sites, extracted from the SCOPPI database. When querying binding sites from a representative set, we search for known similarities among 2819 binding sites. A median area under the ROC curve of 0.98 is observed. For half of the queries, a similar binding site is identified among the first two of 2819 when sorting all binding sites according the proposed similarity measure. Typical examples identified by this method are analyzed and discussed. The nitrogenase iron protein-like SCOP family is clustered hierarchically according to the proposed similarity measure as a case study. AVAILABILITY: Python code is available on request from the authors.

Fully automated segmentation and morphometrical analysis of muscle fiber images.

BACKGROUND: Measurement of muscle fiber size and determination of size distribution is important in the assessment of neuromuscular disease. Fiber size estimation by simple inspection is inaccurate and subjective. Manual segmentation and measurement are time-consuming and tedious. We therefore propose an automated image analysis method for objective, reproducible, and time-saving measurement of muscle fibers in routinely hematoxylin-eosin stained cryostat sections. METHODS: The proposed segmentation technique makes use of recent advances in level set based segmentation, where classical edge based active contours are extended by region based cues, such as color and texture. Segmentation and measurement are performed fully automatically. Multiple morphometric parameters, i.e., cross sectional area, lesser diameter, and perimeter are assessed in a single pass. The performance of the computed method was compared to results obtained by manual measurement by experts. RESULTS: The correct classification rate of the computed method was high (98%). Segmentation and measurement results obtained manually or automatically did not reveal any significant differences. CONCLUSIONS: The presented region based active contour approach has been proven to accurately segment and measure muscle fibers. Complete automation minimizes user interaction, thus, batch processing, as well as objective and reproducible muscle fiber morphometry are provided.

Level set segmentation with multiple regions.

The popularity of level sets for segmentation is mainly based on the sound and convenient treatment of regions and their boundaries. Unfortunately, this convenience is so far not known from level set methods when applied to images with more than two regions. This communication introduces a comparatively simple way how to extend active contours to multiple regions keeping the familiar quality of the two-phase case. We further suggest a strategy to determine the optimum number of regions as well as initializations for the contours.

Domain decomposition for variational optical-flow computation.

We present an approach to parallel variational optical-flow computation by using an arbitrary partition of the image plane and iteratively solving related local variational problems associated with each subdomain. The approach is particularly suited for implementations on PC clusters because interprocess communication is minimized by restricting the exchange of data to a lower dimensional interface. Our mathematical formulation supports various generalizations to linear/nonlinear convex variational approaches, three-dimensional image sequences, spatiotemporal regularization, and unstructured geometries and triangulations. Results concerning the effects of interface preconditioning, as well as runtime and communication volume measurements on a PC cluster, are presented. Our approach provides a major step toward real-time two-dimensional image processing using off-the-shelf PC hardware and facilitates the efficient application of variational approaches to large-scale image processing problems.

Variational optical flow computation in real time.

This paper investigates the usefulness of bidirectional multigrid methods for variational optical flow computations. Although these numerical schemes are among the fastest methods for solving equation systems, they are rarely applied in the field of computer vision. We demonstrate how to employ those numerical methods for the treatment of variational optical flow formulations and show that the efficiency of this approach even allows for real-time performance on standard PCs. As a representative for variational optic flow methods, we consider the recently introduced combined local-global method. It can be considered as a noise-robust generalization of the Horn and Schunck technique. We present a decoupled, as well as a coupled, version of the classical Gauss-Seidel solver, and we develop several multgrid implementations based on a discretization coarse grid approximation. In contrast, with standard bidirectional multigrid algorithms, we take advantage of intergrid transfer operators that allow for nondyadic grid hierarchies. As a consequence, no restrictions concerning the image size or the number of traversed levels have to be imposed. In the experimental section, we juxtapose the developed multigrid schemes and demonstrate their superior performance when compared to unidirectional multgrid methods and nonhierachical solvers. For the well-known 316 x 252 Yosemite sequence, we succeeded in computing the complete set of dense flow fields in three quarters of a second on a 3.06-GHz Pentium4 PC. This corresponds to a frame rate of 18 flow fields per second which outperforms the widely-used Gauss-Seidel method by almost three orders of magnitude.

Diffusion-enhanced visualization and quantification of vascular anomalies in three-dimensional rotational angiography: results of an in-vitro evaluation.

Three-dimensional rotational angiography (3DRA) is a new and promising technique for obtaining high-resolution isotropic 3D images of vascular structures. However, due to the relatively high noise level and the presence of other background structures in clinical 3DRA images, noise reduction is inevitable. In this paper, we evaluate a number of linear and nonlinear diffusion techniques for this purpose. Specifically, we analyze the effects of these techniques on the threshold-based visualization and quantification of vascular anomalies in 3DRA images. The results of in-vitro experiments indicate that edge-enhancing anisotropic diffusion filtering is most suitable: the increase in the user-dependency of visualizations and quantifications is considerably less with this technique compared to linear filtering techniques, and it is better at reducing noise near edges than isotropic nonlinear diffusion. However, in view of the memory and computation-time requirements of this technique, the latter scheme may be considered a useful alternative.