A machine learning framework for the quantification of experimental uveitis in murine OCT.

This paper presents methods for the detection and assessment of non-infectious uveitis, a leading cause of vision loss in working age adults. In the first part, we propose a classification model that can accurately predict the presence of uveitis and differentiate between different stages of the disease using optical coherence tomography (OCT) images. We utilize the Grad-CAM visualization technique to elucidate the decision-making process of the classifier and gain deeper insights into the results obtained. In the second part, we apply and compare three methods for the detection of detached particles in the retina that are indicative of uveitis. The first is a fully supervised detection method, the second is a marked point process (MPP) technique, and the third is a weakly supervised segmentation that produces per-pixel masks as output. The segmentation model is used as a backbone for a fully automated pipeline that can segment small particles of uveitis in two-dimensional (2-D) slices of the retina, reconstruct the volume, and produce centroids as points distribution in space. The number of particles in retinas is used to grade the disease, and point process analysis on centroids in three-dimensional (3-D) shows clustering patterns in the distribution of the particles on the retina.

Current Advances in Computational Lung Ultrasound Imaging: A Review.

In the field of biomedical imaging, ultrasonography has become common practice, and used as an important auxiliary diagnostic tool with unique advantages, such as being non-ionizing and often portable. This article reviews the state-of-the-art in medical ultrasound (US) image processing and in particular its applications in the examination of the lungs. First, we briefly introduce the basis of lung US (LUS) examination. We focus on (i) the characteristics of lung ultrasonography and (ii) its ability to detect a variety of diseases through the identification of various artifacts exhibiting on LUS images. We group medical US image computing methods into two categories: 1) model-based methods and 2) data-driven methods. We particularly discuss inverse problem-based methods exploited in US image despeckling, deconvolution, and line artifacts detection for the former, while we exemplify various works based on deep/machine learning (ML), which exploit various network architectures through supervised, weakly supervised, and unsupervised learning for the data-driven approaches.

Bright-field to fluorescence microscopy image translation for cell nuclei health quantification.

Microscopy is a widely used method in biological research to observe the morphology and structure of cells. Amongst the plethora of microscopy techniques, fluorescent labeling with dyes or antibodies is the most popular method for revealing specific cellular organelles. However, fluorescent labeling also introduces new challenges to cellular observation, as it increases the workload, and the process may result in nonspecific labeling. Recent advances in deep visual learning have shown that there are systematic relationships between fluorescent and bright-field images, thus facilitating image translation between the two. In this article, we propose the cross-attention conditional generative adversarial network (XAcGAN) model. It employs state-of-the-art GANs (GANs) to solve the image translation task. The model uses supervised learning and combines attention-based networks to explore spatial information during translation. In addition, we demonstrate the successful application of XAcGAN to infer the health state of translated nuclei from bright-field microscopy images. The results show that our approach achieves excellent performance both in terms of image translation and nuclei state inference.

Detection of Line Artifacts in Lung Ultrasound Images of COVID-19 Patients Via Nonconvex Regularization.

In this article, we present a novel method for line artifacts quantification in lung ultrasound (LUS) images of COVID-19 patients. We formulate this as a nonconvex regularization problem involving a sparsity-enforcing, Cauchy-based penalty function, and the inverse Radon transform. We employ a simple local maxima detection technique in the Radon transform domain, associated with known clinical definitions of line artifacts. Despite being nonconvex, the proposed technique is guaranteed to convergence through our proposed Cauchy proximal splitting (CPS) method, and accurately identifies both horizontal and vertical line artifacts in LUS images. To reduce the number of false and missed detection, our method includes a two-stage validation mechanism, which is performed in both Radon and image domains. We evaluate the performance of the proposed method in comparison to the current state-of-the-art B-line identification method, and show a considerable performance gain with 87% correctly detected B-lines in LUS images of nine COVID-19 patients.

Classification of Alzheimer's Disease in MRI based on Dictionary Learning and Heavy Tailed Modelling.

Diagnosis of brain diseases is considered one of the most challenging medical tasks to perform, even for medical experts who rely on high-resolution anatomical images to identify signs of abnormalities by visual inspection. However, new computational tools which assist to automate this diagnosis have the potential to significantly improve the speed and accuracy of this process. This work presents a model to aid in the task of classification of structural Magnetic Resonance Imaging scans. The classification is performed using a Support Vector Machine, whilst the features to analyze belong to a dictionary space. Such space was mainly built from a dictionary learning perspective, although a predefined one was also assessed. The results indicate that features learnt from the data of interest lead to improved classification performance. The proposed framework was tested on the ADNI dataset stage I.

Super-Resolution OCT Using Sparse Representations and Heavy-Tailed Models.

This paper introduces a new approach to single-image super-resolution in Optical Coherence Tomography (OCT) images. Retinal OCT images can be used to diagnose various diseases, not only peculiar to the eye, but also some systemic diseases. Nevertheless, as with any imaging modality, the acquired images suffer from degradation due to various causes. To overcome this and enhance image quality, Super-Resolution (SR) techniques are widely used. This work explores a convex regularization approach based on a multivariate generalization of the minimax-concave (GMC) scheme in a forward-backward splitting (FBS) scheme. Based on the assumption that sparse representations of OCT images are heavy-tailed, an α-stable dictionary is employed. This approach is implemented with overlapping and non-overlapping patches. Since the Point Spread Function (PSF) of the images used is generally unknown, it is estimated using a method originally proposed for ultrasound images. The algorithm is tested on OCT images of murine eyes. The results show that the proposed convex regularization method provides results that are competitive with the state-of-the-art. Indeed, significant deblurring and quality enhancement are achieved using the proposed algorithm and in most cases it provides the best results, both objectively and subjectively.

Super-Resolution in Optical Coherence Tomography.

Optical coherence tomography (OCT) is an essential medical imaging tool for retinal disease diagnosis. Nevertheless, as with all optical imaging techniques, image degradation is a very common phenomenon, affecting the quality of the images. In this paper, we address issues related to the resolution of OCT images and propose solutions based on solving inverse problems. A cost function for deconvolution and super-resolution is formulated and the alternating direction method of multiplier (ADMM) and forward-backward splitting (FBS) algorithms are then employed for its minimisation. On the one hand, the standard Ll norm regularisation with soft thresholding is compared with a total variation (TV) regularisation within an ADMM scheme. On the other hand, nonconvex regularisation is also considered via a multivariate generalisation of the minimax-concave scheme in FBS. In the latter case, the regularisation function is judiciously chosen in order to preserve the overall convexity of the cost function. To be able to evaluate our algorithms qualitatively, a number of standard images are initially used. Then, we also assess our algorithms subjectively by applying them to real OCT images of the human eye. Given that the point spread function (PSF) of OCT images is generally unknown, we also propose ways of estimating it in the deconvolution component of our methods. Our results show that the ADMM scheme with soft thresholding achieves the best performance in terms of enhancing the overall quality of OCT images.

Approximate Message Passing Reconstruction of Quantitative Acoustic Microscopy Images.

A novel framework for compressive sensing (CS) data acquisition and reconstruction in quantitative acoustic microscopy (QAM) is presented. Three different CS patterns, adapted to the specifics of QAM systems, were investigated as an alternative to the current raster-scanning approach. They consist of diagonal sampling, a row random, and a spiral scanning pattern and can all significantly reduce both the acquisition time and the amount of sampled data. For subsequent image reconstruction, we design and implement an innovative technique, whereby a recently proposed approximate message passing method is adapted to account for the underlying data statistics. A Cauchy maximum a posteriori image denoising algorithm is thus employed to account for the non-Gaussianity of QAM wavelet coefficients. The proposed methods were tested retrospectively on experimental data acquired with a 250- or 500-MHz QAM system. The experimental data were obtained from a human lymph node sample (250 MHz) and human cornea (500 MHz). Reconstruction results showed that the best performance is obtained using a spiral sensing pattern combined with the Cauchy denoiser in the wavelet domain. The spiral sensing matrix reduced the number of spatial samples by a factor of 2 and led to an excellent peak signal-to-noise ratio of 43.21 dB when reconstructing QAM speed-of-sound images of a human lymph node. These results demonstrate that the CS approach could significantly improve scanning time, while reducing costs of future QAM systems.

Line Detection as an Inverse Problem: Application to Lung Ultrasound Imaging.

This paper presents a novel method for line restoration in speckle images. We address this as a sparse estimation problem using both convex and non-convex optimization techniques based on the Radon transform and sparsity regularization. This breaks into subproblems, which are solved using the alternating direction method of multipliers, thereby achieving line detection and deconvolution simultaneously. We include an additional deblurring step in the Radon domain via a total variation blind deconvolution to enhance line visualization and to improve line recognition. We evaluate our approach on a real clinical application: the identification of B-lines in lung ultrasound images. Thus, an automatic B-line identification method is proposed, using a simple local maxima technique in the Radon transform domain, associated with known clinical definitions of line artefacts. Using all initially detected lines as a starting point, our approach then differentiates between B-lines and other lines of no clinical significance, including Z-lines and A-lines. We evaluated our techniques using as ground truth lines identified visually by clinical experts. The proposed approach achieves the best B-line detection performance as measured by the F score when a non-convex [Formula: see text] regularization is employed for both line detection and deconvolution. The F scores as well as the receiver operating characteristic (ROC) curves show that the proposed approach outperforms the state-of-the-art methods with improvements in B-line detection performance of 54%, 40%, and 33% for [Formula: see text], [Formula: see text], and [Formula: see text], respectively, and of 24% based on ROC curve evaluations.

Contrast Sensitivity of the Wavelet, Dual Tree Complex Wavelet, Curvelet, and Steerable Pyramid Transforms.

Accurate estimation of the contrast sensitivity of the human visual system is crucial for perceptually based image processing in applications such as compression, fusion and denoising. Conventional contrast sensitivity functions (CSFs) have been obtained using fixed-sized Gabor functions. However, the basis functions of multiresolution decompositions such as wavelets often resemble Gabor functions but are of variable size and shape. Therefore to use the conventional CSFs in such cases is not appropriate. We have therefore conducted a set of psychophysical tests in order to obtain the CSF for a range of multiresolution transforms: the discrete wavelet transform, the steerable pyramid, the dual-tree complex wavelet transform, and the curvelet transform. These measures were obtained using contrast variation of each transforms' basis functions in a 2AFC experiment combined with an adapted version of the QUEST psychometric function method. The results enable future image processing applications that exploit these transforms such as signal fusion, superresolution processing, denoising and motion estimation, to be perceptually optimized in a principled fashion. The results are compared with an existing vision model (HDR-VDP2) and are used to show quantitative improvements within a denoising application compared with using conventional CSF values.

Retracing in correlative light electron microscopy: where is my object of interest?

Correlative light electron microscopy (CLEM) combines the strengths of light and electron microscopy in a single experiment. There are many ways to perform a CLEM experiment and a variety of microscopy modalities can be combined either on separate instruments or as completely integrated solutions. In general, however, a CLEM experiment can be divided into three parts: probes, processing, and analysis. Most of the existing technologies are focussed around the development and use of probes or describe processing methodologies that explain or circumvent some of the compromises that need to be made when performing both light and electron microscopy on the same sample. So far, relatively little attention has been paid to the analysis part of CLEM experiments. Although it is an essential part of each CLEM experiment, it is usually a cumbersome manual process. Here, we briefly discuss each of the three above-mentioned steps, with a focus on the analysis part. We will also introduce an automated registration algorithm that can be applied to the analysis stage to enable the accurate registration of LM and EM images. This facilitates tracing back the right cell/object seen in the light microscope in the EM.

Adaptive-weighted bilateral filtering and other pre-processing techniques for optical coherence tomography.

This paper presents novel pre-processing image enhancement algorithms for retinal optical coherence tomography (OCT). These images contain a large amount of speckle causing them to be grainy and of very low contrast. To make these images valuable for clinical interpretation, we propose a novel method to remove speckle, while preserving useful information contained in each retinal layer. The process starts with multi-scale despeckling based on a dual-tree complex wavelet transform (DT-CWT). We further enhance the OCT image through a smoothing process that uses a novel adaptive-weighted bilateral filter (AWBF). This offers the desirable property of preserving texture within the OCT image layers. The enhanced OCT image is then segmented to extract inner retinal layers that contain useful information for eye research. Our layer segmentation technique is also performed in the DT-CWT domain. Finally we describe an OCT/fundus image registration algorithm which is helpful when two modalities are used together for diagnosis and for information fusion.

A novel framework for segmentation of secretory granules in electron micrographs.

It is still a standard practice for biologists to manually analyze transmission electron microscopy images. This is not only time consuming but also not reproducible and prone to induce subjective bias. For large-scale studies of insulin granules inside beta cells of the islet of Langerhans, an automated method for analysis is essential. Due to the complex structure of the images, standard microscopy segmentation techniques cannot be applied. We present a new approach to segment and measure transmission electron microscopy images of insulin granule cores and membranes from beta cells of rat islets of Langerhans. The algorithm is separated into two broad components, core segmentation and membrane segmentation. Core segmentation proceeds through three steps: pre-segmentation using a novel level-set active contour, morphological cleaning and a refining segmentation on each granule using a novel dual level-set active contour. Membrane segmentation is achieved in four steps: morphological cleaning, membrane sampling and scaling, vector field convolution for gap filling and membrane verification using a novel convergence filter. We show results from our algorithm alongside popular microscopy segmentation methods; the advantages of our method are demonstrated. Our algorithm is validated by comparing automated results to a manually defined ground truth. When the number of granules detected is compared to the number of granules in the ground truth a precision of 91% and recall of 87% is observed. The average granule areas differ by 13.35% and 6.08% for core and membranes respectively, when compared to the average areas of the ground truth. These results compare favorably to previously published data.

Gaze location prediction for broadcast football video.

The sensitivity of the human visual system decreases dramatically with increasing distance from the fixation location in a video frame. Accurate prediction of a viewer's gaze location has the potential to improve bit allocation, rate control, error resilience, and quality evaluation in video compression. Commercially, delivery of football video content is of great interest because of the very high number of consumers. In this paper, we propose a gaze location prediction system for high definition broadcast football video. The proposed system uses knowledge about the context, extracted through analysis of a gaze tracking study that we performed, to build a suitable prior map. We further classify the complex context into different categories through shot classification thus allowing our model to prelearn the task pertinence of each object category and build the prior map automatically. We thus avoid the limitation of assigning the viewers a specific task, allowing our gaze prediction system to work under free-viewing conditions. Bayesian integration of bottom-up features and top-down priors is finally applied to predict the gaze locations. Results show that the prediction performance of the proposed model is better than that of other top-down models that we adapted to this context.

Atmospheric turbulence mitigation using complex wavelet-based fusion.

Restoring a scene distorted by atmospheric turbulence is a challenging problem in video surveillance. The effect, caused by random, spatially varying, perturbations, makes a model-based solution difficult and in most cases, impractical. In this paper, we propose a novel method for mitigating the effects of atmospheric distortion on observed images, particularly airborne turbulence which can severely degrade a region of interest (ROI). In order to extract accurate detail about objects behind the distorting layer, a simple and efficient frame selection method is proposed to select informative ROIs only from good-quality frames. The ROIs in each frame are then registered to further reduce offsets and distortions. We solve the space-varying distortion problem using region-level fusion based on the dual tree complex wavelet transform. Finally, contrast enhancement is applied. We further propose a learning-based metric specifically for image quality assessment in the presence of atmospheric distortion. This is capable of estimating quality in both full- and no-reference scenarios. The proposed method is shown to significantly outperform existing methods, providing enhanced situational awareness in a range of surveillance scenarios.

Active contour based segmentation for insulin granule cores in electron micrographs of beta islet cells.

Transmission electron microscopy images of beta islet cells contain many complex structures, making it difficult to accurately segment insulin granule cores. Quantification of sub cellular structures will allow biologists to better understand cellular mechanics. Two novel, level set active contour models are presented in this paper. The first utilizes a shape regularizer to reduce oversegmentation. The second contribution is a dual active contour, which achieves accurate core segmentations. The segmentation algorithm proceeds through three stages: an initial rough segmentation using the first contribution, cleaning using morphological techniques and a refining step using the proposed dual active contour. Our method is validated on a set of manually defined ground truths.

Retinal image registration based on multiscale products and optic disc detection.

This paper presents a new approach to segmentation-driven retinal image registration. The proposed algorithm aims to help physicians to detect changes that occur in the blood vasculature due to various diseases. The proposed approach uses multiscale products, which augment the difference between blood vessels and the rest of the retina. The result of scale multiplication is then iteratively thresholded in order to obtain a binary map of vessels inside the retina. For the registration part, the centre of the optic disc is detected and used as control point. Having determined both the position of the blood vessels and the centre of the optic disc, translational and rotational differences between the images can be eliminated and registration can be achieved. The centroid of the optic disc is used as the center of rotation. The final registration is then achieved by searching the best match between the two images using a XOR operation.

Medical image fusion using the convolution of Meridian distributions.

The aim of this paper is to introduce a novel non-Gaussian statistical model-based approach for medical image fusion based on the Meridian distribution. The paper also includes a new approach to estimate the parameters of generalized Cauchy distribution. The input images are first decomposed using the Dual-Tree Complex Wavelet Transform (DT-CWT) with the subband coefficients modelled as Meridian random variables. Then, the convolution of Meridian distributions is applied as a probabilistic prior to model the fused coefficients, and the weights used to combine the source images are optimised via Maximum Likelihood (ML) estimation. The superior performance of the proposed method is demonstrated using medical images.

Temporal registration for low-quality retinal images of the murine eye.

This paper presents an investigation into different approaches for segmentation-driven retinal image registration. This constitutes an intermediate step towards detecting changes occurring in the topography of blood vessels, which are caused by disease progression. A temporal dataset of retinal images was collected from small animals (i.e. mice). The perceived low quality of the dataset employed favoured the implementation of a simple registration approach that can cope with rotation, translation and scaling, in the presence of major vascular dissimilarities, distortions, noise, and blurring effects. The proposed approach uses a single control point, i.e. the centroid of the optic disc, and achieves accurate registration by matching points in the pair of input images using mean squared error calculation. A number of alternative, more sophisticated methods have been explored alongside the proposed one. While these other methods could prove valuable and perform reasonably well when applied on good quality images, they generally fail when using the dataset at hand.

Compressive sensing for ultrasound RF echoes using a-Stable Distributions.

This paper introduces a novel framework for compressive sensing of biomedical ultrasonic signals based on modelling data with stable distributions. We propose an approach to ℓ(p) norm minimisation that employs the iteratively reweighted least squares (IRLS) algorithm but in which the parameter p is judiciously chosen by relating it to the characteristic exponent of the underlying alpha-stable distributed data. Our results show that the proposed algorithm, which we prefer to call S ± S-IRLS, outperforms previously proposed ℓ(1) minimisation algorithms, such as basis pursuit or orthogonal matching pursuit, both visually and in terms of PSNR.