CIVIT dataset: Integral microscopy with Fourier plane recording.

This article describes a dataset of synthetic images representing biological scenery as captured by a Fourier Lightfield Microscope (FLMic). It includes 22,416 images related to eight scenes composed of 3D models of objects typical for biological samples, such as red blood cells and bacteria, and categorized into Cells and Filaments groups. For each scene, two types of image data structures are provided: 51 × 51 Elemental Images (EIs) representing Densely Sampled Light Fields (DSLF) and 201 images composing Z-Scans of the scenes. Auxiliary data also includes information about camera intrinsic and extrinsic calibration parameters, object descriptions, and MATLAB scripts for camera pose compensation. The images have been generated using Blender. The dataset can be used to develop and assess methods for volumetric reconstruction from Light Field (LF) images captured by a FLMic.

Optical modelling of an accommodative light field display system and prediction of human eye responses.

The spatio-angular resolution of a light field (LF) display is a crucial factor for delivering adequate spatial image quality and eliciting an accommodation response. Previous studies have modelled retinal image formation with an LF display and evaluated whether accommodation would be evoked correctly. The models were mostly based on ray-tracing and a schematic eye model, which pose computational complexity and inaccurately represent the human eye population's behaviour. We propose an efficient wave-optics-based framework to model the human eye and a general LF display. With the model, we simulated the retinal point spread function (PSF) of a point rendered by an LF display at various depths to characterise the retinal image quality. Additionally, accommodation responses to the rendered point were estimated by computing the visual Strehl ratio based on the optical transfer function (VSOTF) from the PSFs. We assumed an ideal LF display that had an infinite spatial resolution and was free from optical aberrations in the simulation. We tested points rendered at 0-4 dioptres of depths having angular resolutions of up to 4x4 viewpoints within a pupil. The simulation predicted small and constant accommodation errors, which contradict the findings of previous studies. An evaluation of the optical resolution on the retina suggested a trade-off between the maximum achievable resolution and the depth range of a rendered point where in-focus resolution is kept high. The proposed framework can be used to evaluate the upper bound of the optical performance of an LF display for realistically aberrated eyes, which may help to find an optimal spatio-angular resolution required to render a high quality 3D scene.

Learning Wavefront Coding for Extended Depth of Field Imaging.

Depth of field is an important factor of imaging systems that highly affects the quality of the acquired spatial information. Extended depth of field (EDoF) imaging is a challenging ill-posed problem and has been extensively addressed in the literature. We propose a computational imaging approach for EDoF, where we employ wavefront coding via a diffractive optical element (DOE) and we achieve deblurring through a convolutional neural network. Thanks to the end-to-end differentiable modeling of optical image formation and computational post-processing, we jointly optimize the optical design, i.e., DOE, and the deblurring through standard gradient descent methods. Based on the properties of the underlying refractive lens and the desired EDoF range, we provide an analytical expression for the search space of the DOE, which is instrumental in the convergence of the end-to-end network. We achieve superior EDoF imaging performance compared to the state of the art, where we demonstrate results with minimal artifacts in various scenarios, including deep 3D scenes and broadband imaging.

Shearlet Transform based Light Field Compression Under Low Bitrates.

Light field (LF) acquisition devices capture spatial and angular information of a scene. In contrast with traditional cameras, the additional angular information enables novel postprocessing applications, such as 3D scene reconstruction, the ability to refocus at different depth planes, and synthetic aperture. In this paper, we present a novel compression scheme for LF data captured using multiple traditional cameras. The input LF views were divided into two groups: key views and decimated views. The key views were compressed using the multi-view extension of high-efficiency video coding (MV-HEVC) scheme, and decimated views were predicted using the shearlet-transform-based prediction (STBP) scheme. Additionally, the residual information of predicted views was also encoded and sent along with the coded stream of key views. The proposed scheme was evaluated over a benchmark multi-camera based LF datasets, demonstrating that incorporating the residual information into the compression scheme increased the overall peak signal to noise ratio (PSNR) by 2 dB. The proposed compression scheme performed significantly better at low bit rates compared to anchor schemes, which have a better level of compression efficiency in high bit-rate scenarios. The sensitivity of the human vision system towards compression artifacts, specifically at low bit rates, favors the proposed compression scheme over anchor schemes.

Fast and Accurate Depth Estimation from Sparse Light Fields.

We present a fast and accurate method for dense depth reconstruction, which is specifically tailored to process sparse, wide-baseline light field data captured with camera arrays. In our method, the source images are over-segmented into non-overlapping compact superpixels. We model superpixel as planar patches in the image space and use them as basic primitives for depth estimation. Such superpixel-based representation yields desired reduction in both memory and computation requirements while preserving image geometry with respect to the object contours. The initial depth maps, obtained by plane-sweeping independently for each view, are jointly refined via iterative belief-propagation-like optimization in superpixel domain. During the optimization, smoothness between the neighboring superpixels and geometric consistency between the views are enforced. To ensure rapid information propagation into textureless and occluded regions, together with the immediate superpixel neighbors, candidates from larger neighborhoods are sampled. Additionally, in order to make full use of the parallel graphics hardware a synchronous message update schedule is employed allowing to process all the superpixels of all the images at once. This way, the distribution of the scene geometry becomes distinctive already after the first iterations, facilitating stability and fast convergence of the refinement procedure. We demonstrate that a few refinement iterations result in globally consistent dense depth maps even in the presence of wide textureless regions and occlusions. The experiments show that while the depth reconstruction takes about a second per full high-definition view, the accuracy of the obtained depth maps is comparable with the state-of-the-art results, which otherwise require much longer processing time.

Methods for Simultaneous Robot-World-Hand-Eye Calibration: A Comparative Study.

In this paper, we propose two novel methods for robot-world-hand-eye calibration and provide a comparative analysis against six state-of-the-art methods. We examine the calibration problem from two alternative geometrical interpretations, called 'hand-eye' and 'robot-world-hand-eye', respectively. The study analyses the effects of specifying the objective function as pose error or reprojection error minimization problem. We provide three real and three simulated datasets with rendered images as part of the study. In addition, we propose a robotic arm error modeling approach to be used along with the simulated datasets for generating a realistic response. The tests on simulated data are performed in both ideal cases and with pseudo-realistic robotic arm pose and visual noise. Our methods show significant improvement and robustness on many metrics in various scenarios compared to state-of-the-art methods.

Time-of-Flight Range Measurement in Low-sensing Environment: Noise Analysis and Complex-domain Non-local Denoising.

In this work, we deal with the problem of denoising 3D scene range measurements acquired by Time-of-flight (ToF) range sensors and composed in the form of 2D image-like depth maps. We address the specific case of ToF low-sensing environment (LSE). Such environment is set by low-light sensing conditions, low-power hardware requirements, and low-reflectivity scenes. We demonstrate that data captured by a device in such mode can be effectively post-processed in order to reach the same measurement accuracy as if the device was working in normal operating mode. In order to achieve this, we first present an elaborated analysis of noise properties of ToF data sensed in LSE and verify the derived noise models by empirical measurements. Then, we develop a related novel non-local denoising approach working in complex domain and demonstrate its superiority against the state of the art for data acquired by an off-the-shelf ToF device.

Speckle reduction method for image-based coherent stereogram generation.

We propose a speckle noise reduction method for generation of coherent holographic stereograms. The method employs densely sampled light field (DSLF) of the scene together with depth information acquired for each ray in the captured DSLF. Speckle reduction is achieved based on the ray separation technique where the scene is first described as a superposition of sparse sets of point sources corresponding to separated sets of rays and then the holographic reconstructions corresponding to these sparse sets of point sources are added incoherently (intensity-wise) to obtain the final reconstruction. The proposed method handles the light propagation between the sparse scene points and hologram elements accurately by utilizing ray resampling based on the notion of DSLF. As a result, as demonstrated via numerical simulations, significant speckle suppression is achieved at no cost of sampling related reconstruction artifacts.

Light Field Reconstruction Using Shearlet Transform.

In this article we develop an image based rendering technique based on light field reconstruction from a limited set of perspective views acquired by cameras. Our approach utilizes sparse representation of epipolar-plane images (EPI) in shearlet transform domain. The shearlet transform has been specifically modified to handle the straight lines characteristic for EPI. The devised iterative regularization algorithm based on adaptive thresholding provides high-quality reconstruction results for relatively big disparities between neighboring views. The generated densely sampled light field of a given 3D scene is thus suitable for all applications which require light field reconstruction. The proposed algorithm compares favorably against state of the art depth image based rendering techniques and shows superior performance specifically in reconstructing scenes containing semi-transparent objects.

Super-resolution in a defocused plenoptic camera: a wave-optics-based approach.

Plenoptic cameras enable the capture of a light field with a single device. However, with traditional light field rendering procedures, they can provide only low-resolution two-dimensional images. Super-resolution is considered to overcome this drawback. In this study, we present a super-resolution method for the defocused plenoptic camera (Plenoptic 1.0), where the imaging system is modeled using wave optics principles and utilizing low-resolution depth information of the scene. We are particularly interested in super-resolution of in-focus and near in-focus scene regions, which constitute the most challenging cases. The simulation results show that the employed wave-optics model makes super-resolution possible for such regions as long as sufficiently accurate depth information is available.

Optimization of light field display-camera configuration based on display properties in spectral domain.

The visualization capability of a light field display is uniquely determined by its angular and spatial resolution referred to as display passband. In this paper we use a multidimensional sampling model for describing the display-camera channel. Based on the model, for a given display passband, we propose a methodology for determining the optimal distribution of ray generators in a projection-based light field display. We also discuss the required camera setup that can provide data with the necessary amount of details for such display that maximizes the visual quality and minimizes the amount of data.

Comparative study of morphological and time-frequency ECG descriptors for heartbeat classification.

The prompt and adequate detection of abnormal cardiac conditions by computer-assisted long-term monitoring systems depends greatly on the reliability of the implemented ECG automatic analysis technique, which has to discriminate between different types of heartbeats. In this paper, we present a comparative study of the heartbeat classification abilities of two techniques for extraction of characteristic heartbeat features from the ECG: (i) QRS pattern recognition method for computation of a large collection of morphological QRS descriptors; (ii) Matching Pursuits algorithm for calculation of expansion coefficients, which represent the time-frequency correlation of the heartbeats with extracted learning basic waveforms. The Kth nearest neighbour classification rule has been applied for assessment of the performances of the two ECG feature sets with the MIT-BIH arrhythmia database for QRS classification in five heartbeat types (normal beats, left and right bundle branch blocks, premature ventricular contractions and paced beats), as well as with five learning datasets-one general learning set (GLS, containing 424 heartbeats) and four local sets (GLS+about 0.5, 3, 6, 12 min from the beginning of the ECG recording). The achieved accuracies by the two methods are sufficiently high and do not show significant differences. Although the GLS was selected to comprise almost all types of appearing heartbeat waveforms in each file, the guaranteed accuracy (sensitivity between 90.7% and 99%, specificity between 95.5% and 99.9%) was reasonably improved when including patient-specific local learning set (sensitivity between 94.8% and 99.9%, specificity between 98.6% and 99.9%), with optimal size found to be about 3 min. The repeating waveforms, like normal beats, blocks, paced beats are better classified by the Matching Pursuits time-frequency descriptors, while the wide variety of bizarre premature ventricular contractions are better recognized by the morphological descriptors.