Miniature color camera via flat hybrid meta-optics.

The race for miniature color cameras using flat meta-optics has rapidly developed the end-to-end design framework using neural networks. Although a large body of work has shown the potential of this methodology, the reported performance is still limited due to fundamental limitations coming from meta-optics, mismatch between simulated and resultant experimental point spread functions, and calibration errors. Here, we use a HIL optics design methodology to solve these limitations and demonstrate a miniature color camera via flat hybrid meta-optics (refractive + meta-mask). The resulting camera achieves high-quality full-color imaging for a 5-mm aperture optics with a focal length of 5 mm. We observed a superior quality of the images captured by the hybrid meta-optical camera compared to a compound multi-lens optics of a mirrorless commercial camera.

Hybrid diffractive optics design via hardware-in-the-loop methodology for achromatic extended-depth-of-field imaging.

End-to-end optimization of diffractive optical elements (DOEs) profile through a digital differentiable model combined with computational imaging have gained an increasing attention in emerging applications due to the compactness of resultant physical setups. Despite recent works have shown the potential of this methodology to design optics, its performance in physical setups is still limited and affected by manufacturing artefacts of DOE, mismatch between simulated and resultant experimental point spread functions, and calibration errors. Additionally, the computational burden of the digital differentiable model to effectively design the DOE is increasing, thus limiting the size of the DOE that can be designed. To overcome the above mentioned limitations, a co-design of hybrid optics and image reconstruction algorithm is produced following the end-to-end hardware-in-the-loop strategy, using for optimization a convolutional neural network equipped with quantitative and qualitative loss functions. The optics of the imaging system consists on the phase-only spatial light modulator (SLM) as DOE and refractive lens. SLM phase-pattern is optimized by applying the Hardware-in-the-loop technique, which helps to eliminate the mismatch between numerical modelling and physical reality of image formation as light propagation is not numerically modelled but is physically done. Comparison with compound multi-lens optics of a last generation smartphone and a mirrorless commercial cameras show that the proposed system is advanced in all-in-focus sharp imaging for a depth range 0.4-1.9 m.

Power-balanced hybrid optics boosted design for achromatic extended depth-of-field imaging via optimized mixed OTF.

A power-balanced hybrid optical imaging system has a diffractive computational camera, introduced in this paper, with image formation by a refractive lens and multilevel phase mask (MPM). This system provides a long focal depth with low chromatic aberrations thanks to MPM and a high energy light concentration due to the refractive lens. We introduce the concept of optical power balance between the lens and MPM, which controls the contribution of each element to modulate the incoming light. Additional features of our MPM design are the inclusion of the quantization of the MPM's shape on the number of levels and the Fresnel order (thickness) using a smoothing function. To optimize the optical power balance as well as the MPM, we built a fully differentiable image formation model for joint optimization of optical and imaging parameters for the proposed camera using neural network techniques. We also optimized a single Wiener-like optical transfer function (OTF) invariant to depth to reconstruct a sharp image. We numerically and experimentally compare the designed system with its counterparts, lensless and just-lens optical systems, for the visible wavelength interval (400-700) nm and the depth-of-field range (0.5-∞ m for numerical and 0.5-2 m for experimental). We believe the attained results demonstrate that the proposed system equipped with the optimal OTF overcomes its counterparts--even when they are used with optimized OTF--in terms of the reconstruction quality for off-focus distances. The simulation results also reveal that optimizing the optical power balance, Fresnel order, and the number of levels parameters are essential for system performance attaining an improvement of up to 5 dB of PSNR using the optimized OTF compared to its counterpart lensless setup.

Polarization holographic recording of vortex diffractive optical elements on azopolymer thin films and 3D analysis via phase-shifting digital holographic microscopy.

Direct fabrication of complex diffractive optical elements (DOEs) on photosensitive thin films is of critical importance for the development of advanced optical instruments. In this paper, we design and investigate DOEs capable of generating optical vortices. Analog and digital approaches for one-step polarization holographic recording of vortex DOEs on new carbazole-based azopolymer thin films are described. First configuration involves analog polarization holographic recording using a vortex phase retarder and has as a result the DOE producing a diffraction pattern with phase singularities aligned in a single line. Similar diffraction picture is achieved by the single-beam digital holographic recording setup with an integrated spatial light modulator. In the third system, the implemented double-beam digital polarization holographic recording setup yields simultaneously a spatial multiplexed vortex pattern. Diffraction efficiency evolution of these three types of DOEs are monitored and compared. The phase-shifting digital holographic microscope with an electrically controlled liquid crystal variable retarder is applied to investigate the phase and surface topography of the inscribed diffractive optical elements. The comparison between the digital and analog micro-patterning techniques contributes new evidence to limited data on the influence of the analog and digital generation of the spiral wavefront on the performance of vortex DOEs.

Lensless hyperspectral phase imaging in a self-reference setup based on Fourier transform spectroscopy and noise suppression.

A novel phase retrieval algorithm for broadband hyperspectral phase imaging from noisy intensity observations is proposed. It utilizes advantages of the Fourier transform spectroscopy in the self-referencing optical setup and provides additional, beyond spectral intensity distribution, reconstruction of the investigated object's phase. The noise amplification Fellgett's disadvantage is relaxed by the application of a sparse wavefront noise filtering embedded in the proposed algorithm. The algorithm reliability is proved by simulation tests and by results of physical experiments for transparent objects. These tests demonstrate precise phase imaging and object depth (profile) reconstruction.

Single exposure lensless subpixel phase imaging: optical system design, modelling, and experimental study.

Design and optimization of lensless phase-retrieval optical system with phase modulation of free-space propagation wavefront is proposed for subpixel imaging to achieve super-resolution reconstruction. Contrary to the traditional super-resolution phase-retrieval, the method in this paper requires a single observation only and uses the advanced Super-Resolution Sparse Phase Amplitude Retrieval (SR-SPAR) iterative technique which contains optimized sparsity based filters and multi-scale filters. The successful object imaging relies on modulation of the object wavefront with a random phase-mask, which generates coded diffracted intensity pattern, allowing us to extract subpixel information. The system's noise-robustness was investigated and verified. The super-resolution phase-imaging is demonstrated by simulations and physical experiments. The simulations included high quality reconstructions with super-resolution factor of 5, and acceptable at factor up to 9. By physical experiments 3 µm details were resolved, which are 2.3 times smaller than the resolution following from the Nyquist-Shannon sampling theorem.

Terahertz pulse time-domain holography with balance detection: complex-domain sparse imaging.

We investigated the peculiarities of the terahertz pulse time-domain holography principle in the case of raster scanning with the balance detection system. The noise in this system represents a Skellam distribution model, which differentiates it from systems based on a photoconductive antenna. We analyzed this Skellam model and provided both numerical and experimental investigations. We found that the variance of the noise in the balance detection system does not depend on the true signal. Complex-domain images obtained in this model are filtered by block-matching algorithms adapted for spatio-temporal and spatiospectral volumetric data. We presented a new cube complex-domain filter algorithm that uses block matching in all 3D data sets simultaneously in spatial and frequency coordinates. A combination of temporal and complex-domain filters allows us to expand the dynamic range of terahertz frequencies for which we can obtain amplitude/phase information. Experimental data demonstrate an improvement in the quality of the resultant images both in the time domain and complex-spectral domain. The simulation and experimental results are in good agreement.

Spectral Object Recognition in Hyperspectral Holography with Complex-Domain Denoising.

In this paper, we have applied a recently developed complex-domain hyperspectral denoiser for the object recognition task, which is performed by the correlation analysis of investigated objects' spectra with the fingerprint spectra from the same object. Extensive experiments carried out on noisy data from digital hyperspectral holography demonstrate a significant enhancement of the recognition accuracy of signals masked by noise, when the advanced noise suppression is applied.

Hyperspectral data denoising for terahertz pulse time-domain holography.

We investigated data denoising in hyperspectral terahertz pulse time-domain holography. Using the block-matching algorithms adapted for spatio-temporal and spatio-spectral volumetric data we studied and optimized parameters of these algorithms to improve phase image reconstruction quality. We propose a sequential application of the two algorithms oriented on work in temporal and spectral domains. Experimental data demonstrate the improvement in the quality of the resultant time-domain images as well as phase images and object's relief. The simulation results are proved by comparison with the experimental ones.

Super-resolution microscopy for biological specimens: lensless phase retrieval in noisy conditions.

The paper is devoted to a computational super-resolution microscopy. A complex-valued wavefront of a transparent biological cellular specimen is restored from multiple intensity diffraction patterns registered with noise. For this problem, the recently developed lensless super-resolution phase retrieval algorithm [Optica, 4(7), 786 (2017)] is modified and tuned. This algorithm is based on a random phase coding of the wavefront and on a sparse complex-domain approximation of the specimen. It is demonstrated in experiments, that the reliable phase and amplitude imaging of the specimen is achieved for the low signal-to-noise ratio provided a low dynamic range of observations. The filterings in the observation domain and specimen variables are specific features of the applied algorithm. If these filterings are omitted the algorithm becomes a super-resolution version of the standard iterative phase retrieval algorithms. In comparison with this simplified algorithm with no filterings, our algorithm shows a valuable improvement in imaging with much smaller number of observations and shorter exposure time. In this way, presented algorithm demonstrates ability to work in a low radiation photon-limited mode.

Single Image Super-Resolution Based on Wiener Filter in Similarity Domain.

Single image super-resolution (SISR) is an ill-posed problem aiming at estimating a plausible high-resolution (HR) image from a single low-resolution image. Current state-of-the-art SISR methods are patch-based. They use either external data or internal self-similarity to learn a prior for an HR image. External data-based methods utilize a large number of patches from the training data, while self-similarity-based approaches leverage one or more similar patches from the input image. In this paper, we propose a self-similarity-based approach that is able to use large groups of similar patches extracted from the input image to solve the SISR problem. We introduce a novel prior leading to the collaborative filtering of patch groups in a 1D similarity domain and couple it with an iterative back-projection framework. The performance of the proposed algorithm is evaluated on a number of SISR benchmark data sets. Without using any external data, the proposed approach outperforms the current non-convolutional neural network-based methods on the tested data sets for various scaling factors. On certain data sets, the gain is over 1 dB, when compared with the recent method A+. For high sampling rate (x4), the proposed method performs similarly to very recent state-of-the-art deep convolutional network-based approaches.

High-accuracy off-axis wavefront reconstruction from noisy data: local least square with multiple adaptive windows.

A variational algorithm to object wavefront reconstruction from noisy intensity observations is developed for the off-axis holography scenario with imaging in the acquisition plane. The algorithm is based on the local least square technique proposed in paper [J. Opt. Soc. Am. A21, 367 (2004)]. First, multiple reconstructions of the wavefront are produced for various size and various directional windows applied for localization of estimation. At the second stage, a special statistical rule is applied in order to select the best window size estimate for each pixel of the image and for each of the directional windows. At the third final stage the estimates of the different directions obtained for each pixel are aggregated in the final one. Simulation experiments and real data processing prove that the developed algorithm demonstrate the performance of the extraordinary quality and accuracy for both the phase and amplitude of the object wavefront.

Texture Classification Using Dense Micro-Block Difference.

This paper is devoted to the problem of texture classification. Motivated by recent advancements in the field of compressive sensing and keypoints descriptors, a set of novel features called dense micro-block difference (DMD) is proposed. These features provide highly descriptive representation of image patches by densely capturing the granularities at multiple scales and orientations. Unlike most of the earlier work on local features, the DMD does not involve any quantization, thus retaining the complete information. We demonstrate that the DMD have dimensionality much lower than Scale Invariant Feature Transform (SIFT) and can be computed using integral image much faster than SIFT. The proposed features are encoded using the Fisher vector method to obtain an image descriptor, which considers high-order statistics. The proposed image representation is combined with the linear support vector machine classifier. Extensive experiments are conducted on five texture data sets (KTH-TIPS, UMD, KTH-TIPS-2a, Brodatz, and Curet) using standard protocols. The results demonstrate that our approach outperforms the state-of-the-art in texture classification.

Nonlocal transform-domain filter for volumetric data denoising and reconstruction.

We present an extension of the BM3D filter to volumetric data. The proposed algorithm, BM4D, implements the grouping and collaborative filtering paradigm, where mutually similar d-dimensional patches are stacked together in a (d+1)-dimensional array and jointly filtered in transform domain. While in BM3D the basic data patches are blocks of pixels, in BM4D we utilize cubes of voxels, which are stacked into a 4-D "group." The 4-D transform applied on the group simultaneously exploits the local correlation present among voxels in each cube and the nonlocal correlation between the corresponding voxels of different cubes. Thus, the spectrum of the group is highly sparse, leading to very effective separation of signal and noise through coefficient shrinkage. After inverse transformation, we obtain estimates of each grouped cube, which are then adaptively aggregated at their original locations. We evaluate the algorithm on denoising of volumetric data corrupted by Gaussian and Rician noise, as well as on reconstruction of volumetric phantom data with non-zero phase from noisy and incomplete Fourier-domain (k-space) measurements. Experimental results demonstrate the state-of-the-art denoising performance of BM4D, and its effectiveness when exploited as a regularizer in volumetric data reconstruction.

Video denoising, deblocking, and enhancement through separable 4-D nonlocal spatiotemporal transforms.

We propose a powerful video filtering algorithm that exploits temporal and spatial redundancy characterizing natural video sequences. The algorithm implements the paradigm of nonlocal grouping and collaborative filtering, where a higher dimensional transform-domain representation of the observations is leveraged to enforce sparsity, and thus regularize the data: 3-D spatiotemporal volumes are constructed by tracking blocks along trajectories defined by the motion vectors. Mutually similar volumes are then grouped together by stacking them along an additional fourth dimension, thus producing a 4-D structure, termed group, where different types of data correlation exist along the different dimensions: local correlation along the two dimensions of the blocks, temporal correlation along the motion trajectories, and nonlocal spatial correlation (i.e., self-similarity) along the fourth dimension of the group. Collaborative filtering is then realized by transforming each group through a decorrelating 4-D separable transform and then by shrinkage and inverse transformation. In this way, the collaborative filtering provides estimates for each volume stacked in the group, which are then returned and adaptively aggregated to their original positions in the video. The proposed filtering procedure addresses several video processing applications, such as denoising, deblocking, and enhancement of both grayscale and color data. Experimental results prove the effectiveness of our method in terms of both subjective and objective visual quality, and show that it outperforms the state of the art in video denoising.

BM3D frames and variational image deblurring.

A family of the block matching 3-D (BM3D) algorithms for various imaging problems has been recently proposed within the framework of nonlocal patchwise image modeling , . In this paper, we construct analysis and synthesis frames, formalizing BM3D image modeling, and use these frames to develop novel iterative deblurring algorithms. We consider two different formulations of the deblurring problem, i.e., one given by the minimization of the single-objective function and another based on the generalized Nash equilibrium (GNE) balance of two objective functions. The latter results in the algorithm where deblurring and denoising operations are decoupled. The convergence of the developed algorithms is proved. Simulation experiments show that the decoupled algorithm derived from the GNE formulation demonstrates the best numerical and visual results and shows superiority with respect to the state of the art in the field, confirming a valuable potential of BM3D-frames as an advanced image modeling tool.

Absolute phase estimation: adaptive local denoising and global unwrapping.

The paper attacks absolute phase estimation with a two-step approach: the first step applies an adaptive local denoising scheme to the modulo-2 pi noisy phase; the second step applies a robust phase unwrapping algorithm to the denoised modulo-2 pi phase obtained in the first step. The adaptive local modulo-2 pi phase denoising is a new algorithm based on local polynomial approximations. The zero-order and the first-order approximations of the phase are calculated in sliding windows of varying size. The zero-order approximation is used for pointwise adaptive window size selection, whereas the first-order approximation is used to filter the phase in the obtained windows. For phase unwrapping, we apply the recently introduced robust (in the sense of discontinuity preserving) PUMA unwrapping algorithm [IEEE Trans. Image Process.16, 698 (2007)] to the denoised wrapped phase. Simulations give evidence that the proposed algorithm yields state-of-the-art performance, enabling strong noise attenuation while preserving image details.

Practical Poissonian-Gaussian noise modeling and fitting for single-image raw-data.

We present a simple and usable noise model for the raw-data of digital imaging sensors. This signal-dependent noise model, which gives the pointwise standard-deviation of the noise as a function of the expectation of the pixel raw-data output, is composed of a Poissonian part, modeling the photon sensing, and Gaussian part, for the remaining stationary disturbances in the output data. We further explicitly take into account the clipping of the data (over- and under-exposure), faithfully reproducing the nonlinear response of the sensor. We propose an algorithm for the fully automatic estimation of the model parameters given a single noisy image. Experiments with synthetic images and with real raw-data from various sensors prove the practical applicability of the method and the accuracy of the proposed model.

Measuring directional coupling between EEG sources.

Directional connectivity in the brain has been typically computed between scalp electroencephalographic (EEG) signals, neglecting the fact that correlations between scalp measurements are partly caused by electrical conduction through the head volume. Although recently proposed techniques are able to identify causality relationships between EEG sources rather than between recording sites, most of them need a priori assumptions about the cerebral regions involved in the EEG generation. We present a novel methodology based on multivariate autoregressive (MVAR) modeling and Independent Component Analysis (ICA) able to determine the temporal activation of the intracerebral EEG sources as well as their approximate locations. The direction of synaptic flow between these EEG sources is then estimated using the directed transfer function (DTF), and the significance of directional coupling strength evaluated with surrogated data. The reliability of this approach was assessed with simulations manipulating the number of data samples, the depth and orientation of the equivalent source dipoles, the presence of different noise sources, and the violation of the non-Gaussianity assumption inherent to the proposed technique. The simulations showed the superior accuracy of the proposed approach over other traditional techniques in most tested scenarios. Its validity was also evaluated analyzing the generation mechanisms of the EEG-alpha rhythm recorded from 20 volunteers under resting conditions. Results suggested that the major generation mechanism underlying EEG-alpha oscillations consists of a strong bidirectional feedback between thalamus and cuneus. The precuneus also seemed to actively participate in the generation of the alpha rhythm although it did not exert a significant causal influence neither on the thalamus nor on the cuneus. All together, these results suggest that the proposed methodology is a promising non-invasive approach for studying directional coupling between mutually interconnected neural populations.

Discrete diffraction transform for propagation, reconstruction, and design of wavefield distributions.

A discrete diffraction transform (DDT) is a novel discrete wavefield propagation model that is aliasing free for a pixelwise invariant object distribution. For this class of distribution, the model is precise and has no typical discretization effects because it corresponds to accurate calculation of the diffraction integral. A spatial light modulator (SLM) is a good example of a system where a pixelwise invariant distribution appears. Frequency domain regularized inverse algorithms are developed for reconstruction of the object wavefield distribution from the distribution given in the sensor plane. The efficiency of developed frequency domain algorithms is demonstrated by simulation.