Embedded Processing for Extended Depth of Field Imaging Systems: From Infinite Impulse Response Wiener Filter to Learned Deconvolution.

Many works in the state of the art are interested in the increase of the camera depth of field (DoF) via the joint optimization of an optical component (typically a phase mask) and a digital processing step with an infinite deconvolution support or a neural network. This can be used either to see sharp objects from a greater distance or to reduce manufacturing costs due to tolerance regarding the sensor position. Here, we study the case of an embedded processing with only one convolution with a finite kernel size. The finite impulse response (FIR) filter coefficients are learned or computed based on a Wiener filter paradigm. It involves an optical model typical of codesigned systems for DoF extension and a scene power spectral density, which is either learned or modeled. We compare different FIR filters and present a method for dimensioning their sizes prior to a joint optimization. We also show that, among the filters compared, the learning approach enables an easy adaptation to a database, but the other approaches are equally robust.

Improved performance of a hybrid optical/digital imaging system with fast piecewise Wiener deconvolution.

We quantitatively investigate how spatially varying deblurring algorithms can improve the imaging performance of hybrid optical/digital systems affected by field aberrations. To this end, we validate a theoretical model of the maximal gain that linear and spatially varying deblurring can bring to any given lens, and derive a practical algorithm to implement this type of deblurring with low computational complexity. The results demonstrate the usefulness to properly coordinate and balance the roles of the imaging optical system and raw image post-processing: optimal final imaging quality can be obtained by a lens that has been optically designed to reduce field aberrations at the price of lower average raw optical quality, associated with a fast and "slightly" spatially varying piecewise Wiener deconvolution algorithm.

End-to-end optimization of optical systems with extended depth of field under wide spectrum illumination.

We study a way to take into account the scene illumination spectrum during end-to-end optimization of optical-digital hybrid systems that include annular binary phase masks to enhance their depth of field (DoF). We show that a phase mask specifically optimized for wide spectrum panchromatic imaging performs better under this type of illumination than phase masks optimized under monochromatic illumination assumption. Indeed, thanks to spectral averaging, the modulation transfer functions of such a phase mask are very close to each other. This guarantees a very homogeneous image quality across the DoF range, which we demonstrate theoretically and experimentally using a dedicated optical setup.

Comparison of methods for end-to-end co-optimization of optical systems and image processing with commercial lens design software.

We compare three different methods to co-optimize hybrid optical/digital imaging systems with a commercial lens design software: conventional optimization based on spot diagram minimization, optimization of a surrogate criterion based on a priori equalization of modulation transfer functions (MTFs), and minimization of the mean square error (MSE) between the ideal sharp image and the image restored by a unique deconvolution filter. To implement the latter method, we integrate - for the first time to our knowledge - MSE optimization to the software Synopsys CodeV. Taking as an application example the design of a Cooke triplet having good image quality everywhere in the field of view (FoV), we show that it is possible, by leveraging deconvolution during the optimization process, to adapt the spatial distribution of imaging performance to a prescribed goal. We also demonstrate the superiority of MSE co-optimization over the other methods, both in terms of quantitative and visual image quality.