Real-time phase retrieval in division of aperture microscopy with the transport of intensity equation.

The transport of intensity equation (TIE) allows to recover the phase of a microscopy sample from differently focused intensity measures along the axial direction of its optical field. In the present work, we propose a cost-effective technique for snapshot phase retrieval with TIE. The optics of a commercially available camera is replaced with a doublet system consisting of a microscope objective and a lenslet array with an extra lens mask attached to it. The system allows to obtain, in real-time and with no mechanical shift of either the sample or the sensor, the in-focus as well as a defocused image of the sample. From these two sub-aperture images, the intensity derivative term in TIE can then be approximated after image rectification. Phase is then retrieved for static as well as dynamic samples over the common view area. Validation experiments are presented.

Spatial perception in stereoscopic augmented reality based on multifocus sensing.

In many areas ranging from medical imaging to visual entertainment, 3D information acquisition and display is a key task. In this regard, in multifocus computational imaging, stacks of images of a certain 3D scene are acquired under different focus configurations and are later combined by means of post-capture algorithms based on image formation model in order to synthesize images with novel viewpoints of the scene. Stereoscopic augmented reality devices, through which is possible to simultaneously visualize the three dimensional real world along with overlaid digital stereoscopic image pair, could benefit from the binocular content allowed by multifocus computational imaging. Spatial perception of the displayed stereo pairs can be controlled by synthesizing the desired point of view of each image of the stereo-pair along with their parallax setting. The proposed method has the potential to alleviate the accommodation-convergence conflict and make augmented reality stereoscopic devices less vulnerable to visual fatigue.

Generalized Hough transform for 3D object recognition and visualization in integral imaging.

Object recognition is an automated image processing application of great interest in areas ranging from defect inspection to robot vision. In this regard, the generalized Hough transform is a well-established technique for the recognition of geometrical features even when they are partially occluded or corrupted by noise. To extend the original algorithm-aimed at detecting 2D geometrical features out of single images-we propose the robust integral generalized Hough transform, which corresponds to transformation under the generalized Hough transform of an elemental image array obtained from a 3D scene under integral imaging capture. The proposed algorithm constitutes a robust approach to pattern recognition in 3D scenes that takes into account information obtained not only from the individual processing of each image of the array but also from the spatial restrictions arising from perspective shifts between images. The problem of global detection of a 3D object of given size, position, and orientation is then exchanged under the robust integral generalized Hough transform for a more easily solved maximum detection in an accumulation (Hough) space dual to the elemental image array of the scene. Detected objects can then be visualized following refocusing schemes of integral imaging. Validation experiments for the detection and visualization of partially occluded 3D objects are presented. To the best of our knowledge, this is the first implementation of the generalized Hough transform for 3D object detection in integral imaging.

Computational multifocus fluorescence microscopy for three-dimensional visualization of multicellular tumor spheroids.

SIGNIFICANCE: Three-dimensional (3D) visualization of multicellular tumor spheroids (MCTS) in fluorescence microscopy can rapidly provide qualitative morphological information about the architecture of these cellular aggregates, which can recapitulate key aspects of their in vivo counterpart. AIM: The present work is aimed at overcoming the shallow depth-of-field (DoF) limitation in fluorescence microscopy while achieving 3D visualization of thick biological samples under study. APPROACH: A custom-built fluorescence microscope with an electrically focus-tunable lens was developed to optically sweep in-depth the structure of MCTS. Acquired multifocus stacks were combined by means of postprocessing algorithms performed in the Fourier domain. RESULTS: Images with relevant characteristics as extended DoF, stereoscopic pairs as well as reconstructed viewpoints of MCTS were obtained without segmentation of the focused regions or estimation of the depth map. The reconstructed images allowed us to observe the 3D morphology of cell aggregates. CONCLUSIONS: Computational multifocus fluorescence microscopy can provide 3D visualization in MCTS. This tool is a promising development in assessing the morphological structure of different cellular aggregates while preserving a robust yet simple optical setup.

Fully invariant generalized Hough transform by out-of-focus multiview sensing with pupil array.

Recognition of geometrical shapes in real-time and fully invariant (i.e., invariant under changes in position, scale, and orientation) is a demanding task in automated image analysis. In particular, the generalized Hough transform (GHT) is a well-known algorithm for the recognition of complex patterns out of edge binary images even with disconnected boundaries or corrupted by noise. In this work we present a space multiplexed optical implementation of the GHT which, by exploiting the redundancy derived from multiview sensing of a two-dimensional image and its out- of-focus capture with an adequate pupil array, allows us to obtain in a single shot the GHT of this image invariant to target shift, scale, and orientation. Experimental validation of the working principle is presented, along with an assessment of the robustness of the system against noise in the input.

Fourier domain post-acquisition aperture reshaping from a multi-focus stack.

Computational optical imaging methods allow the extension of the functionality of traditional cameras. The shape of the aperture in an optical system determines the shape in which the out-of-focus points are blurred in a captured image. In this work we present a method in the Fourier domain that allows us, from an acquired multi-focus image stack, to synthesize images of a three-dimensional scene as if they had been acquired with apertures with arbitrary shapes. Partially extended depth-of-field as well as all-in-focus image reconstruction can be obtained as particular cases.

Image segmentation by nonlinear filtering of optical Hough transform.

The identification and extraction (i.e., segmentation) of geometrical features is crucial in many tasks requiring image analysis. We present a method for the optical segmentation of features of interest from an edge enhanced image. The proposed method is based on the nonlinear filtering (implemented by the use of a spatial light modulator) of the generalized optical Hough transform and is capable of discriminating features by shape and by size. The robustness of the method against noise in the input, low contrast, or overlapping of geometrical features is assessed, and experimental validation of the working principle is presented.

Reconstruction of perspective shifts and refocusing of a three-dimensional scene from a multi-focus image stack.

The convergence of optical imaging acquisition and image processing algorithms is a fast-evolving interdisciplinary research field focused on the reconstruction of images with novel features of interest. We propose a method for post-capture perspective shift reconstruction (in the x, y, and z directions) of a three-dimensional scene as well as refocusing with apertures of arbitrary shapes and sizes from an optimal multi-focus image stack. The approach is based on the reorganization of the acquired visual information considering a depth-variant point-spread function, which allows it to be applied to strongly defocused multi-focus image stacks. Our method is performed without estimating the depth map or segmenting the in-focus regions. A conventional camera combined with an electrically tunable lens is used for image acquisition and does not require scale transformation or registration between the acquired images. Experimental results for both real and synthetic data images are provided and compared to state-of-the-art schemes.

Optical implementation of the generalized Hough transform with totally incoherent light.

The generalized Hough transform is a well-established technique for detecting complex shapes in images containing noisy or missing data. We present an efficient optical implementation of this transform using an electrical lens with variable focal length and a rotating pupil mask matching the pattern to be found. The proposed setup works under fully (i.e., both spatially and temporally) incoherent illumination and can handle orientation changes or scale variations in the pattern. Validation experiments showing its real-time application are presented.

All-in-focus image reconstruction under severe defocus.

Limited depth-of-focus is a problem in many fields of optics, e.g., microscopy and macro-photography. We propose a new physically based method with a space variant point spread function (PSF) to accomplish all-in-focus reconstruction (image fusion) from a multi-focus image sequence in order to extend the depth-of-field. The proposed method works well under strong defocus conditions for color image stacks of arbitrary length. Experimental results are provided to demonstrate that our method outperforms state-of-the-art image fusion algorithms for strong defocus on both synthetic as well as real data images.

Real-time pattern recognition using an optical generalized Hough transform.

We present some pattern recognition applications of a generalized optical Hough transform and the temporal multiplexing strategies for dynamic scale and orientation-variant detection. Unlike computer-based implementations of the Hough transform, in principle its optical implementation does not impose restrictions on the execution time or on the resolution of the images or frame rate of the videos to be processed, which is potentially useful for real-time applications. Validation experiments are presented.

Edge enhancement of color images using a digital micromirror device.

A method for orientation-selective enhancement of edges in color images is proposed. The method utilizes the capacity of digital micromirror devices to generate a positive and a negative color replica of the image used as input. When both images are slightly displaced and imagined together, one obtains an image with enhanced edges. The proposed technique does not require a coherent light source or precise alignment. The proposed method could be potentially useful for processing large image sequences in real time. Validation experiments are presented.

Color encoding of binary fringes for gamma correction in 3-D profiling.

Three-dimensional profiling by sinusoidal fringe projection using PSI-algorithms are distorted by the nonlinear response of digital cameras and commercial video projectors. To solve the problem, we present a fringe generation technique that consists of projecting and acquiring a temporal sequence of strictly binary color patterns, whose (adequately weighted) average leads to sinusoidal fringe patterns with the required number of bits, which allows for a reliable three-dimensional profile using a PSI-algorithm. Validation experiments are presented.

Incoherent optical processor for nondirectional edge enhancement of color images.

We present an optical method for nondirectional edge extraction/enhancement in color images. The method is based on the capability of twisted-nematic LCDs to traduce the image information in changes of the state of polarization of light, which allows us to generate simultaneously two replicas of the digital image displayed on the LCD: a true-color ("positive") image and a complementary-color ("negative") one. In our setup the imaging system consists of a lens plus a pupil mask formed with concentric apertures and orthogonal polarizers. This layout allows us to simultaneously image a well-focused positive replica (due to the circular aperture) superimposed to a slightly defocused negative one (due to the annular aperture). It is not difficult to demonstrate that this generates a nondirectional (Laplacian) edge enhancement. Unlike Fourier, our proposal works with incoherent illumination and does not require precise alignment, and thus, it could be a useful tool for edge extraction/enhancement in large images in real-time applications. Validation experiments are presented.

Optical processing of color images with incoherent illumination: orientation-selective edge enhancement using a modified liquid-crystal display.

We present a novel optical method for edge enhancement in color images based on the polarization properties of liquid-crystal displays (LCD). In principle, a LCD generates simultaneously two color-complementary, orthogonally polarized replicas of the digital image used as input. The currently viewed image in standard LCD monitors and cell phone's screens -which we will refer as the "positive image or true-color image"- is the one obtained by placing an analyzer in front of the LCD, in cross configuration to the back polarizer of the display. The orthogonally polarized replica of this image -the "negative image or complementary-color image"- is absorbed by the front polarizer. In order to generate the positive and negative replica with a slight displacement between them, we used a LCD monitor whose analyzer (originally a linear polarizer) was replaced by a calcite crystal acting as beam displacer. When both images are superimposed laterally displaced across the image plane, one obtains an image with enhanced first-order derivatives along a specific direction. The proposed technique works under incoherent illumination and does not require precise alignment, and thus, it could be potentially useful for processing large color images in real-time applications. Validation experiments are presented.

Three-dimensional profiling with binary fringes using phase-shifting interferometry algorithms.

Three-dimensional shape measurements by sinusoidal fringe projection using phase-shifting interferometry algorithms are distorted by the nonlinear response in intensity of commercial video projectors and digital cameras. To solve the problem, we present a method that consists in projecting and acquiring a temporal sequence of strictly binary patterns, whose (adequately weighted) average leads to a sinusoidal fringe pattern with the required number of bits. Since binary patterns consist of "ones" and "zeros"--and no half-tones are involved--the nonlinear response of the projector and the camera will not play a role, and a nearly unit contrast gray-level sinusoidal fringe pattern is obtained. Validation experiments are presented.

Analog image contouring using a twisted-nematic liquid-crystal display.

We present a novel image contouring method based on the polarization features of the twisted-nematic liquid-crystal displays (TN-LCDs). TN-LCDs are manufactured to work between a crossed polarizer-analyzer pair. When the analyzer is at 45 deg (instead of 90 deg) with respect to the polarizer, one obtains an optically processed image with pronounced outlines (dark contours) at middle intensity, i.e., the borders between illuminated and dark areas are enhanced. The proposed method is quite robust and does not require precise alignment or coherent illumination. Since it does not involve numerical processing, it could be useful for contouring large images in real-time, which presents potential applications in medical and biological imaging. Validation experiments are presented.