Incoherent color digital holography with computational coherent superposition for fluorescence imaging [Invited].

We present color fluorescence imaging using an incoherent digital holographic technique in which holographic multiplexing of multiple wavelengths is exploited. Self-interference incoherent digital holography with a single-path in-line configuration and the computational coherent superposition scheme are adopted to obtain color holographic three-dimensional information of self-luminous objects with a monochrome image sensor and no mechanical scanning. We perform not only simultaneous color three-dimensional sensing of multiple self-luminous objects but also color fluorescence imaging of stained biological samples. Color fluorescence imaging with an improved point spread function is also demonstrated experimentally by adopting a Fresnel incoherent correlation holography system.

Two-step phase-shifting interferometry for self-interference digital holography.

We propose a phase-shifting interferometry technique using only two in-line phase-shifted self-interference holograms. There is no requirement for additional recording or estimation in the measurement. The proposed technique adopts a mathematical model for self-interference digital holography. The effectiveness of the proposed technique is demonstrated by experiments on incoherent digital holographic microscopy and color-multiplexed fluorescence digital holography with computational coherent superposition. Two-color-multiplexed four-step phase-shifting incoherent digital holography is realized for the first time, to the best of our knowledge, using the proposed technique.

Multiwavelength three-dimensional microscopy with spatially incoherent light, based on computational coherent superposition.

In this Letter, we propose spatially incoherent multiwavelength three-dimensional (3D) microscopy that exploits holographic multiplexing and is based on computational coherent superposition (CCS). The proposed microscopy generates spatially incoherent wavelength-multiplexed self-interference holograms with a multiband-pass filter and spatially and temporally incoherent light diffracted from specimens. Selective extractions of 3D spatial information at multiple wavelengths from the holograms are realized using the CCS scheme. We constructed fully mechanical-motion-free holographic multiwavelength 3D microscopy systems and conducted experiments to demonstrate the microscopy.

Multiwavelength-multiplexed phase-shifting incoherent color digital holography.

We propose multiwavelength-multiplexed phase-shifting incoherent color digital holography. In this technique, a monochrome image sensor records wavelength-multiplexed, phase-shifted, and incoherent holograms, and a phase-shifting interferometry technique selectively extracts object waves at multiple wavelengths from the several recorded holograms. Spatially incoherent light that contains multiple wavelengths illuminates objects, and multiwavelength-incoherent object waves are simultaneously obtained without using any wavelength filters. Its effectiveness is experimentally demonstrated for transparent and reflective objects.

Reconstruction of a three-dimensional color-video of a point-cloud object using the projection-type holographic display with a holographic optical element.

Here, we managed to reconstruct a three-dimensional color video of a point-cloud object using a projection-type holographic display with a holographic optical element as an optical screen. The holographic optical element has the function of an off-axis concave mirror and has been created by the wavefront printer digitally. We defined and implemented an algorithm to reconstruct a three-dimensional image at a chosen position considering the specification of the holographic optical element designed digitally. We successfully demonstrated a reconstruction of the color video in question, composed of three-dimensional images through the holographic optical element.

Homography based identification for automatic and robust calibration of projection integral imaging displays.

Recent advances in the creation of microlens arrays as holographic optical elements allow the creation of projector-based see-through light field displays suitable for augmented reality. These systems require an accurate calibration of the projector with relation to the microlens array, as any small misalignment causes the 3D reconstruction to fail. The methods reported so far require precise placement of the calibration camera w.r.t. the lens array screen, which affects the display configuration. We propose a calibration approach which is more robust, and which allows free camera placement. Hence, it does not limit the capabilities of the system. Both a homography-based technique and structured light play a central role in realizing such a method. The method was tested on a projection-based integral imaging display system consisting of a consumer-grade projector and a digitally designed holographic optical element based micromirror array screen. The calibration method compensates for the lens distortion, intrinsics, and positioning of the projector with relation to the screen. The method uses a single camera and does not require the use of obtrusive markers as reference. We give an in-depth explanation of the different steps of the algorithm, and verify the calibration using both a simulated and a real-world setup.

Digitally designed holographic optical element for light field displays.

Concave micro-mirror arrays fabricated as holographic optical elements are used in projector-based light field displays due to their see-through characteristics. The optical axes of each micro-mirror in the array are usually made parallel to each other, which simplifies the fabrication, integral image rendering, and calibration process. However, this demands that the beam from the projector be collimated and made parallel to the optical axis of each elemental micro-mirror. This requires additional collimation optics, which puts serious limitations on the size of the display. In this Letter, we propose a solution to the above issue by introducing a new method to fabricate holographic concave micro-mirror array sheets and explain how they work in detail. 3D light field reconstructions of the size 20 cm×10 cm and 6 cm in depth are achieved using a conventional projector without any collimation optics.

Copying of holograms by spot scanning approach.

To replicate holograms, contact copying has conventionally been used. In this approach, a photosensitive material is fixed together with a master hologram and illuminated with a coherent beam. This method is simple and enables high-quality copies; however, it requires a large optical setup for large-area holograms. In this paper, we present a new method of replicating holograms that uses a relatively compact optical system even for the replication of large holograms. A small laser spot that irradiates only part of the hologram is used to reproduce the hologram by scanning the spot over the whole area of the hologram. We report on the results of experiments carried out to confirm the copy quality, along with a guide to design scanning conditions. The results show the potential effectiveness of the large-area hologram replication technology using a relatively compact apparatus.

Real-time colour hologram generation based on ray-sampling plane with multi-GPU acceleration.

Although electro-holography can reconstruct three-dimensional (3D) motion pictures, its computational cost is too heavy to allow for real-time reconstruction of 3D motion pictures. This study explores accelerating colour hologram generation using light-ray information on a ray-sampling (RS) plane with a graphics processing unit (GPU) to realise a real-time holographic display system. We refer to an image corresponding to light-ray information as an RS image. Colour holograms were generated from three RS images with resolutions of 2,048 × 2,048; 3,072 × 3,072 and 4,096 × 4,096 pixels. The computational results indicate that the generation of the colour holograms using multiple GPUs (NVIDIA Geforce GTX 1080) was approximately 300-500 times faster than those generated using a central processing unit. In addition, the results demonstrate that 3D motion pictures were successfully reconstructed from RS images of 3,072 × 3,072 pixels at approximately 15 frames per second using an electro-holographic reconstruction system in which colour holograms were generated from RS images in real time.

Projection-type see-through holographic three-dimensional display.

Owing to the limited spatio-temporal resolution of display devices, dynamic holographic three-dimensional displays suffer from a critical trade-off between the display size and the visual angle. Here we show a projection-type holographic three-dimensional display, in which a digitally designed holographic optical element and a digital holographic projection technique are combined to increase both factors at the same time. In the experiment, the enlarged holographic image, which is twice as large as the original display device, projected on the screen of the digitally designed holographic optical element was concentrated at the target observation area so as to increase the visual angle, which is six times as large as that for a general holographic display. Because the display size and the visual angle can be designed independently, the proposed system will accelerate the adoption of holographic three-dimensional displays in industrial applications, such as digital signage, in-car head-up displays, smart-glasses and head-mounted displays.

Large size three-dimensional video by electronic holography using multiple spatial light modulators.

In this paper, we propose a new method of using multiple spatial light modulators (SLMs) to increase the size of three-dimensional (3D) images that are displayed using electronic holography. The scalability of images produced by the previous method had an upper limit that was derived from the path length of the image-readout part. We were able to produce larger colour electronic holographic images with a newly devised space-saving image-readout optical system for multiple reflection-type SLMs. This optical system is designed so that the path length of the image-readout part is half that of the previous method. It consists of polarization beam splitters (PBSs), half-wave plates (HWPs), and polarizers. We used 16 (4 × 4) 4K×2K-pixel SLMs for displaying holograms. The experimental device we constructed was able to perform 20 fps video reproduction in colour of full-parallax holographic 3D images with a diagonal image size of 85 mm and a horizontal viewing-zone angle of 5.6 degrees.

Image size scalable full-parallax coloured three-dimensional video by electronic holography.

In electronic holography, various methods have been considered for using multiple spatial light modulators (SLM) to increase the image size. In a previous work, we used a monochrome light source for a method that located an optical system containing lens arrays and other components in front of multiple SLMs. This paper proposes a colourization technique for that system based on time division multiplexing using laser light sources of three colours (red, green, and blue). The experimental device we constructed was able to perform video playback (20 fps) in colour of full parallax holographic three-dimensional (3D) images with an image size of 63 mm and a viewing-zone angle of 5.6 degrees without losing any part of the 3D image.

Band-limited double-step Fresnel diffraction and its application to computer-generated holograms.

Double-step Fresnel diffraction (DSF) is an efficient diffraction calculation in terms of the amount of usage memory and calculation time. This paper describes band-limited DSF, which will be useful for large computer-generated holograms (CGHs) and gigapixel digital holography, mitigating the aliasing noise of the DSF. As the application, we demonstrate a CGH generation with nearly 8K × 4K pixels from texture and depth maps of a three-dimensional scene captured by a depth camera.

3D objects enlargement technique using an optical system and multiple SLMs for electronic holography.

One problem in electronic holography, which is caused by the display performance of spatial light modulators (SLM), is that the size of reconstructed 3D objects is small. Although methods for increasing the size using multiple SLMs have been considered, they typically had the problem that some parts of 3D objects were missing as a result of the gap between adjacent SLMs or 3D objects lost the vertical parallax. This paper proposes a method of resolving this problem by locating an optical system containing a lens array and other components in front of multiple SLMs. We used an optical system and 9 SLMs to construct a device equivalent to an SLM with approximately 74,600,000 pixels and used this to reconstruct 3D objects in both the horizontal and vertical parallax with an image size of 63 mm without losing any part of 3D objects.

Real-time capture and reconstruction system with multiple GPUs for a 3D live scene by a generation from 4K IP images to 8K holograms.

We developed a real-time capture and reconstruction system for three-dimensional (3D) live scenes. In previous research, we used integral photography (IP) to capture 3D images and then generated holograms from the IP images to implement a real-time reconstruction system. In this paper, we use a 4K (3,840 × 2,160) camera to capture IP images and 8K (7,680 × 4,320) liquid crystal display (LCD) panels for the reconstruction of holograms. We investigate two methods for enlarging the 4K images that were captured by integral photography to 8K images. One of the methods increases the number of pixels of each elemental image. The other increases the number of elemental images. In addition, we developed a personal computer (PC) cluster system with graphics processing units (GPUs) for the enlargement of IP images and the generation of holograms from the IP images using fast Fourier transform (FFT). We used the Compute Unified Device Architecture (CUDA) as the development environment for the GPUs. The Fast Fourier transform is performed using the CUFFT (CUDA FFT) library. As a result, we developed an integrated system for performing all processing from the capture to the reconstruction of 3D images by using these components and successfully used this system to reconstruct a 3D live scene at 12 frames per second.

Calculating the Fresnel diffraction of light from a shifted and tilted plane.

We propose a technique for calculating the diffraction of light in the Fresnel region from a plane that is the light source (source plane) to a plane at which the diffracted light is to be calculated (destination plane). When the wavefield of the source plane is described by a group of points on a grid, this technique can be used to calculate the wavefield of the group of points on a grid on the destination plane. The positions of both planes may be shifted, and the plane normal vectors of both planes may have different directions. Since a scaled Fourier transform is used for the calculation, it can be calculated faster than calculating the diffraction by a Fresnel transform at each point. This technique can be used to calculate and generate planar holograms from computer graphics data.

Simplified homography matrix for a large-scale camera array system.

We want to use a large-scale camera array system in which each camera is placed at the desired position to photograph a subject and later render images of the subject viewed from various directions or render images for a three-dimensional display. The homography matrix for each camera should be calculated in advance to correct the captured images. In the case that each camera is physically facing toward the subject as precisely as possible but the captured image still includes geometrical distortion, if the expected error in the deviations from the ideal directions is assumed to be the zero vector, the homography matrix of each camera can be easily obtained.

Fast calculation of computer-generated-hologram on AMD HD5000 series GPU and OpenCL.

In this paper, we report fast calculation of a computer-generated-hologram using a new architecture of the HD5000 series GPU (RV870) made by AMD and its new software development environment, OpenCL. Using a RV870 GPU and OpenCL, we can calculate 1,920 x 1,024 resolution of a CGH from a 3D object consisting of 1,024 points in 30 milli-seconds. The calculation speed realizes a speed approximately two times faster than that of a GPU made by NVIDIA.

One-unit system to reconstruct a 3-D movie at a video-rate via electroholography.

We have developed a one-unit system, including creating and displaying a hologram for real-time reproduction of a three-dimensional image via electroholography. We have constructed this one-unit system by connecting a special-purpose computer for holography and a special display board with a reflective liquid crystal display as a spatial light modulator. Using this one-unit system, we succeeded in reproducing a three-dimensional image composed of 10,000 points at a speed of 30 frames per second, which is the video rate in NTSC format. In addition, we were able to control a three-dimensional image in real-time using our system.

Simplified electroholographic color reconstruction system using graphics processing unit and liquid crystal display projector.

We have constructed a simple color electroholography system that has excellent cost performance. It uses a graphics processing unit (GPU) and a liquid crystal display (LCD) projector. The structure of the GPU is suitable for calculating computer-generated holograms (CGHs). The calculation speed of the GPU is approximately 1,500 times faster than that of a central processing unit. The LCD projector is an inexpensive, high-performance device for displaying CGHs. It has high-definition LCD panels for red, green and blue. Thus, it can be easily used for color electroholography. For a three-dimensional object consisting of 1,000 points, our system succeeded in real-time color holographic reconstruction at rate of 30 frames per second.