Focus issue introduction: 3D image acquisition and display: technology, perception, and applications.

This feature issue of Optics Express is organized in conjunction with the 2020 OSA conference on 3D image acquisition and display: technology, perception and applications which was held virtually in Vancouver from 22 to 26, June 2020 as part of the imaging and sensing congress 2020. This feature issue presents 29 articles based on the topics and scope of the 3D conference. This review provides a summary of these articles.

Digital halftoning method with simultaneously optimized perceptual image quality and drive current for multi-tonal electrophoretic displays.

While using digital halftoning to achieve multi-tones in a 1 bit electrophoretic display (EPD), e.g., a three-pigment chromatic EPD, the drive current is significantly increased because of frequently reversed pixel values. Aimed at this issue, this study first establishes a model that can accurately predict the drive current from image content. Next, based on the direct binary search method, a new halftoning method is proposed by constructing a combined merit function that incorporates both the perceptual image quality and the drive current. As a result, in experiments using a 13.5 in. three-pigment EPD and several test images, compared with the well-developed error-diffusion method, the proposed method produces little image quality degradation, whereas the drive current increase with respect to the minimum current of the EPD is reduced from 71.8 to 33.0 mA, for a significant reduction of 54.0%.

Dual-focal-plane augmented reality head-up display using a single picture generation unit and a single freeform mirror.

Two or more focal planes are required in augmented reality head-up displays (AR HUDs) to respectively present basic and interactive driving information to car drivers; whereas, solutions using two separated picture generation units (PGUs) with two sets of optics incur increased cost, reduced reliability, and expanded volume. To develop an AR HUD using a single PGU and a single curved mirror, we propose to set two logically separated regions on a single PGU and optically relay one of them to a new position to create two focal planes. A single freeform mirror acquired through careful optical and mechanical design optimization produces high image quality in an eyebox of 120 mm by 60 mm, simultaneously for a far image (9 m, 10° by 3°) and a near image (2.5 m, 6° by 2°). Finally, an AR HUD product with a compact volume of 8.5 L is fabricated to experimentally verify the design.

Resolution-enhanced light field displays by recombining subpixels across elemental images.

Light field displays based on integral imaging feature ultra-compact volume and freedom of the vergence-accommodation conflict for advanced virtual reality and augmented reality devices; however, they currently suffer from low visual resolution. Considering that subpixels have an intrinsically tripled resolution compared with triad pixels, this Letter develops a subpixel-level algorithm by recombining subpixels with relatively small raytracing errors from different elemental images. As a result, based on a highly accurate image formation model, the resolution of a typical system (pixel size, 7.8 µm; system thickness, 4.07 mm) is remarkably enhanced from 8.3 to 20.0 pixels per degree, for a gain of 2.41. In addition, the color breakup introduced by the chromatic subpixels is largely suppressed.

Color breakup suppression based on global dimming for field sequential color displays using edge information in images.

Considering the high complexity of local dimming backlight that is necessary to effectively suppress color breakups for field sequential color liquid crystal displays (FSC-LCDs), a global dimming-based solution is proposed. This solution involves considering that the color breakups mainly occur at object edges of an image. By introducing an algorithm to present the edge information in a single field, evaluating color breakup performances, and experimentally verifying based on a 240-Hz LCD, lighter color breakups are revealed compared with mainstream local dimming-based solutions. Therefore, the proposed solution can achieve FSC-LCDs with better performance and practicality for advanced display applications.

Hybrid light field head-mounted display using time-multiplexed liquid crystal lens array for resolution enhancement.

In recent years, head-mounted display technologies have greatly advanced. In order to overcome the accommodation-convergence conflict, light field displays reconstruct three-dimensional (3D) images with a focusing cue but sacrifice resolution. In this paper, a hybrid head-mounted display system that is based on a liquid crystal microlens array is proposed. By using a time-multiplexed method, the display signals can be divided into light field and two-dimensional (2D) modes to show comfortable 3D images with high resolution compensated by the 2D image. According to the experimental results, the prototype supports a 12.28 ppd resolution in the diagonal direction, which reaches 82% of the traditional virtual reality (VR) head-mounted display (HMD).

Digitally designed holographic optical element for light field displays.

Concave micro-mirror arrays fabricated as holographic optical elements are used in projector-based light field displays due to their see-through characteristics. The optical axes of each micro-mirror in the array are usually made parallel to each other, which simplifies the fabrication, integral image rendering, and calibration process. However, this demands that the beam from the projector be collimated and made parallel to the optical axis of each elemental micro-mirror. This requires additional collimation optics, which puts serious limitations on the size of the display. In this Letter, we propose a solution to the above issue by introducing a new method to fabricate holographic concave micro-mirror array sheets and explain how they work in detail. 3D light field reconstructions of the size 20 cm×10 cm and 6 cm in depth are achieved using a conventional projector without any collimation optics.

Long working range light field microscope with fast scanning multifocal liquid crystal microlens array.

The light field microscope has the potential of recording the 3D information of biological specimens in real time with a conventional light source. To further extend the depth of field to broaden its applications, in this paper, we proposed a multifocal high-resistance liquid crystal microlens array instead of the fixed microlens array. The developed multifocal liquid crystal microlens array can provide high quality point spread function in multiple focal lengths. By adjusting the focal length of the liquid crystal microlens array sequentially, the total working range of the light field microscope can be much extended. Furthermore, in our proposed system, the intermediate image was placed in the virtual image space of the microlens array, where the condition of the lenslets numerical aperture was considerably smaller. Consequently, a thin-cell-gap liquid crystal microlens array with fast response time can be implemented for time-multiplexed scanning.

Optimization of nano-honeycomb structures for flexible w-LEDs.

This study presents the low cost fabrication of flexible white-light-emitting diodes (w-LEDs) with nano-honeycomb-structured phosphor films. Extending the dimensions of the nano-honeycomb structures improved the color uniformity of the flexible samples, and the 950-nm pattern sample demonstrated optimal color uniformity because this nano-pattern exhibited an excellent diffusion ability owing to its pitch size. In addition to color uniformity, the use of this nano-pattern improved the luminous efficiency. The 750-nm pattern exhibited the highest luminous efficiency (235.8 lm/W), which was approximately 7% higher than that exhibited by a non-patterned phosphor film sample. Thus, flexible w-LEDs with nano-honeycomb structure optimization have great potential to be used as next-generation lighting sources.

Polychromatic SSVEP stimuli with subtle flickering adapted to brain-display interactions.

OBJECTIVE: Interactive displays armed with natural user interfaces (NUIs) will likely lead the next breakthrough in consumer electronics, and brain-computer interfaces (BCIs) are often regarded as the ultimate NUI-enabling machines to respond to human emotions and mental states. Steady-state visual evoked potentials (SSVEPs) are a commonly used BCI modality due to the ease of detection and high information transfer rates. However, the presence of flickering stimuli may cause user discomfort and can even induce migraines and seizures. With the aim of designing visual stimuli that can be embedded into video images, this study developed a novel approach to induce detectable SSVEPs using a composition of red/green/blue flickering lights. APPROACH: Based on the opponent theory of colour vision, this study used 32 Hz/40 Hz rectangular red-green or red-blue LED light pulses with a 50% duty cycle, balanced/equal luminance and 0°/180° phase shifts as the stimulating light sources and tested their efficacy in producing SSVEP responses with high signal-to-noise ratios (SNRs) while reducing the perceived flickering sensation. MAIN RESULTS: The empirical results from ten healthy subjects showed that dual-colour lights flickering at 32 Hz/40 Hz with a 50% duty cycle and 180° phase shift achieved a greater than 90% detection accuracy with little or no flickering sensation. SIGNIFICANCE: As a first step in developing an embedded SSVEP stimulus in commercial displays, this study provides a foundation for developing a combination of three primary colour flickering backlights with adjustable luminance proportions to create a subtle flickering polychromatic light that can elicit SSVEPs at the basic flickering frequency.

Multi-user 3D film on a time-multiplexed side-emission backlight system.

The desirable features for a portable 3D display include displaying 2D and 3D images without resolution degradation for multiple users, a 2D/3D switchable functionality, and, in particular, a compact volume. To produce a portable 3D display with these desirable features, we propose here a multi-user 3D film combined with a side-emission backlight system that has a directional-sequential light distribution. According to the simulation and experimental results, the multi-user 3D film successfully uses an inverted trapezoid structure to separate the rays of each light source and increases the number of observers from one to three. Additionally, the specification of the inverted trapezoid structure can be determined via equations for different designated viewing positions of the side observer and for the ratio of light intensities for the central and side observers.

Projection-type see-through holographic three-dimensional display.

Owing to the limited spatio-temporal resolution of display devices, dynamic holographic three-dimensional displays suffer from a critical trade-off between the display size and the visual angle. Here we show a projection-type holographic three-dimensional display, in which a digitally designed holographic optical element and a digital holographic projection technique are combined to increase both factors at the same time. In the experiment, the enlarged holographic image, which is twice as large as the original display device, projected on the screen of the digitally designed holographic optical element was concentrated at the target observation area so as to increase the visual angle, which is six times as large as that for a general holographic display. Because the display size and the visual angle can be designed independently, the proposed system will accelerate the adoption of holographic three-dimensional displays in industrial applications, such as digital signage, in-car head-up displays, smart-glasses and head-mounted displays.

Dual layer electrode liquid crystal lens for 2D/3D tunable endoscopy imaging system.

In this paper, we demonstrate a multi-functional liquid-crystal lens (MFLC-lens) based on dual-layer electrode design. Compared with the previous 3D endoscopes, which use double fixed lens capturing, the proposed LC lens is not only switchable between 2D and 3D modes, but also is able to adjust focus in both modes. The diameter of the MFLC-lens is only 1.42mm, which is much smaller than the available 3D endoscopes with double fixed lenses. To achieve the MFLC-lens, a high-resistance layer needs to be coated on the electrode to generate an ideal gradient electric-field distribution, which can induce a lens-like form of LC molecules. The parameters of high-resistive layer are investigated and discussed with an aim to optimize the performance of the MFLC-lens.

Dynamic integral imaging display with electrically moving array lenslet technique using liquid crystal lens.

Integral imaging (InIm) has been widely investigated for three-dimensional (3-D) display applications. Aliasing due to the lenslet arrays is one of the limitations of InIm displays. In this paper, we propose a dynamic InIm display using electrically movable liquid crystal (LC) lens array to implement the moving array lenslet technique (MALT) and to eliminate the multifacet the 3-D images. The improvement of the viewing quality of dynamic InIm display is experimentally verified.

Hexagonal liquid crystal lens array for 3D endoscopy.

A liquid crystal lens array with a hexagonal arrangement is investigated experimentally. The uniqueness of this study exists in the fact that using convex-ring electrode provides a smooth and controllable applied potential profile across the aperture to manage the phase profile. We observed considerable differences between flat electrode and convex-ring electrode; in particular the lens focal length is variable in a wider range from 2.5cm to infinity. This study presents several noteworthy characteristics such as low driving voltage; 30 µm cell gap and the lens is electrically switchable between 2D/3D modes. We demonstrate a hexagonal LC-lens array for capturing 3D images by using single sensor using integral imaging.

Security authentication using phase-encoded nanoparticle structures and polarized light.

Phase-encoded nanostructures such as quick response (QR) codes made of metallic nanoparticles are suggested to be used in security and authentication applications. We present a polarimetric optical method able to authenticate random phase-encoded QR codes. The system is illuminated using polarized light, and the QR code is encoded using a phase-only random mask. Using classification algorithms, it is possible to validate the QR code from the examination of the polarimetric signature of the speckle pattern. We used Kolmogorov-Smirnov statistical test and Support Vector Machine algorithms to authenticate the phase-encoded QR codes using polarimetric signatures.

High-resistance liquid-crystal lens array for rotatable 2D/3D autostereoscopic display.

A 2D/3D switchable and rotatable autostereoscopic display using a high-resistance liquid-crystal (Hi-R LC) lens array is investigated in this paper. Using high-resistance layers in an LC cell, a gradient electric-field distribution can be formed, which can provide a better lens-like shape of the refractive-index distribution. The advantages of the Hi-R LC lens array are its 2D/3D switchability, rotatability (in the horizontal and vertical directions), low driving voltage (~2 volts) and fast response (~0.6 second). In addition, the Hi-R LC lens array requires only a very simple fabrication process.

Habituation of steady-state visual evoked potentials in response to high-frequency polychromatic foveal visual stimulation.

In an attempt to develop safe and robust methods for monitoring migraineurs' brain states, we explores the feasibility of using white, red, green and blue LED lights flickering around their critical flicker fusion (CFF) frequencies as foveal visual stimuli for inducing steady-state visual evoked potentials (SSVEP) and causing discernible habituation trends. After comparing the habituation indices, the multi-scale entropies and the time dependent intrinsic correlations of their SSVEP signals, we reached a tentative conclusion that sharp red and white light pulses flickering barely above their CFF frequencies can replace commonly used 13Hz stimuli to effectively cause SSVEP habituation among normal subjects. Empirical results showed that consecutive short bursts of light can produce more consistent responses than a single prolonged stimulation. Since these high frequency stimuli do not run the risk of triggering migraine or seizure attacks, further tests of these stimuli on migraine patients are warranted in order to verify their effectiveness.

Three-dimensional imaging with axially distributed sensing using electronically controlled liquid crystal lens.

In this Letter, we present a three-dimensional (3D) imaging system with axially distributed sensing (ADS) using an electronically controlled liquid crystal (LC) lens. The proposed system performed an optical image acquisition by varying the focal lengths of the LC lens without mechanical movements of an image sensor. Multiple images with slightly different perspectives were experimentally recorded, and the 3D images were reconstructed according to the ray backprojection algorithm. To the best of our knowledge, this is the first report on 3D ADS using a LC lens. The proposed system is attractive for compact 3D sensing camera systems.

SNR analysis of high-frequency steady-state visual evoked potentials from the foveal and extrafoveal regions of human retina.

With brain-computer interface (BCI) applications in mind, we analyzed the amplitudes and the signal-to-noise ratios (SNR) of steady-state visual evoked potentials (SSVEP) induced in the foveal and extra-foveal regions of human retina. Eight subjects (age 20-55) have been exposed to 2° circular and 16°-18° annular visual stimulation produced by white LED lights flickering between 5Hz and 65Hz in 5Hz increments. Their EEG signals were recorded using a 64-channel NeuroScan system and analyzed using non-parametric spectral and canonical convolution techniques. Subjects' perception of flickering and their levels of comfort towards the visual stimulation were also noted. Almost all subjects showed distinctively higher SNR in their foveal SSVEP responses between 25Hz and 45Hz. They also noticed less flickering and felt more comfortable with the visual stimulation between 30Hz and 45Hz. These empirical evidences suggest that lights flashing above the critical flicker fusion rates (CFF) of human vision may be used as effective and comfortable stimuli in SSVEP BCI applications.