Information-hiding cameras: Optical concealment of object information into ordinary images.

We introduce an information-hiding camera integrated with an electronic decoder that is jointly optimized through deep learning. This system uses a diffractive optical processor, which transforms and hides input images into ordinary-looking patterns that deceive/mislead observers. This information-hiding transformation is valid for infinitely many combinations of secret messages, transformed into ordinary-looking output images through passive light-matter interactions within the diffractive processor. By processing these output patterns, an electronic decoder network accurately reconstructs the original information hidden within the deceptive output. We demonstrated our approach by designing information-hiding diffractive cameras operating under various lighting conditions and noise levels, showing their robustness. We further extended this framework to multispectral operation, allowing the concealment and decoding of multiple images at different wavelengths, performed simultaneously. The feasibility of our framework was also validated experimentally using terahertz radiation. This optical encoder-electronic decoder-based codesign provides a high speed and energy efficient information-hiding camera, offering a powerful solution for visual information security.

All-optical phase conjugation using diffractive wavefront processing.

Optical phase conjugation (OPC) is a nonlinear technique used for counteracting wavefront distortions, with applications ranging from imaging to beam focusing. Here, we present a diffractive wavefront processor to approximate all-optical phase conjugation. Leveraging deep learning, a set of diffractive layers was optimized to all-optically process an arbitrary phase-aberrated input field, producing an output field with a phase distribution that is the conjugate of the input wave. We experimentally validated this wavefront processor by 3D-fabricating diffractive layers and performing OPC on phase distortions never seen during training. Employing terahertz radiation, our diffractive processor successfully performed OPC through a shallow volume that axially spans tens of wavelengths. We also created a diffractive phase-conjugate mirror by combining deep learning-optimized diffractive layers with a standard mirror. Given its compact, passive and multi-wavelength nature, this diffractive wavefront processor can be used for various applications, e.g., turbidity suppression and aberration correction across different spectral bands.

All-optical complex field imaging using diffractive processors.

Complex field imaging, which captures both the amplitude and phase information of input optical fields or objects, can offer rich structural insights into samples, such as their absorption and refractive index distributions. However, conventional image sensors are intensity-based and inherently lack the capability to directly measure the phase distribution of a field. This limitation can be overcome using interferometric or holographic methods, often supplemented by iterative phase retrieval algorithms, leading to a considerable increase in hardware complexity and computational demand. Here, we present a complex field imager design that enables snapshot imaging of both the amplitude and quantitative phase information of input fields using an intensity-based sensor array without any digital processing. Our design utilizes successive deep learning-optimized diffractive surfaces that are structured to collectively modulate the input complex field, forming two independent imaging channels that perform amplitude-to-amplitude and phase-to-intensity transformations between the input and output planes within a compact optical design, axially spanning ~100 wavelengths. The intensity distributions of the output fields at these two channels on the sensor plane directly correspond to the amplitude and quantitative phase profiles of the input complex field, eliminating the need for any digital image reconstruction algorithms. We experimentally validated the efficacy of our complex field diffractive imager designs through 3D-printed prototypes operating at the terahertz spectrum, with the output amplitude and phase channel images closely aligning with our numerical simulations. We envision that this complex field imager will have various applications in security, biomedical imaging, sensing and material science, among others.

Broadband nonlinear modulation of incoherent light using a transparent optoelectronic neuron array.

Nonlinear optical processing of ambient natural light is highly desired for computational imaging and sensing. Strong optical nonlinear response under weak broadband incoherent light is essential for this purpose. By merging 2D transparent phototransistors (TPTs) with liquid crystal (LC) modulators, we create an optoelectronic neuron array that allows self-amplitude modulation of spatially incoherent light, achieving a large nonlinear contrast over a broad spectrum at orders-of-magnitude lower intensity than achievable in most optical nonlinear materials. We fabricated a 10,000-pixel array of optoelectronic neurons, and experimentally demonstrated an intelligent imaging system that instantly attenuates intense glares while retaining the weaker-intensity objects captured by a cellphone camera. This intelligent glare-reduction is important for various imaging applications, including autonomous driving, machine vision, and security cameras. The rapid nonlinear processing of incoherent broadband light might also find applications in optical computing, where nonlinear activation functions for ambient light conditions are highly sought.

All-optical image denoising using a diffractive visual processor.

Image denoising, one of the essential inverse problems, targets to remove noise/artifacts from input images. In general, digital image denoising algorithms, executed on computers, present latency due to several iterations implemented in, e.g., graphics processing units (GPUs). While deep learning-enabled methods can operate non-iteratively, they also introduce latency and impose a significant computational burden, leading to increased power consumption. Here, we introduce an analog diffractive image denoiser to all-optically and non-iteratively clean various forms of noise and artifacts from input images - implemented at the speed of light propagation within a thin diffractive visual processor that axially spans <250 × λ, where λ is the wavelength of light. This all-optical image denoiser comprises passive transmissive layers optimized using deep learning to physically scatter the optical modes that represent various noise features, causing them to miss the output image Field-of-View (FoV) while retaining the object features of interest. Our results show that these diffractive denoisers can efficiently remove salt and pepper noise and image rendering-related spatial artifacts from input phase or intensity images while achieving an output power efficiency of ~30-40%. We experimentally demonstrated the effectiveness of this analog denoiser architecture using a 3D-printed diffractive visual processor operating at the terahertz spectrum. Owing to their speed, power-efficiency, and minimal computational overhead, all-optical diffractive denoisers can be transformative for various image display and projection systems, including, e.g., holographic displays.

Learning diffractive optical communication around arbitrary opaque occlusions.

Free-space optical communication becomes challenging when an occlusion blocks the light path. Here, we demonstrate a direct communication scheme, passing optical information around a fully opaque, arbitrarily shaped occlusion that partially or entirely occludes the transmitter's field-of-view. In this scheme, an electronic neural network encoder and a passive, all-optical diffractive network-based decoder are jointly trained using deep learning to transfer the optical information of interest around the opaque occlusion of an arbitrary shape. Following its training, the encoder-decoder pair can communicate any arbitrary optical information around opaque occlusions, where the information decoding occurs at the speed of light propagation through passive light-matter interactions, with resilience against various unknown changes in the occlusion shape and size. We also validate this framework experimentally in the terahertz spectrum using a 3D-printed diffractive decoder. Scalable for operation in any wavelength regime, this scheme could be particularly useful in emerging high data-rate free-space communication systems.

Rapid sensing of hidden objects and defects using a single-pixel diffractive terahertz sensor.

Terahertz waves offer advantages for nondestructive detection of hidden objects/defects in materials, as they can penetrate most optically-opaque materials. However, existing terahertz inspection systems face throughput and accuracy restrictions due to their limited imaging speed and resolution. Furthermore, machine-vision-based systems using large-pixel-count imaging encounter bottlenecks due to their data storage, transmission and processing requirements. Here, we report a diffractive sensor that rapidly detects hidden defects/objects within a 3D sample using a single-pixel terahertz detector, eliminating sample scanning or image formation/processing. Leveraging deep-learning-optimized diffractive layers, this diffractive sensor can all-optically probe the 3D structural information of samples by outputting a spectrum, directly indicating the presence/absence of hidden structures or defects. We experimentally validated this framework using a single-pixel terahertz time-domain spectroscopy set-up and 3D-printed diffractive layers, successfully detecting unknown hidden defects inside silicon samples. This technique is valuable for applications including security screening, biomedical sensing and industrial quality control.

High-throughput terahertz imaging: progress and challenges.

Many exciting terahertz imaging applications, such as non-destructive evaluation, biomedical diagnosis, and security screening, have been historically limited in practical usage due to the raster-scanning requirement of imaging systems, which impose very low imaging speeds. However, recent advancements in terahertz imaging systems have greatly increased the imaging throughput and brought the promising potential of terahertz radiation from research laboratories closer to real-world applications. Here, we review the development of terahertz imaging technologies from both hardware and computational imaging perspectives. We introduce and compare different types of hardware enabling frequency-domain and time-domain imaging using various thermal, photon, and field image sensor arrays. We discuss how different imaging hardware and computational imaging algorithms provide opportunities for capturing time-of-flight, spectroscopic, phase, and intensity image data at high throughputs. Furthermore, the new prospects and challenges for the development of future high-throughput terahertz imaging systems are briefly introduced.

Universal Polarization Transformations: Spatial Programming of Polarization Scattering Matrices Using a Deep Learning-Designed Diffractive Polarization Transformer.

Controlled synthesis of optical fields having nonuniform polarization distributions presents a challenging task. Here, a universal polarization transformer is demonstrated that can synthesize a large set of arbitrarily-selected, complex-valued polarization scattering matrices between the polarization states at different positions within its input and output field-of-views (FOVs). This framework comprises 2D arrays of linear polarizers positioned between isotropic diffractive layers, each containing tens of thousands of diffractive features with optimizable transmission coefficients. After its deep learning-based training, this diffractive polarization transformer can successfully implement Ni No = 10 000 different spatially-encoded polarization scattering matrices with negligible error, where Ni and No represent the number of pixels in the input and output FOVs, respectively. This universal polarization transformation framework is experimentally validated in the terahertz spectrum by fabricating wire-grid polarizers and integrating them with 3D-printed diffractive layers to form a physical polarization transformer. Through this set-up, an all-optical polarization permutation operation of spatially-varying polarization fields is demonstrated, and distinct spatially-encoded polarization scattering matrices are simultaneously implemented between the input and output FOVs of a compact diffractive processor. This framework opens up new avenues for developing novel devices for universal polarization control and may find applications in, e.g., remote sensing, medical imaging, security, material inspection, and machine vision.

Low baseline intraspecific variation in leaf pressure-volume traits: Biophysical basis and implications for spectroscopic sensing.

Intra-specific trait variation (ITV) plays a role in processes at a wide range of scales from organs to ecosystems across climate gradients. Yet, ITV remains rarely quantified for many ecophysiological traits typically assessed for species means, such as pressure volume (PV) curve parameters including osmotic potential at full turgor and modulus of elasticity, which are important in plant water relations. We defined a baseline "reference ITV" (ITVref ) as the variation among fully exposed, mature sun leaves of replicate individuals of a given species grown in similar, well-watered conditions, representing the conservative sampling design commonly used for species-level ecophysiological traits. We hypothesized that PV parameters would show low ITVref relative to other leaf morphological traits, and that their intraspecific relationships would be similar to those previously established across species and proposed to arise from biophysical constraints. In a database of novel and published PV curves and additional leaf structural traits for 50 diverse species, we found low ITVref for PV parameters relative to other morphological traits, and strong intraspecific relationships among PV traits. Simulation modeling showed that conservative ITVref enables the use of species-mean PV parameters for scaling up from spectroscopic measurements of leaf water content to enable sensing of leaf water potential.

Data-Class-Specific All-Optical Transformations and Encryption.

Diffractive optical networks provide rich opportunities for visual computing tasks. Here, data-class-specific transformations that are all-optically performed between the input and output fields-of-view (FOVs) of a diffractive network are presented. The visual information of the objects is encoded into the amplitude (A), phase (P), or intensity (I) of the optical field at the input, which is all-optically processed by a data-class-specific diffractive network. At the output, an image sensor-array directly measures the transformed patterns, all-optically encrypted using the transformation matrices preassigned to different data classes, i.e., a separate matrix for each data class. The original input images can be recovered by applying the correct decryption key (the inverse transformation) corresponding to the matching data class, while applying any other key will lead to loss of information. All-optical class-specific transformations covering A → A, I → I, and P → I transformations using various image datasets are numerically demonstrated. The feasibility of this framework is also experimentally validated by fabricating class-specific I → I transformation diffractive networks and is successfully tested at different parts of the electromagnetic spectrum, i.e., 1550 nm and 0.75 mm wavelengths. Data-class-specific all-optical transformations provide a fast and energy-efficient method for image and data encryption, enhancing data security and privacy.

Frequency-domain terahertz spectroscopy using long-carrier-lifetime photoconductive antennas.

We present a telecommunication-compatible frequency-domain terahertz spectroscopy system realized by novel photoconductive antennas without using short-carrier-lifetime photoconductors. Built on a high-mobility InGaAs photoactive layer, these photoconductive antennas are designed with plasmonics-enhanced contact electrodes to achieve highly confined optical generation near the metal/semiconductor surface, which offers ultrafast photocarrier transport and, hence, efficient continuous-wave terahertz operation including both generation and detection. Consequently, using two plasmonic photoconductive antennas as a terahertz source and a terahertz detector, we successfully demonstrate frequency-domain spectroscopy with a dynamic range more than 95 dB and an operation bandwidth of 2.5 THz. Moreover, this novel approach to terahertz antenna design opens up a wide range of new possibilities for many different semiconductors and optical excitation wavelengths to be utilized, therefore bypassing short-carrier-lifetime photoconductors with limited availability.

Unidirectional imaging using deep learning-designed materials.

A unidirectional imager would only permit image formation along one direction, from an input field-of-view (FOV) A to an output FOV B, and in the reverse path, B â†’ A, the image formation would be blocked. We report the first demonstration of unidirectional imagers, presenting polarization-insensitive and broadband unidirectional imaging based on successive diffractive layers that are linear and isotropic. After their deep learning-based training, the resulting diffractive layers are fabricated to form a unidirectional imager. Although trained using monochromatic illumination, the diffractive unidirectional imager maintains its functionality over a large spectral band and works under broadband illumination. We experimentally validated this unidirectional imager using terahertz radiation, well matching our numerical results. We also created a wavelength-selective unidirectional imager, where two unidirectional imaging operations, in reverse directions, are multiplexed through different illumination wavelengths. Diffractive unidirectional imaging using structured materials will have numerous applications in, e.g., security, defense, telecommunications, and privacy protection.

Snapshot multispectral imaging using a diffractive optical network.

Multispectral imaging has been used for numerous applications in e.g., environmental monitoring, aerospace, defense, and biomedicine. Here, we present a diffractive optical network-based multispectral imaging system trained using deep learning to create a virtual spectral filter array at the output image field-of-view. This diffractive multispectral imager performs spatially-coherent imaging over a large spectrum, and at the same time, routes a pre-determined set of spectral channels onto an array of pixels at the output plane, converting a monochrome focal-plane array or image sensor into a multispectral imaging device without any spectral filters or image recovery algorithms. Furthermore, the spectral responsivity of this diffractive multispectral imager is not sensitive to input polarization states. Through numerical simulations, we present different diffractive network designs that achieve snapshot multispectral imaging with 4, 9 and 16 unique spectral bands within the visible spectrum, based on passive spatially-structured diffractive surfaces, with a compact design that axially spans ~72λm, where λm is the mean wavelength of the spectral band of interest. Moreover, we experimentally demonstrate a diffractive multispectral imager based on a 3D-printed diffractive network that creates at its output image plane a spatially repeating virtual spectral filter array with 2 × 2 = 4 unique bands at terahertz spectrum. Due to their compact form factor and computation-free, power-efficient and polarization-insensitive forward operation, diffractive multispectral imagers can be transformative for various imaging and sensing applications and be used at different parts of the electromagnetic spectrum where high-density and wide-area multispectral pixel arrays are not widely available.

All-optical image classification through unknown random diffusers using a single-pixel diffractive network.

Classification of an object behind a random and unknown scattering medium sets a challenging task for computational imaging and machine vision fields. Recent deep learning-based approaches demonstrated the classification of objects using diffuser-distorted patterns collected by an image sensor. These methods demand relatively large-scale computing using deep neural networks running on digital computers. Here, we present an all-optical processor to directly classify unknown objects through unknown, random phase diffusers using broadband illumination detected with a single pixel. A set of transmissive diffractive layers, optimized using deep learning, forms a physical network that all-optically maps the spatial information of an input object behind a random diffuser into the power spectrum of the output light detected through a single pixel at the output plane of the diffractive network. We numerically demonstrated the accuracy of this framework using broadband radiation to classify unknown handwritten digits through random new diffusers, never used during the training phase, and achieved a blind testing accuracy of 87.74 ± 1.12%. We also experimentally validated our single-pixel broadband diffractive network by classifying handwritten digits "0" and "1" through a random diffuser using terahertz waves and a 3D-printed diffractive network. This single-pixel all-optical object classification system through random diffusers is based on passive diffractive layers that process broadband input light and can operate at any part of the electromagnetic spectrum by simply scaling the diffractive features proportional to the wavelength range of interest. These results have various potential applications in, e.g., biomedical imaging, security, robotics, and autonomous driving.

Super-resolution image display using diffractive decoders.

High-resolution image projection over a large field of view (FOV) is hindered by the restricted space-bandwidth product (SBP) of wavefront modulators. We report a deep learning-enabled diffractive display based on a jointly trained pair of an electronic encoder and a diffractive decoder to synthesize/project super-resolved images using low-resolution wavefront modulators. The digital encoder rapidly preprocesses the high-resolution images so that their spatial information is encoded into low-resolution patterns, projected via a low SBP wavefront modulator. The diffractive decoder processes these low-resolution patterns using transmissive layers structured using deep learning to all-optically synthesize/project super-resolved images at its output FOV. This diffractive image display can achieve a super-resolution factor of ~4, increasing the SBP by ~16-fold. We experimentally validate its success using 3D-printed diffractive decoders that operate at the terahertz spectrum. This diffractive image decoder can be scaled to operate at visible wavelengths and used to design large SBP displays that are compact, low power, and computationally efficient.

Coherent terahertz radiation with 2.8-octave tunability through chip-scale photomixed microresonator optical parametric oscillation.

High-spectral-purity frequency-agile room-temperature sources in the terahertz spectrum are foundational elements for imaging, sensing, metrology, and communications. Here we present a chip-scale optical parametric oscillator based on an integrated nonlinear microresonator that provides broadly tunable single-frequency and multi-frequency oscillators in the terahertz regime. Through optical-to-terahertz down-conversion using a plasmonic nanoantenna array, coherent terahertz radiation spanning 2.8-octaves is achieved from 330 GHz to 2.3 THz, with ≈20 GHz cavity-mode-limited frequency tuning step and ≈10 MHz intracavity-mode continuous frequency tuning range at each step. By controlling the microresonator intracavity power and pump-resonance detuning, tunable multi-frequency terahertz oscillators are also realized. Furthermore, by stabilizing the microresonator pump power and wavelength, sub-100 Hz linewidth of the terahertz radiation with 10-15 residual frequency instability is demonstrated. The room-temperature generation of both single-frequency, frequency-agile terahertz radiation and multi-frequency terahertz oscillators in the chip-scale platform offers unique capabilities in metrology, sensing, imaging and communications.

High-power terahertz pulse generation from bias-free nanoantennas on graded composition InGaAs structures.

We present a bias-free photoconductive emitter that uses an array of nanoantennas on an InGaAs layer with a linearly graded Indium composition. The graded InGaAs structure creates a built-in electric field that extends through the entire photoconductive active region, enabling the efficient drift of the photo-generated electrons to the nanoantennas. The nanoantenna geometry is chosen so that surface plasmon waves are excited in response to a 1550 nm optical pump to maximize photo-generated carrier concentration near the nanoantennas, where the built-in electric field strength is maximized. With the combination of the plasmonic enhancement and built-in electric field, high-power terahertz pulses are generated without using any external bias voltage. We demonstrate the generation of terahertz pulses with 860 µW average power at an average optical pump power of 900 mW, exhibiting the highest radiation power compared to previously demonstrated telecommunication-compatible terahertz pulse emitters.

Wavelength conversion through plasmon-coupled surface states.

Surface states generally degrade semiconductor device performance by raising the charge injection barrier height, introducing localized trap states, inducing surface leakage current, and altering the electric potential. We show that the giant built-in electric field created by the surface states can be harnessed to enable passive wavelength conversion without utilizing any nonlinear optical phenomena. Photo-excited surface plasmons are coupled to the surface states to generate an electron gas, which is routed to a nanoantenna array through the giant electric field created by the surface states. The induced current on the nanoantennas, which contains mixing product of different optical frequency components, generates radiation at the beat frequencies of the incident photons. We utilize the functionalities of plasmon-coupled surface states to demonstrate passive wavelength conversion of nanojoule optical pulses at a 1550 nm center wavelength to terahertz regime with efficiencies that exceed nonlinear optical methods by 4-orders of magnitude.