High forward thrust metasurface beam-riding sail.

The radiation pressure force and torque on a one-dimensional bi-grating composed of a Si-SiO2 high contrast binary metagrating is analyzed for the purpose of stable beam riding whereupon a high power laser having an expanding Gaussian irradiance distribution propels the grating in outer space, free from gravitational forces. The binary metagrating structure has been simultaneously optimized to afford high forward thrust, and corrective restoring forces and torques in the event of small linear and angular disturbances. We demonstrate that stability may be enhanced at the expense of forward thrust. The validity of our metamaterial findings is reinforced owing to good agreements between finite-difference time-domain and finite element numerical methods. To reduce mass and enhance forward acceleration this laser-driven sail was designed to be free of a stabilizing boom.

Simultaneous spectral differentiation of multiple fluorophores in super-resolution imaging using a glass phase plate.

By engineering the point-spread function (PSF) of single molecules, different fluorophore species can be imaged simultaneously and distinguished by their unique PSF patterns. Here, we insert a silicon-dioxide phase plate at the Fourier plane of the detection path of a wide-field fluorescence microscope to produce distinguishable PSFs (X-PSFs) at different wavelengths. We demonstrate that the resulting PSFs can be localized spatially and spectrally using a maximum-likelihood estimation algorithm and can be utilized for hyper-spectral super-resolution microscopy of biological samples. We produced superresolution images of fixed U2OS cells using X-PSFs for dSTORM imaging with simultaneous illumination of up to three fluorophore species. The species were distinguished only by the PSF pattern. We achieved ∼21-nm lateral localization precision (FWHM) and ∼17-nm axial precision (FWHM) with an average of 1,800 - 3,500 photons per PSF and a background as high as 130 - 400 photons per pixel. The modified PSF distinguished fluorescent probes with ∼80 nm separation between spectral peaks.

Multifocal multilevel diffractive lens by wavelength multiplexing.

Flat lenses with focal length tunability can enable the development of highly integrated imaging systems. This work explores machine learning to inverse design a multifocal multilevel diffractive lens (MMDL) by wavelength multiplexing. The MMDL output is multiplexed in three color channels, red (650 nm), green (550 nm), and blue (450 nm), to achieve varied focal lengths of 4 mm, 20 mm, and 40 mm at these three color channels, respectively. The focal lengths of the MMDL scale significantly with the wavelength in contrast to conventional diffractive lenses. The MMDL consists of concentric rings with equal widths and varied heights. The machine learning method is utilized to optimize the height of each concentric ring to obtain the desired phase distribution so as to achieve varied focal lengths multiplexed by wavelengths. The designed MMDL is fabricated through a direct-write laser lithography system with gray-scale exposure. The demonstrated singlet lens is miniature and polarization insensitive, and thus can potentially be applied in integrated optical imaging systems to achieve zooming functions.

Machine learning enables the design of a bidirectional focusing diffractive lens.

Machine learning can efficiently empower the inverse design of cascaded diffractive optical elements. In this work, we explore the inverse design of a bidirectional focusing diffractive lens in a cascaded configuration through the diffractive optical neural network (DONN) machine learning method. The bidirectional focusing diffractive lens consists of two on-axially cascaded multi-level diffractive lenses. Each lens consists of concentric rings with equal widths and varying heights. The height of each concentric ring is optimized as part of the design algorithm. The diffractive lens has a focal length f+ as light propagates in the forward (Z+) direction. As light propagates in the backward (Z-) direction, the focal length changes to f-. The designed lens is fabricated through a two-photon polymerization 3D printing technique. The proposed design is polarization insensitive and miniature and can be readily applied in future functional optical imaging systems.

Ultra-compact synthesis of space-time wave packets.

Space-time wave packets (STWPs) are pulsed fields in which a strictly prescribed association between the spatial and temporal frequencies yields surprising and useful behavior. However, STWPs to date have been synthesized using bulky free-space optical systems that require precise alignment. We describe a compact system that makes use of a novel optical component: a chirped volume Bragg grating that is rotated by 45° with respect to the plane-parallel device facets. By virtue of this grating's unique structure, cascaded gratings resolve and recombine the spectrum without free-space propagation or collimation. We produce STWPs by placing a phase plate that spatially modulates the resolved spectrum between such cascaded gratings, with a device volume of 25 × 25 × 8 mm3, which is orders-of-magnitude smaller than previous arrangements.

Multilevel diffractive lens in the MWIR with extended depth-of-focus and wide field-of-view.

Optics in the mid-wave-infra-red (MWIR) band are generally heavy, thick and expensive. Here, we demonstrate multi-level diffractive lenses; one designed using inverse design and another using the conventional propagation phase (the Fresnel zone plate or FZP) with diameter = 25 mm and focal length = 25 mm operating at λ=4µm. We fabricated the lenses by optical lithography and compared their performance. We show that the inverse-designed MDL achieves larger depth-of-focus and better off-axis performance when compared to the FZP at the expense of larger spot size and reduced focusing efficiency. Both lenses are flat with thickness ≤0.5 mm and weigh ≤3.63 g, which are far smaller than their conventional refractive counterparts.

Structural basis for Fc receptor recognition of immunoglobulin M.

Immunoglobulin Fc receptors are cell surface transmembrane proteins that bind to the Fc constant region of antibodies and play critical roles in regulating immune responses by activation of immune cells, clearance of immune complexes and regulation of antibody production. FcµR is the immunoglobulin M (IgM) antibody isotype-specific Fc receptor involved in the survival and activation of B cells. Here we reveal eight binding sites for the human FcµR immunoglobulin domain on the IgM pentamer by cryogenic electron microscopy. One of the sites overlaps with the binding site for the polymeric immunoglobulin receptor (pIgR), but a different mode of FcµR binding explains its antibody isotype specificity. Variation in FcµR binding sites and their occupancy reflects the asymmetry of the IgM pentameric core and the versatility of FcµR binding. The complex explains engagement with polymeric serum IgM and the monomeric IgM B-cell receptor (BCR).

Overcoming the field-of-view to diameter trade-off in microendoscopy via computational optrode-array microscopy.

High-resolution microscopy of deep tissue with large field-of-view (FOV) is critical for elucidating organization of cellular structures in plant biology. Microscopy with an implanted probe offers an effective solution. However, there exists a fundamental trade-off between the FOV and probe diameter arising from aberrations inherent in conventional imaging optics (typically, FOV < 30% of diameter). Here, we demonstrate the use of microfabricated non-imaging probes (optrodes) that when combined with a trained machine-learning algorithm is able to achieve FOV of 1x to 5x the probe diameter. Further increase in FOV is achieved by using multiple optrodes in parallel. With a 1 × 2 optrode array, we demonstrate imaging of fluorescent beads (including 30 FPS video), stained plant stem sections and stained living stems. Our demonstration lays the foundation for fast, high-resolution microscopy with large FOV in deep tissue via microfabricated non-imaging probes and advanced machine learning.

Cryomicroscopy reveals the structural basis for a flexible hinge motion in the immunoglobulin M pentamer.

Immunoglobulin M (IgM) is the most ancient of the five isotypes of immunoglobulin (Ig) molecules and serves as the first line of defence against pathogens. Here, we use cryo-EM to image the structure of the human full-length IgM pentamer, revealing antigen binding domains flexibly attached to the asymmetric and rigid core formed by the Cµ4 and Cµ3 constant regions and the J-chain. A hinge is located at the Cµ3/Cµ2 domain interface, allowing Fabs and Cµ2 to pivot as a unit both in-plane and out-of-plane. This motion is different from that observed in IgG and IgA, where the two Fab arms are able to swing independently. A biased orientation of one pair of Fab arms results from asymmetry in the constant domain (Cµ3) at the IgM subunit interacting most extensively with the J-chain. This may influence the multi-valent binding to surface-associated antigens and complement pathway activation. By comparison, the structure of the Fc fragment in the IgM monomer is similar to that of the pentamer, but is more dynamic in the Cµ4 domain.

Visible and near-infrared programmable multi-level diffractive lenses with phase change material Sb2S3.

In this paper, we discuss flat programmable multi-level diffractive lenses (PMDL) enabled by phase change materials working in the near-infrared and visible ranges. The high real part refractive index contrast (Δn ∼ 0.6) of Sb2S3 between amorphous and crystalline states, and extremely low losses in the near-infrared, enable the PMDL to effectively shift the lens focus when the phase of the material is altered between its crystalline and amorphous states. In the visible band, although losses can become significant as the wavelength is reduced, the lenses can still provide good performance as a result of their relatively small thickness (∼ 1.5λ to 3λ). The PMDL consists of Sb2S3 concentric rings with equal width and varying heights embedded in a glass substrate. The height of each concentric ring was optimized by a modified direct binary search algorithm. The proposed designs show the possibility of realizing programmable lenses at design wavelengths from the near-infrared (850 nm) up to the blue (450 nm) through engineering PMDLs with Sb2S3. Operation at these short wavelengths, to the best of our knowledge, has not been studied so far in reconfigurable lenses with phase-change materials. Therefore, our results open a wider range of applications for phase-change materials, and show the prospect of Sb2S3 for such applications. The proposed lenses are polarization insensitive and can have the potential to be applied in dual-functionality devices, optical imaging, and biomedical science.

Scan-less machine-learning-enabled incoherent microscopy for minimally-invasive deep-brain imaging.

Deep-brain microscopy is strongly limited by the size of the imaging probe, both in terms of achievable resolution and potential trauma due to surgery. Here, we show that a segment of an ultra-thin multi-mode fiber (cannula) can replace the bulky microscope objective inside the brain. By creating a self-consistent deep neural network that is trained to reconstruct anthropocentric images from the raw signal transported by the cannula, we demonstrate a single-cell resolution (< 10µm), depth sectioning resolution of 40 µm, and field of view of 200 µm, all with green-fluorescent-protein labelled neurons imaged at depths as large as 1.4 mm from the brain surface. Since ground-truth images at these depths are challenging to obtain in vivo, we propose a novel ensemble method that averages the reconstructed images from disparate deep-neural-network architectures. Finally, we demonstrate dynamic imaging of moving GCaMp-labelled C. elegans worms. Our approach dramatically simplifies deep-brain microscopy.

Broadband point-spread function engineering via a freeform diffractive microlens array.

We utilized inverse design to engineer the point-spread function (PSF) of a low-f-number, freeform diffractive microlens in an array, so as to enable extended depth of focus (DOF). Each square microlens of side 69 µm and focal length 40 µm (in a polymer film, n∼1.47) generated a square PSF of side ∼10 µm that was achromatic over the visible band (450 to 750 nm), and also exhibited an extended DOF of ∼ ± 2 µm. The microlens has a geometric f/# (focal length divided by aperture size) of 0.58 in the polymer material (0.39 in air). Since each microlens is a square, the microlens array (MLA) can achieve 100% fill factor. By placing this microlens array (MLA) directly on a high-resolution print, we demonstrated integral imaging with applications in physical security. The extended DOF preserves the optical effects even with expected film-thickness variations, thereby increasing robustness in practical applications. Since these multi-level diffractive MLAs are fabricated using UV-nanoimprint lithography, they have the potential for low-cost large volume manufacturing.

Bijective-constrained cycle-consistent deep learning for optics-free imaging and classification.

Many deep learning approaches to solve computational imaging problems have proven successful through relying solely on the data. However, when applied to the raw output of a bare (optics-free) image sensor, these methods fail to reconstruct target images that are structurally diverse. In this work we propose a self-consistent supervised model that learns not only the inverse, but also the forward model to better constrain the predictions through encouraging the network to model the ideal bijective imaging system. To do this, we employ cycle consistency alongside traditional reconstruction losses, both of which we show are needed for incoherent optics-free image reconstruction. By eliminating all optics, we demonstrate imaging with the thinnest camera possible.

Computational microscopy for fast widefield deep-tissue fluorescence imaging using a commercial dual-cannula probe.

A solid-glass cannula serves as a micro-endoscope that can deliver excitation light deep inside tissue while also collecting emitted fluorescence. Then, we utilize deep neural networks to reconstruct images from the collected intensity distributions. By using a commercially available dual-cannula probe, and training a separate deep neural network for each cannula, we effectively double the field of view compared to prior work. We demonstrated ex vivo imaging of fluorescent beads and brain slices and in vivo imaging from whole brains. We clearly resolved 4 µm beads, with FOV from each cannula of 0.2 mm (diameter), and produced images from a depth of ~1.2 mm in the whole brain, currently limited primarily by the labeling. Since no scanning is required, fast widefield fluorescence imaging limited primarily by the brightness of the fluorophores, collection efficiency of our system, and the frame rate of the camera becomes possible.

Parametric control of a diffractive axicon beam rider.

A laser beam rider is a large-scale optical structure designed so that it is attracted toward the optical axis, while also affording forward propulsion via radiation pressure along the beam path. Such structures form the basis of laser-driven light sails. Experimental measurements are described whereby a thin diffractive axicon film is shown to exhibit a natural restoring force when its axis is displaced from the optical axis. This effect is attributed to the optical momentum change of diffracted light. Whereas continuous illumination supports harmonic motion, modulated illumination is shown to support both parametric gain and parametric damping. The 12.7µm period photopolymer axicon grating was suspended in a vacuum torsion oscillator and irradiated with a 1.5 W near-infrared laser modulated at a period of 38 s.

Monolithic all-silicon flat lens for broadband LWIR imaging.

We designed, fabricated, and characterized a flat multi-level diffractive lens comprised of only silicon with diameter=15.2mm, focal length=19mm, numerical aperture of 0.371, and operating over the long-wave infrared (LWIR) spectrum=8µm to 14 µm. We experimentally demonstrated a field of view of 46°, depth of focus >5mm, and wavelength-averaged Strehl ratio of 0.46. All of these metrics were comparable to those of a conventional refractive lens. The active device thickness is only 8 µm, and its weight (including the silicon substrate) is less than 0.2 g.

Imaging from the visible to the longwave infrared wavelengths via an inverse-designed flat lens.

It is generally assumed that correcting chromatic aberrations in imaging requires multiple optical elements. Here, we show that by allowing the phase in the image plane to be a free parameter, it is possible to correct chromatic variation of focal length over an extremely large bandwidth, from the visible (Vis) to the longwave infrared (LWIR) wavelengths using a single diffractive surface, i.e., a flat lens. Specifically, we designed, fabricated and characterized a flat, multi-level diffractive lens (MDL) with a thickness of ≤ 10µm, diameter of ∼1mm, and focal length of 18mm, which was constant over the operating bandwidth of λ=0.45µm (blue) to 15µm (LWIR). We experimentally characterized the point-spread functions, aberrations and imaging performance of cameras comprised of this MDL and appropriate image sensors for λ=0.45µm to 11µm. We further show using simulations that such extreme achromatic MDLs can be achieved even at high numerical apertures (NA=0.81). By drastically increasing the operating bandwidth and eliminating several refractive lenses, our approach enables thinner, lighter and simpler imaging systems.

Needle-based deep-neural-network camera.

We experimentally demonstrate a camera whose primary optic is a cannula/needle (diameter=0.22mm and length=12.5mm) that acts as a light pipe transporting light intensity from an object plane (35 cm away) to its opposite end. Deep neural networks (DNNs) are used to reconstruct color and grayscale images with a field of view of 18° and angular resolution of ∼0.4∘. We showed a large effective demagnification of 127×. Most interestingly, we showed that such a camera could achieve close to diffraction-limited performance with an effective numerical aperture of 0.045, depth of focus ∼16µm, and resolution close to the sensor pixel size (3.2 µm). When trained on images with depth information, the DNN can create depth maps. Finally, we show DNN-based classification of the EMNIST dataset before and after image reconstructions. The former could be useful for imaging with enhanced privacy.

Learning Wavefront Coding for Extended Depth of Field Imaging.

Depth of field is an important factor of imaging systems that highly affects the quality of the acquired spatial information. Extended depth of field (EDoF) imaging is a challenging ill-posed problem and has been extensively addressed in the literature. We propose a computational imaging approach for EDoF, where we employ wavefront coding via a diffractive optical element (DOE) and we achieve deblurring through a convolutional neural network. Thanks to the end-to-end differentiable modeling of optical image formation and computational post-processing, we jointly optimize the optical design, i.e., DOE, and the deblurring through standard gradient descent methods. Based on the properties of the underlying refractive lens and the desired EDoF range, we provide an analytical expression for the search space of the DOE, which is instrumental in the convergence of the end-to-end network. We achieve superior EDoF imaging performance compared to the state of the art, where we demonstrate results with minimal artifacts in various scenarios, including deep 3D scenes and broadband imaging.

Optics-free imaging of complex, non-sparse and color QR-codes with deep neural networks.

We demonstrate optics-free imaging of complex color and monochrome QR-codes using a bare image sensor and trained artificial neural networks (ANNs). The ANN is trained to interpret the raw sensor data for human visualization. The image sensor is placed at a specified gap (1mm, 5mm and 10mm) from the QR code. We studied the robustness of our approach by experimentally testing the output of the ANNs with system perturbations of this gap, and the translational and rotational alignments of the QR code to the image sensor. Our demonstration opens us the possibility of using completely optics-free, non-anthropocentric cameras for application-specific imaging of complex, non-sparse objects.