Quantitative comparison of the computational complexity of optical, digital and hybrid neural network architectures for image classification tasks.

By implementing neuromorphic paradigms in processing visual information, machine learning became crucial in an ever-increasing number of applications of our everyday lives, ever more performing but also computationally demanding. While a pre-processing of the information passively in the optical domain, before optical-electronic conversion, can reduce the computational requirements for a machine learning task, a comprehensive analysis of computational requirements for hybrid optical-digital neural networks is thus far missing. In this work we critically compare and analyze the performance of different optical, digital and hybrid neural network architectures with respect to their classification accuracy and computational requirements for analog classification tasks of different complexity. We show that certain hybrid architectures exhibit a reduction of computational requirements of a factor >10 while maintaining their performance. This may inspire a new generation of co-designed optical-digital neural network architectures, aimed for applications that require low power consumption like remote sensing devices.

Direct retrieval of Zernike-based pupil functions using integrated diffractive deep neural networks.

Retrieving the pupil phase of a beam path is a central problem for optical systems across scales, from telescopes, where the phase information allows for aberration correction, to the imaging of near-transparent biological samples in phase contrast microscopy. Current phase retrieval schemes rely on complex digital algorithms that process data acquired from precise wavefront sensors, reconstructing the optical phase information at great expense of computational resources. Here, we present a compact optical-electronic module based on multi-layered diffractive neural networks printed on imaging sensors, capable of directly retrieving Zernike-based pupil phase distributions from an incident point spread function. We demonstrate this concept numerically and experimentally, showing the direct pupil phase retrieval of superpositions of the first 14 Zernike polynomials. The integrability of the diffractive elements with CMOS sensors shows the potential for the direct extraction of the pupil phase information from a detector module without additional digital post-processing.

Neuron-Inspired Steiner Tree Networks for 3D Low-Density Metastructures.

Three-dimensional (3D) micro-and nanostructures have played an important role in topological photonics, microfluidics, acoustic, and mechanical engineering. Incorporating biomimetic geometries into the design of metastructures has created low-density metamaterials with extraordinary physical and photonic properties. However, the use of surface-based biomimetic geometries restricts the freedom to tune the relative density, mechanical strength, and topological phase. The Steiner tree method inspired by the feature of the shortest connection distance in biological neural networks is applied, to create 3D metastructures and, through two-photon nanolithography, neuron-inspired 3D structures with nanoscale features are successfully achieved. Two solutions are presented to the 3D Steiner tree problem: the Steiner tree networks (STNs) and the twisted Steiner tree networks (T-STNs). STNs and T-STNs possess a lower density than surface-based metamaterials and that T-STNs have Young's modulus enhanced by 20% than the STNs. Through the analysis of the space groups and symmetries, a topological nontrivial Dirac-like conical dispersion in the T-STNs is predicted, and the results are based on calculations with true predictive power and readily realizable from microwave to optical frequencies. The neuron-inspired 3D metastructures opens a new space for designing low-density metamaterials and topological photonics with extraordinary properties triggered by a twisting degree-of-freedom.

Nanoprinted high-neuron-density optical linear perceptrons performing near-infrared inference on a CMOS chip.

Optical machine learning has emerged as an important research area that, by leveraging the advantages inherent to optical signals, such as parallelism and high speed, paves the way for a future where optical hardware can process data at the speed of light. In this work, we present such optical devices for data processing in the form of single-layer nanoscale holographic perceptrons trained to perform optical inference tasks. We experimentally show the functionality of these passive optical devices in the example of decryptors trained to perform optical inference of single or whole classes of keys through symmetric and asymmetric decryption. The decryptors, designed for operation in the near-infrared region, are nanoprinted on complementary metal-oxide-semiconductor chips by galvo-dithered two-photon nanolithography with axial nanostepping of 10 nm1,2, achieving a neuron density of >500 million neurons per square centimetre. This power-efficient commixture of machine learning and on-chip integration may have a transformative impact on optical decryption3, sensing4, medical diagnostics5 and computing6,7.