Single-shot optical neural network.

Analog optical and electronic hardware has emerged as a promising alternative to digital electronics to improve the efficiency of deep neural networks (DNNs). However, previous work has been limited in scalability (input vector length K ≈ 100 elements) or has required nonstandard DNN models and retraining, hindering widespread adoption. Here, we present an analog, CMOS-compatible DNN processor that uses free-space optics to reconfigurably distribute an input vector and optoelectronics for static, updatable weighting and the nonlinearity-with K ≈ 1000 and beyond. We demonstrate single-shot-per-layer classification of the MNIST, Fashion-MNIST, and QuickDraw datasets with standard fully connected DNNs, achieving respective accuracies of 95.6, 83.3, and 79.0% without preprocessing or retraining. We also experimentally determine the fundamental upper bound on throughput (∼0.9 exaMAC/s), set by the maximum optical bandwidth before substantial increase in error. Our combination of wide spectral and spatial bandwidths enables highly efficient computing for next-generation DNNs.

Delocalized photonic deep learning on the internet's edge.

Advanced machine learning models are currently impossible to run on edge devices such as smart sensors and unmanned aerial vehicles owing to constraints on power, processing, and memory. We introduce an approach to machine learning inference based on delocalized analog processing across networks. In this approach, named Netcast, cloud-based "smart transceivers" stream weight data to edge devices, enabling ultraefficient photonic inference. We demonstrate image recognition at ultralow optical energy of 40 attojoules per multiply (<1 photon per multiply) at 98.8% (93%) classification accuracy. We reproduce this performance in a Boston-area field trial over 86 kilometers of deployed optical fiber, wavelength multiplexed over 3 terahertz of optical bandwidth. Netcast allows milliwatt-class edge devices with minimal memory and processing to compute at teraFLOPS rates reserved for high-power (>100 watts) cloud computers.

Ultrahigh resolution spectral-domain optical coherence tomography using the 1000-1600†nm spectral band.

Ultrahigh resolution optical coherence tomography (UHR-OCT) can image microscopic features that are not visible with the standard OCT resolution of 5-15 µm. In previous studies, high-speed UHR-OCT has been accomplished within the visible (VIS) and near-infrared (NIR-I) spectral ranges, specifically within 550-950 nm. Here, we present a spectral domain UHR-OCT system operating in a short-wavelength infrared (SWIR) range from 1000 to 1600 nm using a supercontinuum light source and an InGaAs-based spectrometer. We obtained an axial resolution of 2.6 µm in air, the highest ever recorded in the SWIR window to our knowledge, with deeper penetration into tissues than VIS or NIR-I light. We demonstrate imaging of conduction fibers of the left bundle branch in freshly excised porcine hearts. These results suggest a potential for deep-penetration, ultrahigh resolution OCT in intraoperative applications.

Label-free histological imaging of tissues using Brillouin light scattering contrast.

Brillouin light scattering offers a unique label-free approach to measure biomechanical properties non-invasively. While this technique is used in biomechanical analysis of cells and tissues, its potential for visualizing structural features of tissues based on the biomechanical contrast has not been much exploited. Here, we present high-resolution Brillouin microscopy images of four basic tissue types: muscular, connective, epithelial, and nervous tissues. The Brillouin contrast distinguishes between muscle fiber cells and endomysium in skeletal muscle and reveals chondrocytes along with spatially varying stiffness of the extracellular matrix in articular cartilage. The hydration-sensitive contrast can visualize the stratum corneum, epidermis, and dermis in the skin epithelium. In brain tissues, the Brillouin images show the mechanical heterogeneity across the cortex and deeper regions. This work demonstrates the versatility of using the Brillouin shift as histological contrast for examining intact tissue substructures via longitudinal modulus without the need for laborious tissue processing steps.

Freely scalable and reconfigurable optical hardware for deep learning.

As deep neural network (DNN) models grow ever-larger, they can achieve higher accuracy and solve more complex problems. This trend has been enabled by an increase in available compute power; however, efforts to continue to scale electronic processors are impeded by the costs of communication, thermal management, power delivery and clocking. To improve scalability, we propose a digital optical neural network (DONN) with intralayer optical interconnects and reconfigurable input values. The path-length-independence of optical energy consumption enables information locality between a transmitter and a large number of arbitrarily arranged receivers, which allows greater flexibility in architecture design to circumvent scaling limitations. In a proof-of-concept experiment, we demonstrate optical multicast in the classification of 500 MNIST images with a 3-layer, fully-connected network. We also analyze the energy consumption of the DONN and find that digital optical data transfer is beneficial over electronics when the spacing of computational units is on the order of [Formula: see text]m.

Intraoperative brain cancer detection with Raman spectroscopy in humans.

Cancers are often impossible to visually distinguish from normal tissue. This is critical for brain cancer where residual invasive cancer cells frequently remain after surgery, leading to disease recurrence and a negative impact on overall survival. No preoperative or intraoperative technology exists to identify all cancer cells that have invaded normal brain. To address this problem, we developed a handheld contact Raman spectroscopy probe technique for live, local detection of cancer cells in the human brain. Using this probe intraoperatively, we were able to accurately differentiate normal brain from dense cancer and normal brain invaded by cancer cells, with a sensitivity of 93% and a specificity of 91%. This Raman-based probe enabled detection of the previously undetectable diffusely invasive brain cancer cells at cellular resolution in patients with grade 2 to 4 gliomas. This intraoperative technology may therefore be able to classify cell populations in real time, making it an ideal guide for surgical resection and decision-making.