AI-driven projection tomography with multicore fibre-optic cell rotation.

Optical tomography has emerged as a non-invasive imaging method, providing three-dimensional insights into subcellular structures and thereby enabling a deeper understanding of cellular functions, interactions, and processes. Conventional optical tomography methods are constrained by a limited illumination scanning range, leading to anisotropic resolution and incomplete imaging of cellular structures. To overcome this problem, we employ a compact multi-core fibre-optic cell rotator system that facilitates precise optical manipulation of cells within a microfluidic chip, achieving full-angle projection tomography with isotropic resolution. Moreover, we demonstrate an AI-driven tomographic reconstruction workflow, which can be a paradigm shift from conventional computational methods, often demanding manual processing, to a fully autonomous process. The performance of the proposed cell rotation tomography approach is validated through the three-dimensional reconstruction of cell phantoms and HL60 human cancer cells. The versatility of this learning-based tomographic reconstruction workflow paves the way for its broad application across diverse tomographic imaging modalities, including but not limited to flow cytometry tomography and acoustic rotation tomography. Therefore, this AI-driven approach can propel advancements in cell biology, aiding in the inception of pioneering therapeutics, and augmenting early-stage cancer diagnostics.

Calibration-free quantitative phase imaging in multi-core fiber endoscopes using end-to-end deep learning.

Quantitative phase imaging (QPI) through multi-core fibers (MCFs) has been an emerging in vivo label-free endoscopic imaging modality with minimal invasiveness. However, the computational demands of conventional iterative phase retrieval algorithms have limited their real-time imaging potential. We demonstrate a learning-based MCF phase imaging method that significantly reduced the phase reconstruction time to 5.5 ms, enabling video-rate imaging at 181 fps. Moreover, we introduce an innovative optical system that automatically generated the first, to the best of our knowledge, open-source dataset tailored for MCF phase imaging, comprising 50,176 paired speckles and phase images. Our trained deep neural network (DNN) demonstrates a robust phase reconstruction performance in experiments with a mean fidelity of up to 99.8%. Such an efficient fiber phase imaging approach can broaden the applications of QPI in hard-to-reach areas.

Fully refractive telecentric f-theta microscope based on adaptive elements for 3D raster scanning of biological tissues.

Various techniques in microscopy are based on point-wise acquisition, which provides advantages in acquiring sectioned images, for example in confocal or two-photon microscopy. The advantages come along with the need to perform three-dimensional scanning, which is often realized by mechanical movement achieved by stage-scanning or piezo-based scanning in the axial direction. Lateral scanning often employs galvo-mirrors, leading to a reflective setup and hence to a folded beam path. In this paper, we introduce a fully refractive microscope capable of three-dimensional scanning, which employs the combination of an adaptive lens, an adaptive prism, and a tailored telecentric f-theta objective. Our results show that this microscope is capable to perform flexible three-dimensional scanning, with low scan-induced aberrations, at a uniform resolution over a large tuning range of X=Y=6300 µ m and Z=480 µ m with only transmissive components. We demonstrate the capabilities at the example of volumetric measurements on the transgenic fluorescence of the thyroid of a zebrafish embryo and mixed pollen grains. This is the first step towards flexible aberration-free volumetric smart microscopy of three-dimensional samples like embryos and organoids, which could be exploited for the demands in both lateral and axial dimensions in biomedical samples without compromising image quality.

Multimode Optical Interconnects on Silicon Interposer Enable Confidential Hardware-to-Hardware Communication.

Following Moore's law, the density of integrated circuits is increasing in all dimensions, for instance, in 3D stacked chip networks. Amongst other electro-optic solutions, multimode optical interconnects on a silicon interposer promise to enable high throughput for modern hardware platforms in a restricted space. Such integrated architectures require confidential communication between multiple chips as a key factor for high-performance infrastructures in the 5G era and beyond. Physical layer security is an approach providing information theoretic security among network participants, exploiting the uniqueness of the data channel. We experimentally project orthogonal and non-orthogonal symbols through 380 µm long multimode on-chip interconnects by wavefront shaping. These interconnects are investigated for their uniqueness by repeating these experiments across multiple channels and samples. We show that the detected speckle patterns resulting from modal crosstalk can be recognized by training a deep neural network, which is used to transform these patterns into a corresponding readable output. The results showcase the feasibility of applying physical layer security to multimode interconnects on silicon interposers for confidential optical 3D chip networks.

Chromatic aberration correction employing reinforcement learning.

In fluorescence microscopy a multitude of labels are used that bind to different structures of biological samples. These often require excitation at different wavelengths and lead to different emission wavelengths. The presence of different wavelengths can induce chromatic aberrations, both in the optical system and induced by the sample. These lead to a detuning of the optical system, as the focal positions shift in a wavelength dependent manner and finally to a decrease in the spatial resolution. We present the correction of chromatic aberrations by using an electrical tunable achromatic lens driven by reinforcement learning. The tunable achromatic lens consists of two lens chambers filled with different optical oils and sealed with deformable glass membranes. By deforming the membranes of both chambers in a targeted manner, the chromatic aberrations present in the system can be manipulated to tackle both systematic and sample induced aberrations. We demonstrate chromatic aberration correction of up to 2200 mm and shift of the focal spot positions of 4000 mm. For control of this non-linear system with four input voltages, several reinforcement learning agents are trained and compared. The experimental results show that the trained agent can correct system and sample induced aberration and thereby improve the imaging quality, this is demonstrated using biomedical samples. In this case human thyroid was used for demonstration.

Securing Data in Multimode Fibers by Exploiting Mode-Dependent Light Propagation Effects.

Multimode fibers hold great promise to advance data rates in optical communications but come with the challenge to compensate for modal crosstalk and mode-dependent losses, resulting in strong distortions. The holographic measurement of the transmission matrix enables not only correcting distortions but also harnessing these effects for creating a confidential data connection between legitimate communication parties, Alice and Bob. The feasibility of this physical-layer-security-based approach is demonstrated experimentally for the first time on a multimode fiber link to which the eavesdropper Eve is physically coupled. Once the proper structured light field is launched at Alice's side, the message can be delivered to Bob, and, simultaneously, the decipherment for an illegitimate wiretapper Eve is destroyed. Within a real communication scenario, we implement wiretap codes and demonstrate confidentiality by quantifying the level of secrecy. Compared to an uncoded data transmission, the amount of securely exchanged data is enhanced by a factor of 538. The complex light transportation phenomena that have long been considered limiting and have restricted the widespread use of multimode fiber are exploited for opening new perspectives on information security in spatial multiplexing communication systems.

Quantitative phase imaging through an ultra-thin lensless fiber endoscope.

Quantitative phase imaging (QPI) is a label-free technique providing both morphology and quantitative biophysical information in biomedicine. However, applying such a powerful technique to in vivo pathological diagnosis remains challenging. Multi-core fiber bundles (MCFs) enable ultra-thin probes for in vivo imaging, but current MCF imaging techniques are limited to amplitude imaging modalities. We demonstrate a computational lensless microendoscope that uses an ultra-thin bare MCF to perform quantitative phase imaging with microscale lateral resolution and nanoscale axial sensitivity of the optical path length. The incident complex light field at the measurement side is precisely reconstructed from the far-field speckle pattern at the detection side, enabling digital refocusing in a multi-layer sample without any mechanical movement. The accuracy of the quantitative phase reconstruction is validated by imaging the phase target and hydrogel beads through the MCF. With the proposed imaging modality, three-dimensional imaging of human cancer cells is achieved through the ultra-thin fiber endoscope, promising widespread clinical applications.

Real-time complex light field generation through a multi-core fiber with deep learning.

The generation of tailored complex light fields with multi-core fiber (MCF) lensless microendoscopes is widely used in biomedicine. However, the computer-generated holograms (CGHs) used for such applications are typically generated by iterative algorithms, which demand high computation effort, limiting advanced applications like fiber-optic cell manipulation. The random and discrete distribution of the fiber cores in an MCF induces strong spatial aliasing to the CGHs, hence, an approach that can rapidly generate tailored CGHs for MCFs is highly demanded. We demonstrate a novel deep neural network-CoreNet, providing accurate tailored CGHs generation for MCFs at a near video rate. The CoreNet is trained by unsupervised learning and speeds up the computation time by two magnitudes with high fidelity light field generation compared to the previously reported CGH algorithms for MCFs. Real-time generated tailored CGHs are on-the-fly loaded to the phase-only spatial light modulator (SLM) for near video-rate complex light fields generation through the MCF microendoscope. This paves the avenue for real-time cell rotation and several further applications that require real-time high-fidelity light delivery in biomedicine.

Surveillance of few-mode fiber-communication channels with a single hidden layer neural network.

Multi- and few-mode fibers (FMFs) promise to enhance the capacity of optical communication networks by orders of magnitude. The key for this evolution was the strong advancement of computational approaches that allowed inherent complex light transmission to be surpassed, learned, or controlled, reined in by modal crosstalk and mode-dependent losses. However, complex light transmission through FMFs can be learned by a single hidden layer neural network (NN). The emerging developments in NNs additionally allow the implementation of novel concepts for security enhancements in optical communication. Once the transmission characteristics of FMFs are learned, it is possible to survey the incoming and outgoing light fields via monitoring channels during data transmission. If an eavesdropper tries to gain unauthorized access to the FMF, its transmission properties are impaired through sensitive modal crosstalk. This process is registered by the NN and thus the eavesdropper is revealed. With our solution, the security of optical communication can be improved.

Nonlinear microscopy using impulsive stimulated Brillouin scattering for high-speed elastography.

The impulsive stimulated Brillouin microscopy promises fast, non-contact measurements of the elastic properties of biological samples. The used pump-probe approach employs an ultra-short pulse laser and a cw laser to generate Brillouin signals. Modeling of the microscopy technique has already been carried out partially, but not for biomedical applications. The nonlinear relationship between pulse energy and Brillouin signal amplitude is proven with both simulations and experiments. Tayloring of the excitation parameters on the biologically relevant polyacrylamide hydrogels outline sub-ms temporal resolutions at a relative precision of <1%. Brillouin microscopy using the impulsive stimulated scattering therefore exhibits high potential for the measurements of viscoelastic properties of cells and tissues.

Benchmarking analysis of computer generated holograms for complex wavefront shaping using pixelated phase modulators.

Wavefront shaping with spatial light modulators (SLMs) enables aberration correction, especially for light control through complex media, like biological tissues and multimode fibres. High-fidelity light field shaping is associated with the calculation of computer generated holograms (CGHs), of which there are a variety of algorithms. The achievable performance of CGH algorithms depends on various parameters. In this paper, four different algorithms for CGHs are presented and compared for complex light field generation. Two iterative, double constraint Gerchberg-Saxton and direct search, and the two analytical, superpixel and phase encoding, algorithms are investigated. For each algorithm, a parameter study is performed varying the modulator's pixel number and phase resolution. The analysis refers to mode field generation in multimode fibre endoscopes and communication. This enables generality by generating specific mode combinations according to certain spatial frequency power spectra. Thus, the algorithms are compared varying spatial frequencies applied to different implementation scenarios. Our results demonstrate that the choice of algorithms has a significant impact on the achievable performance. This comprehensive study provides the required guide for CGH algorithm selection, improving holographic systems towards multimode fibre endoscopy and communications.

Rapid computational cell-rotation around arbitrary axes in 3D with multi-core fiber.

Optical trapping is a vital tool in biology, allowing precise optical manipulation of nanoparticles, micro-robots, and cells. Due to the low risk of photodamage and high trap stiffness, fiber-based dual-beam traps are widely used for optical manipulation of large cells. Besides trapping, advanced applications like 3D refractive index tomography need a rotation of cells, which requires precise control of the forces, for example, the acting-point of the forces and the intensities in the region of interest (ROI). A precise rotation of large cells in 3D about arbitrary axes has not been reported yet in dual-beam traps. We introduce a novel dual-beam optical trap in which a multi-core fiber (MCF) is transformed to a phased array, using wavefront shaping and computationally programmable light. The light-field distribution in the trapping region is holographically controlled within 0.1 s, which determines the orientation and the rotation axis of the cell with small retardation. We demonstrate real-time controlled rotation of HL60 cells about all 3D axes with a very high degree of freedom by holographic controlled light through an MCF with a resolution close to the diffraction limit. For the first time, the orientation of the cell can be precisely controlled about all 3D axes in a dual-beam trap. MCFs provide much higher flexibility beyond the bulky optics, enabling lab-on-a-chip applications and can be easily integrated for applications like contactless cell surgery, refractive index tomography, cell-elasticity measurement, which require precise 3D manipulation of cells.

In situ measurement of the isoplanatic patch for imaging through intact bone.

Wavefront-shaping (WS) enables imaging through scattering tissues like bone, which is important for neuroscience and bone-regeneration research. WS corrects for the optical aberrations at a given depth and field-of-view (FOV) within the sample; the extent of the validity of which is limited to a region known as the isoplanatic patch (IP). Knowing this parameter helps to estimate the number of corrections needed for WS imaging over a given FOV. In this paper, we first present direct transmissive measurement of murine skull IP using digital optical phase conjugation based focusing. Second, we extend our previously reported phase accumulation ray tracing (PART) method to provide in-situ in-silico estimation of IP, called correlative PART (cPART). Our results show an IP range of 1 to 3 µm for mice within an age range of 8 to 14 days old and 1.00 ± 0.25 µm in a 12-week old adult skull. Consistency between the two measurement approaches indicates that cPART can be used to approximate the IP before a WS experiment, which can be used to calculate the number of corrections required within a given field of view.

Medical Imaging of Microrobots: Toward In Vivo Applications.

Medical microrobots (MRs) have been demonstrated for a variety of non-invasive biomedical applications, such as tissue engineering, drug delivery, and assisted fertilization, among others. However, most of these demonstrations have been carried out in in vitro settings and under optical microscopy, being significantly different from the clinical practice. Thus, medical imaging techniques are required for localizing and tracking such tiny therapeutic machines when used in medical-relevant applications. This review aims at analyzing the state of the art of microrobots imaging by critically discussing the potentialities and limitations of the techniques employed in this field. Moreover, the physics and the working principle behind each analyzed imaging strategy, the spatiotemporal resolution, and the penetration depth are thoroughly discussed. The paper deals with the suitability of each imaging technique for tracking single or swarms of MRs and discusses the scenarios where contrast or imaging agent's inclusion is required, either to absorb, emit, or reflect a determined physical signal detected by an external system. Finally, the review highlights the existing challenges and perspective solutions which could be promising for future in vivo applications.

Physical Layer Security in Multimode Fiber Optical Networks.

The light propagation through a multimode fiber is used to increase information security during data transmission without the need for cryptographic approaches. The use of an inverse precoding method in a multimode fiber-optic communication network is based on mode-dependent losses on the physical layer. This leads to an asymmetry between legitimate (Bob) and illegitimate (Eve) recipients of messages, resulting in significant SNR advantage for Bob. In combination with dynamic mode channel changes, there are defined hurdles for Eve to reconstruct a sent message even in a worst-case scenario in which she knows the channel completely. This is the first time that physical layer security has been investigated in a fiber optical network based on measured transmission matrices. The results show that messages can be sent securely using traditional communication techniques. The technology introduced is a step towards the development of cyber physical systems with increased security.

Impulsive stimulated Brillouin microscopy for non-contact, fast mechanical investigations of hydrogels.

The mechanical properties of tissues and cells are increasingly recognized as an important feature for the understanding of pathological processes and as a diagnostic tool in biomedicine. Impulsive stimulated Brillouin scattering (ISBS) is promising to overcome shortcomings of other measurement methods such as invasiveness, low spatial resolution and long acquisition time. In this paper, we present for the first time ISBS measurements of hydrogels, which are model materials for biological samples. We demonstrate ISBS measurements discriminating hydrogels of different stiffness. ISBS measurements with lateral resolution close to cellular level are presented. These results underline that ISBS microscopy has a high potential for biomedical applications.

Diffraction-limited axial scanning in thick biological tissue with an aberration-correcting adaptive lens.

Diffraction-limited deep focusing into biological tissue is challenging due to aberrations that lead to a broadening of the focal spot. The diffraction limit can be restored by employing aberration correction for example with a deformable mirror. However, this results in a bulky setup due to the required beam folding. We propose a bi-actuator adaptive lens that simultaneously enables axial scanning and the correction of specimen-induced spherical aberrations with a compact setup. Using the bi-actuator lens in a confocal microscope, we show diffraction-limited axial scanning up to 340 µm deep inside a phantom specimen. The application of this technique to in vivo measurements of zebrafish embryos with reporter-gene-driven fluorescence in a thyroid gland reveals substructures of the thyroid follicles, indicating that the bi-actuator adaptive lens is a meaningful supplement to the existing adaptive optics toolset.

Self-calibration of lensless holographic endoscope using programmable guide stars.

Coherent fiber bundle (CFB)-based endoscopes enable optical keyhole access in applications such as biophotonics. In conjunction with objective lenses, CFBs allow imaging of intensity patterns. In contrast, digital optical phase conjugation enables lensless holographic endoscopes for the generation of pixelation-free arbitrary light patterns. For real-world applications, however, this requires a non-invasive in situ calibration of the complex optical transfer function of the CFB with only single-sided access. We show that after an initial calibration in a forward direction, a differential phase measurement of the back-reflected light allows for tracking and compensating of bending-induced phase distortions. Furthermore, we present a novel in situ calibration procedure based on a programmable guide star, which requires access to only one side of the fiber.

Transmission of Multiple Signals through an Optical Fiber Using Wavefront Shaping.

The transmission of multiple independent optical signals through a multimode fiber is accomplished using wavefront shaping in order to compensate for the light distortion during the propagation within the fiber. Our methodology is based on digital optical phase conjugation employing only a single spatial light modulator, where the optical wavefront is individually modulated at different regions of the modulator, one region per light signal. Digital optical phase conjugation approaches are considered to be faster than other wavefront shaping approaches, where (for example) a complete determination of the wave propagation behavior of the fiber is performed. In contrast, the presented approach is time-efficient since it only requires one calibration per light signal. The proposed method is potentially appropriate for spatial division multiplexing in communications engineering. Further application fields are endoscopic light delivery in biophotonics, especially in optogenetics, where single cells in biological tissue have to be selectively illuminated with high spatial and temporal resolution.

Wavefront shaping for imaging-based flow velocity measurements through distortions using a Fresnel guide star.

Imaging-based flow measurement techniques, like particle image velocimetry (PIV), are vulnerable to time-varying distortions like refractive index inhomogeneities or fluctuating phase boundaries. Such distortions strongly increase the velocity error, as the position assignment of the tracer particles and the decrease of image contrast exhibit significant uncertainties. We demonstrate that wavefront shaping based on spatially distributed guide stars has the potential to significantly reduce the measurement uncertainty. Proof of concept experiments show an improvement by more than one order of magnitude. Possible applications for the wavefront shaping PIV range from measurements in jets and film flows to biomedical applications.