Resolution-enhanced multi-core fiber imaging learned on a digital twin for cancer diagnosis.

Significance: Deep learning enables label-free all-optical biopsies and automated tissue classification. Endoscopic systems provide intraoperative diagnostics to deep tissue and speed up treatment without harmful tissue removal. However, conventional multi-core fiber (MCF) endoscopes suffer from low resolution and artifacts, which hinder tumor diagnostics. Aim: We introduce a method to enable unpixelated, high-resolution tumor imaging through a given MCF with a diameter of around 0.65 mm and arbitrary core arrangement and inhomogeneous transmissivity. Approach: Image reconstruction is based on deep learning and the digital twin concept of the single-reference-based simulation with inhomogeneous optical properties of MCF and transfer learning on a small experimental dataset of biological tissue. The reference provided physical information about the MCF during the training processes. Results: For the simulated data, hallucination caused by the MCF inhomogeneity was eliminated, and the averaged peak signal-to-noise ratio and structural similarity were increased from 11.2 dB and 0.20 to 23.4 dB and 0.74, respectively. By transfer learning, the metrics of independent test images experimentally acquired on glioblastoma tissue ex vivo can reach up to 31.6 dB and 0.97 with 14 fps computing speed. Conclusions: With the proposed approach, a single reference image was required in the pre-training stage and laborious acquisition of training data was bypassed. Validation on glioblastoma cryosections with transfer learning on only 50 image pairs showed the capability for high-resolution deep tissue retrieval and high clinical feasibility.

Fully refractive telecentric f-theta microscope based on adaptive elements for 3D raster scanning of biological tissues.

Various techniques in microscopy are based on point-wise acquisition, which provides advantages in acquiring sectioned images, for example in confocal or two-photon microscopy. The advantages come along with the need to perform three-dimensional scanning, which is often realized by mechanical movement achieved by stage-scanning or piezo-based scanning in the axial direction. Lateral scanning often employs galvo-mirrors, leading to a reflective setup and hence to a folded beam path. In this paper, we introduce a fully refractive microscope capable of three-dimensional scanning, which employs the combination of an adaptive lens, an adaptive prism, and a tailored telecentric f-theta objective. Our results show that this microscope is capable to perform flexible three-dimensional scanning, with low scan-induced aberrations, at a uniform resolution over a large tuning range of X=Y=6300 µ m and Z=480 µ m with only transmissive components. We demonstrate the capabilities at the example of volumetric measurements on the transgenic fluorescence of the thyroid of a zebrafish embryo and mixed pollen grains. This is the first step towards flexible aberration-free volumetric smart microscopy of three-dimensional samples like embryos and organoids, which could be exploited for the demands in both lateral and axial dimensions in biomedical samples without compromising image quality.

Securing Data in Multimode Fibers by Exploiting Mode-Dependent Light Propagation Effects.

Multimode fibers hold great promise to advance data rates in optical communications but come with the challenge to compensate for modal crosstalk and mode-dependent losses, resulting in strong distortions. The holographic measurement of the transmission matrix enables not only correcting distortions but also harnessing these effects for creating a confidential data connection between legitimate communication parties, Alice and Bob. The feasibility of this physical-layer-security-based approach is demonstrated experimentally for the first time on a multimode fiber link to which the eavesdropper Eve is physically coupled. Once the proper structured light field is launched at Alice's side, the message can be delivered to Bob, and, simultaneously, the decipherment for an illegitimate wiretapper Eve is destroyed. Within a real communication scenario, we implement wiretap codes and demonstrate confidentiality by quantifying the level of secrecy. Compared to an uncoded data transmission, the amount of securely exchanged data is enhanced by a factor of 538. The complex light transportation phenomena that have long been considered limiting and have restricted the widespread use of multimode fiber are exploited for opening new perspectives on information security in spatial multiplexing communication systems.

Long-term morphological and functional dynamics of human stem cell-derived neuronal networks on high-density micro-electrode arrays.

Comprehensive electrophysiological characterizations of human induced pluripotent stem cell (hiPSC)-derived neuronal networks are essential to determine to what extent these in vitro models recapitulate the functional features of in vivo neuronal circuits. High-density micro-electrode arrays (HD-MEAs) offer non-invasive recording with the best spatial and temporal resolution possible to date. For 3 months, we tracked the morphology and activity features of developing networks derived from a transgenic hiPSC line in which neurogenesis is inducible by neurogenic transcription factor overexpression. Our morphological data revealed large-scale structural changes from homogeneously distributed neurons in the first month to the formation of neuronal clusters over time. This led to a constant shift in position of neuronal cells and clusters on HD-MEAs and corresponding changes in spatial distribution of the network activity maps. Network activity appeared as scarce action potentials (APs), evolved as local bursts with longer duration and changed to network-wide synchronized bursts with higher frequencies but shorter duration over time, resembling the emerging burst features found in the developing human brain. Instantaneous firing rate data indicated that the fraction of fast spiking neurons (150-600 Hz) increases sharply after 63 days post induction (dpi). Inhibition of glutamatergic synapses erased burst features from network activity profiles and confirmed the presence of mature excitatory neurotransmission. The application of GABAergic receptor antagonists profoundly changed the bursting profile of the network at 120 dpi. This indicated a GABAergic switch from excitatory to inhibitory neurotransmission during circuit development and maturation. Our results suggested that an emerging GABAergic system at older culture ages is involved in regulating spontaneous network bursts. In conclusion, our data showed that long-term and continuous microscopy and electrophysiology readouts are crucial for a meaningful characterization of morphological and functional maturation in stem cell-derived human networks. Most importantly, assessing the level and duration of functional maturation is key to subject these human neuronal circuits on HD-MEAs for basic and biomedical applications.

Beyond Janus Geometry: Characterization of Flow Fields around Nonspherical Photocatalytic Microswimmers.

Catalytic microswimmers that move by a phoretic mechanism in response to a self-induced chemical gradient are often obtained by the design of spherical janus microparticles, which suffer from multi-step fabrication and low yields. Approaches that circumvent laborious multi-step fabrication include the exploitation of the possibility of nonuniform catalytic activity along the surface of irregular particle shapes, local excitation or intrinsic asymmetry. Unfortunately, the effects on the generation of motion remain poorly understood. In this work, single crystalline BiVO4 microswimmers are presented that rely on a strict inherent asymmetry of charge-carrier distribution under illumination. The origin of the asymmetrical flow pattern is elucidated because of the high spatial resolution of measured flow fields around pinned BiVO4 colloids. As a result the flow from oxidative to reductive particle sides is confirmed. Distribution of oxidation and reduction reactions suggests a dominant self-electrophoretic motion mechanism with a source quadrupole as the origin of the induced flows. It is shown that the symmetry of the flow fields is broken by self-shadowing of the particles and synthetic surface defects that impact the photocatalytic activity of the microswimmers. The results demonstrate the complexity of symmetry breaking in nonspherical microswimmers and emphasize the role of self-shadowing for photocatalytic microswimmers. The findings are leading the way toward understanding of propulsion mechanisms of phoretic colloids of various shapes.

Tracking connectivity maps in human stem cell-derived neuronal networks by holographic optogenetics.

Neuronal networks derived from human induced pluripotent stem cells have been exploited widely for modeling neuronal circuits, neurological diseases, and drug screening. As these networks require extended culturing periods to functionally mature in vitro, most studies are based on immature networks. To obtain insights on long-term functional features, we improved a glia-neuron co-culture protocol within multi-electrode arrays, facilitating continuous assessment of electrical features in weekly intervals. By full-field optogenetic stimulation, we detected an earlier onset of neuronal firing and burst activity compared with spontaneous activity. Full-field stimulation enhanced the number of active neurons and their firing rates. Compared with full-field stimulation, which evoked synchronized activity across all neurons, holographic stimulation of individual neurons resulted in local activity. Single-cell holographic stimulation facilitated to trace propagating evoked activities of 400 individually stimulated neurons per multi-electrode array. Thereby, we revealed precise functional neuronal connectivity motifs. Holographic stimulation data over time showed increasing connection numbers and strength with culture age. This holographic stimulation setup has the potential to establish a profound functional testbed for in-depth analysis of human-induced pluripotent stem cell-derived neuronal networks.

Holographic optogenetic stimulation with calcium imaging as an all optical tool for cardiac electrophysiology.

All optical approaches to control and read out the electrical activity in a cardiac syncytium can improve our understanding of cardiac electrophysiology. Here, we demonstrate optogenetic stimulation of cardiomyocytes with high spatial precision using light foci generated with a ferroelectric spatial light modulator. Computer generated holograms binarized by bidirectional error diffusion create multiple foci with more even intensity distribution compared with thresholding approach. We evoke the electrical activity of cardiac HL1 cells expressing the channelrhodopsin-2 variant, ChR2(H134R) using single and multiple light foci and at the same time visualize the action potential using a calcium sensitive indicator called Cal-630. We show that localized regions in the cardiac monolayer can be stimulated enabling us to initiate signal propagation from a precise location. Furthermore, we demonstrate that probing the cardiac cells with multiple light foci enhances the excitability of the cardiac network. This approach opens new applications in manipulating and visualizing the electrical activity in a cardiac syncytium.

Surveillance of few-mode fiber-communication channels with a single hidden layer neural network.

Multi- and few-mode fibers (FMFs) promise to enhance the capacity of optical communication networks by orders of magnitude. The key for this evolution was the strong advancement of computational approaches that allowed inherent complex light transmission to be surpassed, learned, or controlled, reined in by modal crosstalk and mode-dependent losses. However, complex light transmission through FMFs can be learned by a single hidden layer neural network (NN). The emerging developments in NNs additionally allow the implementation of novel concepts for security enhancements in optical communication. Once the transmission characteristics of FMFs are learned, it is possible to survey the incoming and outgoing light fields via monitoring channels during data transmission. If an eavesdropper tries to gain unauthorized access to the FMF, its transmission properties are impaired through sensitive modal crosstalk. This process is registered by the NN and thus the eavesdropper is revealed. With our solution, the security of optical communication can be improved.

Nonlinear microscopy using impulsive stimulated Brillouin scattering for high-speed elastography.

The impulsive stimulated Brillouin microscopy promises fast, non-contact measurements of the elastic properties of biological samples. The used pump-probe approach employs an ultra-short pulse laser and a cw laser to generate Brillouin signals. Modeling of the microscopy technique has already been carried out partially, but not for biomedical applications. The nonlinear relationship between pulse energy and Brillouin signal amplitude is proven with both simulations and experiments. Tayloring of the excitation parameters on the biologically relevant polyacrylamide hydrogels outline sub-ms temporal resolutions at a relative precision of <1%. Brillouin microscopy using the impulsive stimulated scattering therefore exhibits high potential for the measurements of viscoelastic properties of cells and tissues.

Benchmarking analysis of computer generated holograms for complex wavefront shaping using pixelated phase modulators.

Wavefront shaping with spatial light modulators (SLMs) enables aberration correction, especially for light control through complex media, like biological tissues and multimode fibres. High-fidelity light field shaping is associated with the calculation of computer generated holograms (CGHs), of which there are a variety of algorithms. The achievable performance of CGH algorithms depends on various parameters. In this paper, four different algorithms for CGHs are presented and compared for complex light field generation. Two iterative, double constraint Gerchberg-Saxton and direct search, and the two analytical, superpixel and phase encoding, algorithms are investigated. For each algorithm, a parameter study is performed varying the modulator's pixel number and phase resolution. The analysis refers to mode field generation in multimode fibre endoscopes and communication. This enables generality by generating specific mode combinations according to certain spatial frequency power spectra. Thus, the algorithms are compared varying spatial frequencies applied to different implementation scenarios. Our results demonstrate that the choice of algorithms has a significant impact on the achievable performance. This comprehensive study provides the required guide for CGH algorithm selection, improving holographic systems towards multimode fibre endoscopy and communications.

Rapid computational cell-rotation around arbitrary axes in 3D with multi-core fiber.

Optical trapping is a vital tool in biology, allowing precise optical manipulation of nanoparticles, micro-robots, and cells. Due to the low risk of photodamage and high trap stiffness, fiber-based dual-beam traps are widely used for optical manipulation of large cells. Besides trapping, advanced applications like 3D refractive index tomography need a rotation of cells, which requires precise control of the forces, for example, the acting-point of the forces and the intensities in the region of interest (ROI). A precise rotation of large cells in 3D about arbitrary axes has not been reported yet in dual-beam traps. We introduce a novel dual-beam optical trap in which a multi-core fiber (MCF) is transformed to a phased array, using wavefront shaping and computationally programmable light. The light-field distribution in the trapping region is holographically controlled within 0.1 s, which determines the orientation and the rotation axis of the cell with small retardation. We demonstrate real-time controlled rotation of HL60 cells about all 3D axes with a very high degree of freedom by holographic controlled light through an MCF with a resolution close to the diffraction limit. For the first time, the orientation of the cell can be precisely controlled about all 3D axes in a dual-beam trap. MCFs provide much higher flexibility beyond the bulky optics, enabling lab-on-a-chip applications and can be easily integrated for applications like contactless cell surgery, refractive index tomography, cell-elasticity measurement, which require precise 3D manipulation of cells.

Distortion correction for particle image velocimetry using multiple-input deep convolutional neural network and Hartmann-Shack sensing.

Aberrations degrade the accuracy of quantitative, imaging-based measurements, like particle image velocimetry (PIV). Adaptive optical elements can in principle correct the wavefront distortions, but are limited by their technical specifications. Here we propose an actuator-free correction based on a multiple-input deep convolutional neural network which uses an additional input from a wavefront sensor to correct time-varying distortions. It is applied for imaging flow velocimetry to conduct measurements through a fluctuating air-water phase boundary. Dataset for neural network is generated by an experimental setup with a deformable mirror. Correction performance of trained model is estimated in terms of image quality, which is improved significantly, and flow measurement results, where the errors induced by the distortion from fluctuating phase boundary can be corrected by 82 %. The technique has the potential to replace classical closed-loop adaptive optical systems where the performance of the actuators is not sufficient.

In situ measurement of the isoplanatic patch for imaging through intact bone.

Wavefront-shaping (WS) enables imaging through scattering tissues like bone, which is important for neuroscience and bone-regeneration research. WS corrects for the optical aberrations at a given depth and field-of-view (FOV) within the sample; the extent of the validity of which is limited to a region known as the isoplanatic patch (IP). Knowing this parameter helps to estimate the number of corrections needed for WS imaging over a given FOV. In this paper, we first present direct transmissive measurement of murine skull IP using digital optical phase conjugation based focusing. Second, we extend our previously reported phase accumulation ray tracing (PART) method to provide in-situ in-silico estimation of IP, called correlative PART (cPART). Our results show an IP range of 1 to 3 µm for mice within an age range of 8 to 14 days old and 1.00 ± 0.25 µm in a 12-week old adult skull. Consistency between the two measurement approaches indicates that cPART can be used to approximate the IP before a WS experiment, which can be used to calculate the number of corrections required within a given field of view.

Ultrasound Super-Resolution Flow Measurement of Suspensions in Narrow Channels.

Zinc-air flow batteries provide a scalable and cost-efficient energy storage solution. However, the achieved power density depends on the local flow conditions of the zinc particle suspension in the electrochemical cell. Numerical modeling is challenging due to the complex multiphase fluid and the interaction of flow and electrochemistry. Hence, performing experiments is crucial to investigate the influence of the flow conditions on the electrical performance, which requires flow instrumentation for the opaque suspension. To resolve the flow field across the 2.6-mm-wide flow channel of the investigated zinc-air flow battery (ZAB), a spatial resolution below 100 [Formula: see text] has to be typically achieved. Using ultrasound techniques, the achieved spatial resolution is limited by the trade-off between ultrasound frequency and imaging depth. This trade-off is even more critical for suspensions due to the scattering of the ultrasound, which increases strongly with frequency. We propose super-resolution particle tracking velocimetry (SRPTV) to overcome this limitation by achieving the required spatial resolution at a low ultrasound frequency. SRPTV is based on the super-resolution technique ultrasound localization microscopy, which is adapted to strongly scattering suspensions by using a dual-frequency-phased array and applying a coherence weighting beamformer to suppress speckles, which result from the scattering at the zinc particles of the suspension. The spatial resolution and the velocity uncertainty are characterized through calibration measurement and numerical simulation. A spatial resolution of 66 [Formula: see text] at an excitation wavelength of 330 [Formula: see text] was achieved, which is sufficient for performing flow investigation in an operational ZAB. The measured flow profile reveals shear-thinning properties and wall slip and therefore differs significantly from a parabolic flow profile of a Newtonian fluid. The presented technique offers potential for performing flow investigations of suspensions in small geometries with a spatial resolution beyond the diffraction limit.

Medical Imaging of Microrobots: Toward In Vivo Applications.

Medical microrobots (MRs) have been demonstrated for a variety of non-invasive biomedical applications, such as tissue engineering, drug delivery, and assisted fertilization, among others. However, most of these demonstrations have been carried out in in vitro settings and under optical microscopy, being significantly different from the clinical practice. Thus, medical imaging techniques are required for localizing and tracking such tiny therapeutic machines when used in medical-relevant applications. This review aims at analyzing the state of the art of microrobots imaging by critically discussing the potentialities and limitations of the techniques employed in this field. Moreover, the physics and the working principle behind each analyzed imaging strategy, the spatiotemporal resolution, and the penetration depth are thoroughly discussed. The paper deals with the suitability of each imaging technique for tracking single or swarms of MRs and discusses the scenarios where contrast or imaging agent's inclusion is required, either to absorb, emit, or reflect a determined physical signal detected by an external system. Finally, the review highlights the existing challenges and perspective solutions which could be promising for future in vivo applications.

Video-rate lensless endoscope with self-calibration using wavefront shaping.

Lensless fiber endoscopes are of great importance for keyhole imaging. Coherent fiber bundles (CFB) can be used in endoscopes as remote phased arrays to capture images. One challenge is to image at high speed while correcting aberrations induced by the CFB. We propose the combination of digital optical phase conjugation, using a spatial light modulator, with fast scanning, for which a 2D galvo scanner and an adaptive lens are employed. We achieve the transmission of laser and image scanning through the CFB. Video-rate imaging at 20 Hz in 2D with subcellular resolution is demonstrated in 3D with 1 Hz. The sub-millimeter-diameter scanning endoscope has a great potential in biomedicine, for manipulation, e.g., in optogenetics, as well as in imaging.

Recent progress and current opinions in Brillouin microscopy for life science applications.

Many important biological functions and processes are reflected in cell and tissue mechanical properties such as elasticity and viscosity. However, current techniques used for measuring these properties have major limitations, such as that they can often not measure inside intact cells and/or require physical contact-which cells can react to and change. Brillouin light scattering offers the ability to measure mechanical properties in a non-contact and label-free manner inside of objects with high spatial resolution using light, and hence has emerged as an attractive method during the past decade. This new approach, coined "Brillouin microscopy," which integrates highly interdisciplinary concepts from physics, engineering, and mechanobiology, has led to a vibrant new community that has organized itself via a European funded (COST Action) network. Here we share our current assessment and opinion of the field, as emerged from a recent dedicated workshop. In particular, we discuss the prospects towards improved and more bio-compatible instrumentation, novel strategies to infer more accurate and quantitative mechanical measurements, as well as our current view on the biomechanical interpretation of the Brillouin spectra.

Velocity measurements with structured light transmitted through a multimode optical fiber using digital optical phase conjugation.

Lensless fiber microendoscopes enable optical diagnostics and therapy with minimal invasiveness. Because of their small diameters, multimode fibers are ideal candidates, but mode scrambling hinders the transmission of structured light fields. We present the generation of a localized fringe system at variable distances from the distal fiber end by exploiting digital optical phase conjugation. The replayed fringe system was used for quantitative metrology. Velocity measurements of a microchannel flow in the immediate proximity of the fiber end without the use of any imaging lenses are shown. Lensless multimode fiber systems are of interest especially for biomedical imaging and stimulation as well as technical inspection and flow measurements.

Physical Layer Security in Multimode Fiber Optical Networks.

The light propagation through a multimode fiber is used to increase information security during data transmission without the need for cryptographic approaches. The use of an inverse precoding method in a multimode fiber-optic communication network is based on mode-dependent losses on the physical layer. This leads to an asymmetry between legitimate (Bob) and illegitimate (Eve) recipients of messages, resulting in significant SNR advantage for Bob. In combination with dynamic mode channel changes, there are defined hurdles for Eve to reconstruct a sent message even in a worst-case scenario in which she knows the channel completely. This is the first time that physical layer security has been investigated in a fiber optical network based on measured transmission matrices. The results show that messages can be sent securely using traditional communication techniques. The technology introduced is a step towards the development of cyber physical systems with increased security.

Quantitative measurement of density fluctuations with a full-field laser interferometric vibrometer.

ABSTRACT: Modern, lean and premixed gas turbine combustion concepts for low NOx emissions are prone to combustion instabilities. In a previous work it was shown that laser interferometric vibrometry (LIV) can be used to record global as well as local heat release fluctuations in swirl-stabilized premixed methane flames quantitatively, if other effects influencing density are small. In this work a newly developed camera-based full-field LIV system (CLIV) was applied to a lean, confined, premixed and swirl-stabilized methane flame under atmospheric conditions. Instead of time-consuming pointwise scanning of the flame, CLIV records full-field line-of-sight density fluctuations with high spatio-temporal resolution. With a recording rate of 200 kHz, CLIV enables the visualization of highly unsteady processes in fluid dynamics and combustion research. As an example for an unsteady process, the propagation of the flame front through a lean, premixed gas volume is visualized during an ignition process. A discussion of algorithms and assumptions necessary to calculate heat release oscillations from density oscillations is presented and applied to phase-averaged data recorded with CLIV for this type of flame. As reference, OH* chemiluminescence data were recorded simultaneously. While density gradients travelling with the flow are recorded by LIV and CLIV, chemiluminescence imaging will show nothing in the absence of chemical reaction. GRAPHIC ABSTRACT: a Time-averaged density gradient within the combustor in lateral direction. b Density fluctuations along line-of-sight 7 ms after ignition. c Phase-averaged and local heat release fluctuations at 225 Hz perturbation frequency.