Deep learning-based color holographic microscopy.

We report a framework based on a generative adversarial network that performs high-fidelity color image reconstruction using a single hologram of a sample that is illuminated simultaneously by light at three different wavelengths. The trained network learns to eliminate missing-phase-related artifacts, and generates an accurate color transformation for the reconstructed image. Our framework is experimentally demonstrated using lung and prostate tissue sections that are labeled with different histological stains. This framework is envisaged to be applicable to point-of-care histopathology and presents a significant improvement in the throughput of coherent microscopy systems given that only a single hologram of the specimen is required for accurate color imaging.

Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning.

The histological analysis of tissue samples, widely used for disease diagnosis, involves lengthy and laborious tissue preparation. Here, we show that a convolutional neural network trained using a generative adversarial-network model can transform wide-field autofluorescence images of unlabelled tissue sections into images that are equivalent to the bright-field images of histologically stained versions of the same samples. A blind comparison, by board-certified pathologists, of this virtual staining method and standard histological staining using microscopic images of human tissue sections of the salivary gland, thyroid, kidney, liver and lung, and involving different types of stain, showed no major discordances. The virtual-staining method bypasses the typically labour-intensive and costly histological staining procedures, and could be used as a blueprint for the virtual staining of tissue images acquired with other label-free imaging modalities.

Deep learning-based super-resolution in coherent imaging systems.

We present a deep learning framework based on a generative adversarial network (GAN) to perform super-resolution in coherent imaging systems. We demonstrate that this framework can enhance the resolution of both pixel size-limited and diffraction-limited coherent imaging systems. The capabilities of this approach are experimentally validated by super-resolving complex-valued images acquired using a lensfree on-chip holographic microscope, the resolution of which was pixel size-limited. Using the same GAN-based approach, we also improved the resolution of a lens-based holographic imaging system that was limited in resolution by the numerical aperture of its objective lens. This deep learning-based super-resolution framework can be broadly applied to enhance the space-bandwidth product of coherent imaging systems using image data and convolutional neural networks, and provides a rapid, non-iterative method for solving inverse image reconstruction or enhancement problems in optics.

PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning.

Using a deep neural network, we demonstrate a digital staining technique, which we term PhaseStain, to transform the quantitative phase images (QPI) of label-free tissue sections into images that are equivalent to the brightfield microscopy images of the same samples that are histologically stained. Through pairs of image data (QPI and the corresponding brightfield images, acquired after staining), we train a generative adversarial network and demonstrate the effectiveness of this virtual-staining approach using sections of human skin, kidney, and liver tissue, matching the brightfield microscopy images of the same samples stained with Hematoxylin and Eosin, Jones' stain, and Masson's trichrome stain, respectively. This digital-staining framework may further strengthen various uses of label-free QPI techniques in pathology applications and biomedical research in general, by eliminating the need for histological staining, reducing sample preparation related costs and saving time. Our results provide a powerful example of some of the unique opportunities created by data-driven image transformations enabled by deep learning.

Deep learning enables cross-modality super-resolution in fluorescence microscopy.

We present deep-learning-enabled super-resolution across different fluorescence microscopy modalities. This data-driven approach does not require numerical modeling of the imaging process or the estimation of a point-spread-function, and is based on training a generative adversarial network (GAN) to transform diffraction-limited input images into super-resolved ones. Using this framework, we improve the resolution of wide-field images acquired with low-numerical-aperture objectives, matching the resolution that is acquired using high-numerical-aperture objectives. We also demonstrate cross-modality super-resolution, transforming confocal microscopy images to match the resolution acquired with a stimulated emission depletion (STED) microscope. We further demonstrate that total internal reflection fluorescence (TIRF) microscopy images of subcellular structures within cells and tissues can be transformed to match the results obtained with a TIRF-based structured illumination microscope. The deep network rapidly outputs these super-resolved images, without any iterations or parameter search, and could serve to democratize super-resolution imaging.

Motility-based label-free detection of parasites in bodily fluids using holographic speckle analysis and deep learning.

Parasitic infections constitute a major global public health issue. Existing screening methods that are based on manual microscopic examination often struggle to provide sufficient volumetric throughput and sensitivity to facilitate early diagnosis. Here, we demonstrate a motility-based label-free computational imaging platform to rapidly detect motile parasites in optically dense bodily fluids by utilizing the locomotion of the parasites as a specific biomarker and endogenous contrast mechanism. Based on this principle, a cost-effective and mobile instrument, which rapidly screens ~3.2 mL of fluid sample in three dimensions, was built to automatically detect and count motile microorganisms using their holographic time-lapse speckle patterns. We demonstrate the capabilities of our platform by detecting trypanosomes, which are motile protozoan parasites, with various species that cause deadly diseases affecting millions of people worldwide. Using a holographic speckle analysis algorithm combined with deep learning-based classification, we demonstrate sensitive and label-free detection of trypanosomes within spiked whole blood and artificial cerebrospinal fluid (CSF) samples, achieving a limit of detection of ten trypanosomes per mL of whole blood (~five-fold better than the current state-of-the-art parasitological method) and three trypanosomes per mL of CSF. We further demonstrate that this platform can be applied to detect other motile parasites by imaging Trichomonas vaginalis, the causative agent of trichomoniasis, which affects 275 million people worldwide. With its cost-effective, portable design and rapid screening time, this unique platform has the potential to be applied for sensitive and timely diagnosis of neglected tropical diseases caused by motile parasites and other parasitic infections in resource-limited regions.

Role of tilted adhesion fibrils (setae) in the adhesion and locomotion of gecko-like systems.

Geckos are super climbers: they can readily and rapidly stick to almost any surface, whether hydrophilic or hydrophobic, rough or smooth, in dry or wet conditions, and detach with equal rapidity within tens of milliseconds. In this paper, we discuss the rapid switching between the strong adhesion/friction (attached) state and zero adhesion/friction (detached) state, and present a finite element analysis of gecko setae in terms of their adhesion and friction forces. The analysis shows why the asymmetric, naturally curved setae with a directional tilt play a crucial role in the gecko's articulation mechanism, consistent with recent experimental studies of gecko setal arrays. We derive guidelines for designing synthetic versions of gecko adhesive pads, and propose a design for a "gecko-inspired" adhesive surface consisting of arrays of curved, asymmetric, and directionally oriented microfibrils, attached to a semirigid backing, and suggest a method for its actuation.

Analysis and interpretation of stress fiber organization in cells subject to cyclic stretch.

Numerical simulations that incorporate a biochemomechanical model for the contractility of the cytoskeleton have been used to rationalize the following observations. Uniaxial cyclic stretching of cells causes stress fibers to align perpendicular to the stretch direction, with degree of alignment dependent on the stretch strain magnitude, as well as the frequency and the transverse contraction of the substrate. Conversely, equibiaxial cyclic stretching induces a uniform distribution of stress fiber orientations. Demonstrations that the model successfully predicts the alignments experimentally found are followed by a parameter study to investigate the influence of a range of key variables including the stretch magnitude, the intrinsic rate sensitivity of the stress fibers, the straining frequency, and the transverse contraction of the substrate. The primary predictions are as follows. The rate sensitivity has a strong influence on alignment, equivalent to that attained by a few percent of additional stretch. The fiber alignment increases with increasing cycling frequency. Transverse contraction of the substrate causes the stress fibers to organize into two symmetrical orientations with respect to the primary stretch direction.