Lung Cancer Diagnosis on Virtual Histologically Stained Tissue Using Weakly Supervised Learning.

Lung adenocarcinoma (LUAD) is the most common primary lung cancer and accounts for 40% of all lung cancer cases. The current gold standard for lung cancer analysis is based on the pathologists' interpretation of hematoxylin and eosin (H&E)-stained tissue slices viewed under a brightfield microscope or a digital slide scanner. Computational pathology using deep learning has been proposed to detect lung cancer on histology images. However, the histological staining workflow to acquire the H&E-stained images and the subsequent cancer diagnosis procedures are labor-intensive and time-consuming with tedious sample preparation steps and repetitive manual interpretation, respectively. In this work, we propose a weakly supervised learning method for LUAD classification on label-free tissue slices with virtual histological staining. The autofluorescence images of label-free tissue with histopathological information can be converted into virtual H&E-stained images by a weakly supervised deep generative model. For the downstream LUAD classification task, we trained the attention-based multiple-instance learning model with different settings on the open-source LUAD H&E-stained whole-slide images (WSIs) dataset from the Cancer Genome Atlas (TCGA). The model was validated on the 150 H&E-stained WSIs collected from patients in Queen Mary Hospital and Prince of Wales Hospital with an average area under the curve (AUC) of 0.961. The model also achieved an average AUC of 0.973 on 58 virtual H&E-stained WSIs, comparable to the results on 58 standard H&E-stained WSIs with an average AUC of 0.977. The attention heatmaps of virtual H&E-stained WSIs and ground-truth H&E-stained WSIs can indicate tumor regions of LUAD tissue slices. In conclusion, the proposed diagnostic workflow on virtual H&E-stained WSIs of label-free tissue is a rapid, cost effective, and interpretable approach to assist clinicians in postoperative pathological examinations. The method could serve as a blueprint for other label-free imaging modalities and disease contexts.

Deep learning-assisted low-cost autofluorescence microscopy for rapid slide-free imaging with virtual histological staining.

Slide-free imaging techniques have shown great promise in improving the histological workflow. For example, computational high-throughput autofluorescence microscopy by pattern illumination (CHAMP) has achieved high resolution with a long depth of field, which, however, requires a costly ultraviolet laser. Here, simply using a low-cost light-emitting diode (LED), we propose a deep learning-assisted framework of enhanced widefield microscopy, termed EW-LED, to generate results similar to CHAMP (the learning target). Comparing EW-LED and CHAMP, EW-LED reduces the cost by 85×, shortening the image acquisition time and computation time by 36× and 17×, respectively. This framework can be applied to other imaging modalities, enhancing widefield images for better virtual histology.

Deep learning-assisted smartphone-based quantitative microscopy for label-free peripheral blood smear analysis.

Hematologists evaluate alterations in blood cell enumeration and morphology to confirm peripheral blood smear findings through manual microscopic examination. However, routine peripheral blood smear analysis is both time-consuming and labor-intensive. Here, we propose using smartphone-based autofluorescence microscopy (Smart-AM) for imaging label-free blood smears at subcellular resolution with automatic hematological analysis. Smart-AM enables rapid and label-free visualization of morphological features of normal and abnormal blood cells (including leukocytes, erythrocytes, and thrombocytes). Moreover, assisted with deep-learning algorithms, this technique can automatically detect and classify different leukocytes with high accuracy, and transform the autofluorescence images into virtual Giemsa-stained images which show clear cellular features. The proposed technique is portable, cost-effective, and user-friendly, making it significant for broad point-of-care applications.

Exceeding the limit for microscopic image translation with a deep learning-based unified framework.

Deep learning algorithms have been widely used in microscopic image translation. The corresponding data-driven models can be trained by supervised or unsupervised learning depending on the availability of paired data. However, general cases are where the data are only roughly paired such that supervised learning could be invalid due to data unalignment, and unsupervised learning would be less ideal as the roughly paired information is not utilized. In this work, we propose a unified framework (U-Frame) that unifies supervised and unsupervised learning by introducing a tolerance size that can be adjusted automatically according to the degree of data misalignment. Together with the implementation of a global sampling rule, we demonstrate that U-Frame consistently outperforms both supervised and unsupervised learning in all levels of data misalignments (even for perfectly aligned image pairs) in a myriad of image translation applications, including pseudo-optical sectioning, virtual histological staining (with clinical evaluations for cancer diagnosis), improvement of signal-to-noise ratio or resolution, and prediction of fluorescent labels, potentially serving as new standard for image translation.

Co-aggregation as A Simple Strategy for Preparing Fluorogenic Tetrazine Probes with On-Demand Fluorogen Selection.

Life science has progressed with applications of fluorescent probes-fluorophores linked to functional units responding to biological events. To meet the varied demands across experiments, simple organic reactions to connect fluorophores and functional units have been developed, enabling the on-demand selection of fluorophore-functional unit combinations. However, organic synthesis requires professional equipment and skills, standing as a daunting task for life scientists. In this study, we present a simple, fast, and convenient strategy for probe preparation: co-aggregation of hydrophobic molecules. We focused on tetrazine-a difficult-to-prepare yet useful functional unit that provides effective bioorthogonal reactivity and strong fluorogenicity. Simply mixing the tetrazine molecules and aggregation-induced emission (AIE) luminogens in water, co-aggregation is induced, and the emission of AIE luminogens is quenched. Subsequent click reaction bioorthogonally turns on the emission, identifying these coaggregates as fluorogenic probes. Thanks to this bioorthogonal fluorogenicity, we established a new time-gated fluorescence bioimaging technique to distinguish overlapping emission signals, enabling multi-organelle imaging with two same-color fluorophores. Our study showcases the potential of this co-aggregation method for the on-demand preparation of fluorescent probes as well as protocols and molecular design principles in this approach, offering an effective solution to evolving needs in life science research.

Label-free differential imaging of cellular components in mouse brain tissue by wide-band photoacoustic microscopy.

Mapping diverse cellular components with high spatial resolution is important to interrogate biological systems and study disease pathogenesis. Conventional optical imaging techniques for mapping biomolecular profiles with differential staining and labeling methods are cumbersome. Different types of cellular components exhibit distinctive characteristic absorption spectra across a wide wavelength range. By virtue of this property, a lab-made wide-band optical-resolution photoacoustic microscopy (wbOR-PAM) system, which covers wavelengths from the ultraviolet and visible to the shortwave infrared regions, was designed and developed to capture multiple cellular components in 300-µm-thick brain slices at nine different wavelengths without repetitive staining and complicated processing. This wbOR-PAM system provides abundant spectral information. A reflective objective lens with an infinite conjugate design was applied to focus laser beams with different wavelengths, avoiding chromatic aberration. The molecular components of complex brain slices were probed without labeling. The findings of the present study demonstrated a distinctive absorption of phospholipids, a major component of the cell membrane, brain, and nervous system, at 1690 nm and revealed their precise distribution with microscopic resolution in a mouse brain, for the first time. This novel imaging modality provides a new opportunity to investigate important biomolecular components without either labeling or lengthy specimen processing, thus, laying the groundwork for revealing cellular mechanisms involved in disease pathogenesis.

Translational rapid ultraviolet-excited sectioning tomography for whole-organ multicolor imaging with real-time molecular staining.

Rapid multicolor three-dimensional (3D) imaging for centimeter-scale specimens with subcellular resolution remains a challenging but captivating scientific pursuit. Here, we present a fast, cost-effective, and robust multicolor whole-organ 3D imaging method assisted with ultraviolet (UV) surface excitation and vibratomy-assisted sectioning, termed translational rapid ultraviolet-excited sectioning tomography (TRUST). With an inexpensive UV light-emitting diode (UV-LED) and a color camera, TRUST achieves widefield exogenous molecular-specific fluorescence and endogenous content-rich autofluorescence imaging simultaneously while preserving low system complexity and system cost. Formalin-fixed specimens are stained layer by layer along with serial mechanical sectioning to achieve automated 3D imaging with high staining uniformity and time efficiency. 3D models of all vital organs in wild-type C57BL/6 mice with the 3D structure of their internal components (e.g., vessel network, glomeruli, and nerve tracts) can be reconstructed after imaging with TRUST to demonstrate its fast, robust, and high-content multicolor 3D imaging capability. Moreover, its potential for developmental biology has also been validated by imaging entire mouse embryos (~2 days for the embryo at the embryonic day of 15). TRUST offers a fast and cost-effective approach for high-resolution whole-organ multicolor 3D imaging while relieving researchers from the heavy sample preparation workload.

Low-cost high-resolution photoacoustic microscopy of blood oxygenation with two laser diodes.

Optical-resolution photoacoustic microscopy (OR-PAM) has been widely used for imaging blood vessel and oxygen saturation of hemoglobin (sO2), providing high-resolution functional images of living animals in vivo. However, most of them require one or multiple bulky and costly pulsed lasers, hindering their applicability in preclinical and clinical settings. In this paper, we demonstrate a reflection-mode low-cost high-resolution OR-PAM system by using two cost-effective and compact laser diodes (LDs), achieving microvasculature and sO2 imaging with a high lateral resolution of ∼6 µm. The cost of the excitation sources has dramatically reduced by ∼20-40 times compared to that of the pulsed lasers used in state-of-the-art OR-PAM systems. A blood phantom study was performed to show a determination coefficient R 2 of 0.96 in linear regression analysis. Experimental results of in vivo mouse ear imaging show that the proposed dual-wavelength LD-based PAM system can provide high-resolution functional images at a low cost.

Rapid slide-free and non-destructive histological imaging using wide-field optical-sectioning microscopy.

Histopathology based on formalin-fixed and paraffin-embedded tissues has long been the gold standard for surgical margin assessment (SMA). However, routine pathological practice is lengthy and laborious, failing to guide surgeons intraoperatively. In this report, we propose a practical and low-cost histological imaging method with wide-field optical-sectioning microscopy (i.e., High-and-Low-frequency (HiLo) microscopy). HiLo can achieve rapid and non-destructive imaging of freshly-excised tissues at an extremely high acquisition speed of 5 cm2/min with a spatial resolution of 1.3 µm (lateral) and 5.8 µm (axial), showing great potential as an SMA tool that can provide immediate feedback to surgeons and pathologists for intraoperative decision-making. We demonstrate that HiLo enables rapid extraction of diagnostic features for different subtypes of human lung adenocarcinoma and hepatocellular carcinoma, producing surface images of rough specimens with large field-of-views and cellular features that are comparable to the clinical standard. Our results show promising clinical translations of HiLo microscopy to improve the current standard of care.

High-Speed Ultraviolet Photoacoustic Microscopy for Histological Imaging with Virtual-Staining assisted by Deep Learning.

Surgical margin analysis (SMA), an essential procedure to confirm the complete excision of cancerous tissue in tumor resection surgery, requires intraoperative diagnostic tools to avoid repeated surgeries due to a positive surgical margin. Recently, by taking the advantage of the high intrinsic optical absorption of DNA/RNA at 266 nm wavelength, ultraviolet photoacoustic microscopy (UV-PAM) has been developed to provide high-resolution histological images without labeling, showing great promise as an intraoperative tool for SMA. To enable the development of UV-PAM for SMA, here, a high-speed and open-top UV-PAM system is presented, which can be operated similarly to conventional optical microscopies. The UV-PAM system provides a high lateral resolution of 1.2 µm, and a high imaging speed of 55 kHz A-line rate with one-axis galvanometer mirror scanning. Moreover, to ensure UV-PAM images can be easily interpreted by pathologists without additional training, the original grayscale UV-PAM images are virtually stained by a deep-learning algorithm to mimic the standard hematoxylin- and eosin-stained images, enabling training-free histological analysis. Mouse brain slice imaging is performed to demonstrate the high performance of the open-top UV-PAM system, illustrating its great potential for SMA applications.

Three-dimensional label-free histological imaging of whole organs by microtomy-assisted autofluorescence tomography.

Three-dimensional (3D) histology is vitally important to characterize disease-induced tissue heterogeneity at the individual cell level. However, it remains challenging for both high-throughput 3D imaging and volumetric reconstruction. Here we propose a label-free, cost-effective, and ready-to-use 3D histological imaging technique, termed microtomy-assisted autofluorescence tomography with ultraviolet excitation (MATE). With the combination of block-face imaging and serial microtome sectioning, MATE can achieve rapid and label-free imaging of paraffin-embedded whole organs at an acquisition speed of 1 cm3 per 4 h with a voxel resolution of 1.2 × 1.2 × 10 µm3. We demonstrate that MATE enables simultaneous visualization of cell nuclei, fiber tracts, and blood vessels in mouse/human brains without tissue staining or clearing. Moreover, diagnostic features, including nuclear size and packing density, can be quantitatively extracted with high accuracy. MATE is augmented to the current slide-based 2D histology, holding great promise to facilitate histopathological interpretation at the organelle level.

Deep learning enables ultraviolet photoacoustic microscopy based histological imaging with near real-time virtual staining.

Histological images can reveal rich cellular information of tissue sections, which are widely used by pathologists in disease diagnosis. However, the gold standard for histopathological examination is based on thin sections on slides, which involves inevitable time-consuming and labor-intensive tissue processing steps, hindering the possibility of intraoperative pathological assessment of the precious patient specimens. Here, by incorporating ultraviolet photoacoustic microscopy (UV-PAM) with deep learning, we show a rapid and label-free histological imaging method that can generate virtually stained histological images (termed Deep-PAM) for both thin sections and thick fresh tissue specimens. With the tissue non-destructive nature of UV-PAM, the imaged intact specimens can be reused for other ancillary tests. We demonstrated Deep-PAM on various tissue preparation protocols, including formalin-fixation and paraffin-embedding sections (7-µm thick) and frozen sections (7-µm thick) in traditional histology, and rapid assessment of intact fresh tissue (~ 2-mm thick, within 15 min for a tissue with a surface area of 5 mm × 5 mm). Deep-PAM potentially serves as a comprehensive histological imaging method that can be simultaneously applied in preoperative, intraoperative, and postoperative disease diagnosis.

High-Throughput, Label-Free and Slide-Free Histological Imaging by Computational Microscopy and Unsupervised Learning.

Rapid and high-resolution histological imaging with minimal tissue preparation has long been a challenging and yet captivating medical pursuit. Here, the authors propose a promising and transformative histological imaging method, termed computational high-throughput autofluorescence microscopy by pattern illumination (CHAMP). With the assistance of computational microscopy, CHAMP enables high-throughput and label-free imaging of thick and unprocessed tissues with large surface irregularity at an acquisition speed of 10 mm2 /10 s with 1.1-µm lateral resolution. Moreover, the CHAMP image can be transformed into a virtually stained histological image (Deep-CHAMP) through unsupervised learning within 15 s, where significant cellular features are quantitatively extracted with high accuracy. The versatility of CHAMP is experimentally demonstrated using mouse brain/kidney and human lung tissues prepared with various clinical protocols, which enables a rapid and accurate intraoperative/postoperative pathological examination without tissue processing or staining, demonstrating its great potential as an assistive imaging platform for surgeons and pathologists to provide optimal adjuvant treatment.

Ultraviolet photoacoustic microscopy with tissue clearing for high-contrast histological imaging.

Ultraviolet photoacoustic microscopy (UV-PAM) has been investigated to provide label-free and registration-free volumetric histological images for whole organs, offering new insights into complex biological organs. However, because of the high UV absorption of lipids and pigments in tissue, UV-PAM suffers from low image contrast and shallow image depth, hindering its capability for revealing various microstructures in organs. To improve the UV-PAM imaging contrast and imaging depth, here we propose to implement a state-of-the-art optical clearing technique, CUBIC (clear, unobstructed brain/body imaging cocktails and computational analysis), to wash out the lipids and pigments from tissues. Our results show that the UV-PAM imaging contrast and quality can be significantly improved after tissue clearing. With the cleared tissue, multilayers of cell nuclei can also be extracted from time-resolved PA signals. Tissue clearing-enhanced UV-PAM can provide fine details for organ imaging.

Deep-learning-assisted microscopy with ultraviolet surface excitation for rapid slide-free histological imaging.

Histopathological examination of tissue sections is the gold standard for disease diagnosis. However, the conventional histopathology workflow requires lengthy and laborious sample preparation to obtain thin tissue slices, causing about a one-week delay to generate an accurate diagnostic report. Recently, microscopy with ultraviolet surface excitation (MUSE), a rapid and slide-free imaging technique, has been developed to image fresh and thick tissues with specific molecular contrast. Here, we propose to apply an unsupervised generative adversarial network framework to translate colorful MUSE images into Deep-MUSE images that highly resemble hematoxylin and eosin staining, allowing easy adaptation by pathologists. By eliminating the needs of all sample processing steps (except staining), a MUSE image with subcellular resolution for a typical brain biopsy (5 mm × 5 mm) can be acquired in 5 minutes, which is further translated into a Deep-MUSE image in 40 seconds, simplifying the standard histopathology workflow dramatically and providing histological images intraoperatively.

High-speed high-resolution laser diode-based photoacoustic microscopy for in vivo microvasculature imaging.

Laser diodes (LDs) have been considered as cost-effective and compact excitation sources to overcome the requirement of costly and bulky pulsed laser sources that are commonly used in photoacoustic microscopy (PAM). However, the spatial resolution and/or imaging speed of previously reported LD-based PAM systems have not been optimized simultaneously. In this paper, we developed a high-speed and high-resolution LD-based PAM system using a continuous wave LD, operating at a pulsed mode, with a repetition rate of 30 kHz, as an excitation source. A hybrid scanning mechanism that synchronizes a one-dimensional galvanometer mirror and a two-dimensional motorized stage is applied to achieve a fast imaging capability without signal averaging due to the high signal-to-noise ratio. By optimizing the optical system, a high lateral resolution of 4.8 µm has been achieved. In vivo microvasculature imaging of a mouse ear has been demonstrated to show the high performance of our LD-based PAM system.

High-speed label-free ultraviolet photoacoustic microscopy for histology-like imaging of unprocessed biological tissues.

Ultraviolet photoacoustic microscopy (UV-PAM) has recently been demonstrated as a potential imaging tool for surgical margin analysis (SMA). UV-PAM does not require staining or micrometer-thick slicing, which is inevitable in conventional histological imaging. To promote UV-PAM as a practical intraoperative diagnostic tool, the imaging speed should be improved while preserving the high-resolution imaging capability and simplistic system design. In this Letter, we developed a galvanometer mirror-based UV-PAM (GM-UV-PAM) system for high-speed histology-like imaging. By using a UV laser with a high repetition rate (55 kHz) and a one-dimensional galvanometer mirror, our GM-UV-PAM system can generate subcellular images in less than 15 min for a typical brain biopsy (5mm×5mm), with a lateral resolution of ∼1.0µm. The images of mouse brain slices obtained by our GM-UV-PAM system show that it can provide histological information for SMA.

A Review of Endogenous and Exogenous Contrast Agents Used in Photoacoustic Tomography with Different Sensing Configurations.

Optical-based sensing approaches have long been an indispensable way to detect molecules in biological tissues for various biomedical research and applications. The advancement in optical microscopy is one of the main drivers for discoveries and innovations in both life science and biomedical imaging. However, the shallow imaging depth due to the use of ballistic photons fundamentally limits optical imaging approaches' translational potential to a clinical setting. Photoacoustic (PA) tomography (PAT) is a rapidly growing hybrid imaging modality that is capable of acoustically detecting optical contrast. PAT uniquely enjoys high-resolution deep-tissue imaging owing to the utilization of diffused photons. The exploration of endogenous contrast agents and the development of exogenous contrast agents further improve the molecular specificity for PAT. PAT's versatile design and non-invasive nature have proven its great potential as a biomedical imaging tool for a multitude of biomedical applications. In this review, representative endogenous and exogenous PA contrast agents will be introduced alongside common PAT system configurations, including the latest advances of all-optical acoustic sensing techniques.

Multifocal photoacoustic microscopy using a single-element ultrasonic transducer through an ergodic relay.

Optical-resolution photoacoustic microscopy (OR-PAM) has demonstrated high-spatial-resolution imaging of optical absorption in biological tissue. To date, most OR-PAM systems rely on mechanical scanning with confocally aligned optical excitation and ultrasonic detection, limiting the wide-field imaging speed of these systems. Although several multifocal OR-PA (MFOR-PA) systems have attempted to address this limitation, they are hindered by the complex design in a constrained physical space. Here, we present a two-dimensional (2D) MFOR-PAM system that utilizes a 2D microlens array and an acoustic ergodic relay. Using a single-element ultrasonic transducer, this system can detect PA signals generated from 400 optical foci in parallel and then raster scan the optical foci patterns to form an MFOR-PAM image. This system improves the imaging resolution of an acoustic ergodic relay system from 220 to 13 µm and enables 400-folds shorter scanning time than that of a conventional OR-PAM system at the same resolution and laser repetition rate. We demonstrated the imaging ability of the system with both in vitro and in vivo experiments.

High-resolution, high-contrast mid-infrared imaging of fresh biological samples with ultraviolet-localized photoacoustic microscopy.

Mid-infrared (MIR) microscopy provides rich chemical and structural information about biological samples, without staining. Conventionally, the long MIR wavelength severely limits the lateral resolution owing to optical diffraction; moreover, the strong MIR absorption of water ubiquitous in fresh biological samples results in high background and low contrast. To overcome these limitations, we propose a method that employs photoacoustic detection highly localized with a pulsed ultraviolet (UV) laser on the basis of the Grüneisen relaxation effect. For cultured cells, our method achieves water-background suppressed MIR imaging of lipids and proteins at UV resolution, at least an order of magnitude finer than the MIR diffraction limits. Label-free histology using this method is also demonstrated in thick brain slices. Our approach provides convenient high-resolution and high-contrast MIR imaging, which can benefit diagnosis of fresh biological samples.