Open-source data pipeline for street-view images: A case study on community mobility during COVID-19 pandemic.

Street View Images (SVI) are a common source of valuable data for researchers. Researchers have used SVI data for estimating pedestrian volumes, demographic surveillance, and to better understand built and natural environments in cityscapes. However, the most common source of publicly available SVI data is Google Street View. Google Street View images are collected infrequently, making temporal analysis challenging, especially in low population density areas. Our main contribution is the development of an open-source data pipeline for processing 360-degree video recorded from a car-mounted camera. The video data is used to generate SVIs, which then can be used as an input for longitudinal analysis. We demonstrate the use of the pipeline by collecting an SVI dataset over a 38-month longitudinal survey of Seattle, WA, USA during the COVID-19 pandemic. The output of our pipeline is validated through statistical analyses of pedestrian traffic in the images. We confirm known results in the literature and provide new insights into outdoor pedestrian traffic patterns. This study demonstrates the feasibility and value of collecting and using SVI for research purposes beyond what is possible with currently available SVI data. Our methods and dataset represent a first of its kind longitudinal collection and application of SVI data for research purposes. Limitations and future improvements to the data pipeline and case study are also discussed.

Simultaneous deep ultraviolet transmission and scattering microscopy for virtual histology.

In recent years, the emergence of a variety of novel optical microscopy techniques has enabled the generation of virtual optical stains of unlabeled tissue specimens, which have the potential to transform existing clinical histopathology workflows. In this work, we present a simultaneous deep ultraviolet transmission and scattering microscopy system that can produce virtual histology images that show concordance to conventional gold-standard histological processing techniques. The results of this work demonstrate the system's diagnostic potential for characterizing unlabeled thin tissue sections and streamlining histological workflows.

Deep learning-enabled realistic virtual histology with ultraviolet photoacoustic remote sensing microscopy.

The goal of oncologic surgeries is complete tumor resection, yet positive margins are frequently found postoperatively using gold standard H&E-stained histology methods. Frozen section analysis is sometimes performed for rapid intraoperative margin evaluation, albeit with known inaccuracies. Here, we introduce a label-free histological imaging method based on an ultraviolet photoacoustic remote sensing and scattering microscope, combined with unsupervised deep learning using a cycle-consistent generative adversarial network for realistic virtual staining. Unstained tissues are scanned at rates of up to 7 mins/cm2, at resolution equivalent to 400x digital histopathology. Quantitative validation suggests strong concordance with conventional histology in benign and malignant prostate and breast tissues. In diagnostic utility studies we demonstrate a mean sensitivity and specificity of 0.96 and 0.91 in breast specimens, and respectively 0.87 and 0.94 in prostate specimens. We also find virtual stain quality is preferred (P = 0.03) compared to frozen section analysis in a blinded survey of pathologists.

Rapid ultraviolet photoacoustic remote sensing microscopy using voice-coil stage scanning.

There is an unmet need for fast virtual histology technologies that exhibit histological realism and can scan large sections of fresh tissue within intraoperative time-frames. Ultraviolet photoacoustic remote sensing microscopy (UV-PARS) is an emerging imaging modality capable of producing virtual histology images that show good concordance to conventional histology stains. However, a UV-PARS scanning system that can perform rapid intraoperative imaging over mm-scale fields-of-view at fine resolution (<500 nm) has yet to be demonstrated. In this work, we present a UV-PARS system which utilizes voice-coil stage scanning to demonstrate finely resolved images for 2×2 mm2 areas at 500 nm sampling resolution in 1.33 minutes and coarsely resolved images for 4×4 mm2 areas at 900 nm sampling resolution in 2.5 minutes. The results of this work demonstrate the speed and resolution capabilities of the UV-PARS voice-coil system and further develop the potential for UV-PARS microscopy to be employed in a clinical setting.

Investigating mechanisms of laser pulse-induced reflectivity modulations in photoacoustic remote sensing with a 10 million frames-per-second camera.

Photoacoustic remote sensing has been recently developed as an all-optical imaging modality capable of imaging a variety of endogenous contrast agents label-free. Initially predicted laser pulse-induced refractive index perturbation-based interrogation beam reflectivity modulations have been found to be orders of magnitude smaller than those typically observed experimentally. In this report we utilize a 10 million frames-per-second camera to further investigate these predicted reflectivity modulations, while also exploring other potential mechanisms of laser pulse-induced reflectivity modulations. Laser-induced motion is demonstrated both laterally for gold wires suspended and submerged in air and water, respectively, and carbon fibers submerged in water, and axial motion is observed in gold wires submerged in a depth gradient of intralipid solution. This laser-induced sample motion is anticipated to cause reflectivity modulations local to the interrogation beam profile in microscopy set-ups. Non-motion-based maximum intensity modulations of 3% are also observed in gold wires submerged in water, indicating the presence of the originally predicted reflectivity modulations. Overall, these observations are important as they provide a widefield view of laser-pulse interactions unavailable in previous point scanning-based photoacoustic remote sensing microscopy configurations, where observed mechanisms occur on time-scales orders of magnitude faster than equivalent field of view point scanning capabilities.

Fast hybrid optomechanical scanning photoacoustic remote sensing microscopy for virtual histology.

A rapid scanning microscopy method for hematoxylin and eosin (H&E) like images is sought after for interoperative diagnosis of solid tumor margins. The rapid observation and diagnosis of histological samples can greatly lower surgical risk and improve patient outcomes from solid tumor resection surgeries. Photoacoustic remote sensing (PARS) has recently been demonstrated to provide images of virtual H&E stains with excellent concordance with true H&E staining of formalin-fixed, paraffin embedded tissues. By using PARS with constant velocity and 1D galvanometer mirror scanning we acquire large virtual H&E images (10mm x 5mm) of prostate tissue in less than 3.5 minutes without staining, and over two orders of magnitude faster data acquisition than the current PARS imaging speed.

Miniature Deformable MEMS Mirrors for Ultrafast Optical Focusing.

Here, we introduce ultrafast tunable MEMS mirrors consisting of a miniature circular mirrored membrane, which can be electrostatically actuated to change the mirror curvature at unprecedented speeds. The central deflection zone is a close approximation to a parabolic mirror. The device is fabricated with a minimal membrane diameter, but at least double the size of a focused optical spot. The theory and simulations are used to predict maximum relative focal shifts as a function of membrane size and deflection, beam waist, and incident focal position. These devices are demonstrated to enable fast tuning of the focal wavefront of laser beams at ≈MHz tuning rates, two to three orders of magnitude faster than current optical focusing technologies. The fabricated devices have a silicon membrane with a 30-100 µm radius and a 350 nm gap spacing between the top and bottom electrodes. These devices can change the focal position of a tightly focused beam by ≈1 mm at rates up to 4.9 MHz and with response times smaller than 5 µs.

Virtual histopathology with ultraviolet scattering and photoacoustic remote sensing microscopy.

Realistic label-free virtual histopathology has been a long sought-after goal not yet achieved with current methods. Here, we introduce high-resolution hematoxylin and eosin (H&E)-like virtual histology of unstained human breast lumpectomy specimen sections using ultraviolet scattering-augmented photoacoustic remote sensing microscopy. Together with a colormap-matching algorithm based on blind stain separation from a reference true H&E image, we are able to produce virtual H&E images of unstained tissues with close concordance to true H&E-stained sections, with promising diagnostic utility.

Multimodal 3D photoacoustic remote sensing and confocal fluorescence microscopy imaging.

SIGNIFICANCE: Complementary absorption and fluorescence contrast could prove useful for a wide range of biomedical applications. However, current absorption-based photoacoustic microscopy systems require the ultrasound transducers to physically touch the samples, thereby increasing contamination and limiting strong optical focusing in reflection mode. AIM: We sought to develop an all-optical system for imaging cells and tissues using the three combined imaging modalities: photoacoustic remote sensing (PARS), epifluorescence, and confocal laser scanning microscopy (CLSM). APPROACH: A PARS subsystem with ultraviolet excitation was used to obtain label-free absorption-contrast images of nucleic acids in ex vivo tissue samples. Co-integrated epifluorescence and CLSM subsystems were used to verify the 2D and 3D nuclei distribution. RESULTS: Complementary absorption and fluorescence contrast were demonstrated in phantom imaging experiments and subsequent cell and tissue imaging experiments. Lateral and axial resolution of ultraviolet-PARS (UV-PARS) is shown to be 0.39 and 1.6 µm, respectively, with 266-nm light. CLSM lateral and axial resolution was measured as 0.97 and 2.0 µm, respectively. This resolution is sufficient to image individual cell layers with fine optical sectioning. UV-PARS images of cell nuclei are validated in thick tissue using CLSM. CONCLUSIONS: Multimodal absorption and fluorescence contrast are obtained with a non-contact all-optical microscopy system for the first time and utilized to obtain images of cells and tissues with subcellular resolution.

F-mode ultraviolet photoacoustic remote sensing for label-free virtual H&E histopathology using a single excitation wavelength.

Photoacoustic remote sensing (PARS) is a novel all-optical imaging modality that allows for non-contact detection of initial photoacoustic pressures. Using 266-nm excitation pulses, ultraviolet PARS (UV-PARS) has previously demonstrated imaging contrast for cell nuclei in histological samples with <400nm resolution. In prior PARS-based imaging schemes, the signal amplitude at an interrogation point was determined by the maximum deflection from the DC scattering signal in response to a pulsed excitation. This method, however, does not take into consideration additional information encoded in the frequency domain of the recorded PARS signals. Here, we present a frequency domain technique called F-mode PARS that can be used to generate images with nuclear and cytoplasmic enhanced contrast, enabling label-free virtual hematoxylin-and-eosin-like microscopy, using only a single excitation wavelength. With F-mode processing, we have been able to demonstrate contrast-to-noise ratios of up to 38 dB between cell nuclei and surrounding cytoplasm, which represents up to a 25-dB improvement over previous implementations of UV-PARS systems.

Fiber-based photoacoustic remote sensing microscopy and spectral-domain optical coherence tomography with a dual-function 1050-nm interrogation source.

SIGNIFICANCE: Spectral-domain optical coherence tomography (SD-OCT) offers depth-resolved imaging of optical scattering contrast but is limited in sensitivity to optical absorption. Dual-modality imaging combined with the noncontact absorption contrast of photoacoustic remote sensing (PARS) microscopy can augment SD-OCT applications with specific molecular and functional contrasts in an all-optical, fiber-based platform. AIM: To develop a fiber-based multimodal PARS and SD-OCT imaging system, which efficiently uses a common 1050-nm light source for SD-OCT and PARS interrogation. APPROACH: PARS microscopy has predominantly utilized a 1310-nm interrogation light source to date. Hence, a recent dual-modality PARS and 1050-nm SD-OCT imaging system required three distinct wavelengths including a 532-nm PARS excitation, necessitating a free-space optical architecture with discrete subsystems. Here, we validate the first use of a 1050-nm interrogation wavelength for PARS. This enables the transition to fiber-based interferometry as is standard in modern SD-OCT systems, though infeasible with inclusion of an additional 1310-nm wavelength. PARS interrogation functionality is integrated using a broadband optical circulator. RESULTS: Dual-modality imaging is demonstrated in carbon fiber phantoms and a mouse ear in vivo. SD-OCT provided a 4.5-µm lateral resolution, 8.8-µm axial resolution in air, and >101 dB of sensitivity, and PARS contributed 532-nm optical absorption contrast with a 47-dB SNR, and lateral and axial resolutions of 2.4 and 35 µm, respectively. Total interrogation power was reduced from 90% to 58% of the ANSI limit compared to a previous three-wavelength approach. CONCLUSIONS: Adapting PARS to use the 1050-nm SD-OCT light source for interrogation enabled implementation of a fiber-based dual-modality system configuration, with image quality maintained. This will facilitate development of potential applications demanding handheld, catheter-based, or endoscopic form factors.

Virtual hematoxylin and eosin histopathology using simultaneous photoacoustic remote sensing and scattering microscopy.

Hematoxylin and eosin (H&E) staining is the gold standard for most histopathological diagnostics but requires lengthy processing times not suitable for point-of-care diagnosis. Here we demonstrate a 266-nm excitation ultraviolet photoacoustic remote sensing (UV-PARS) and 1310-nm microscopy system capable of virtual H&E 3D imaging of tissues. Virtual hematoxylin staining of nuclei is achieved with UV-PARS, while virtual eosin staining is achieved using the already implemented interrogation laser from UV-PARS for scattering contrast. We demonstrate the capabilities of this dual-contrast system for en-face planar and depth-resolved imaging of human tissue samples exhibiting high concordance with H&E staining procedures and confocal fluorescence microscopy. To our knowledge, this is the first microscopy approach capable of depth-resolved imaging of unstained thick tissues with virtual H&E contrast.

Multimodal imaging with spectral-domain optical coherence tomography and photoacoustic remote sensing microscopy.

We develop a multimodal imaging platform, combining depth-resolved scattering contrast from spectral-domain optical coherence tomography (SD-OCT) with complementary, non-contact absorption contrast using photoacoustic remote sensing (PARS) microscopy. The system provides a widefield OCT mode using a telecentric scan lens, and a high-resolution, dual-contrast mode using a 0.26 numerical aperture apochromatic objective. An interlaced acquisition approach is used to achieve simultaneous, co-registered imaging. The SD-OCT modality provides a 9.7 µm axial resolution. Comprehensive in vivo imaging of a nude mouse ear is demonstrated, with the SD-OCT scattering intensity revealing dermal morphology, and PARS microscopy providing a map of microvasculature.

Label-free lipid contrast imaging using non-contact near-infrared photoacoustic remote sensing microscopy.

Histopathology of lipid-rich tissues is often a difficult endeavor, owing to the limited tissue processing workflows that can appropriately preserve tissue while keeping fatty deposits intact. Here, we present the first usage of near-infrared (NIR) photoacoustic remote sensing (PARS) to achieve imaging contrast from lipids without the need for exogenous stains or labels. In our system, the facile production of 1225 nm excitation pulses is achieved by the stimulated Raman scattering of a 1064 nm source propagating through an optical fiber. PARS-based detection is achieved by monitoring the change in the scattering profile of a co-aligned 1550 nm continuous-wave interrogation beam in response to absorption of the 1225 nm light by lipids. Our non-contact, reflection-mode approach can achieve a FWHM resolution of up to 0.96 µm and signal-to-noise ratios as high as 45 dB from carbon fibers and 9.7 dB from a lipid phantom. NIR-PARS offers a promising approach to image lipid-rich samples with a simplified workflow.