SPICEMIX enables integrative single-cell spatial modeling of cell identity.

Spatial transcriptomics can reveal spatially resolved gene expression of diverse cells in complex tissues. However, the development of computational methods that can use the unique properties of spatial transcriptome data to unveil cell identities remains a challenge. Here we introduce SPICEMIX, an interpretable method based on probabilistic, latent variable modeling for joint analysis of spatial information and gene expression from spatial transcriptome data. Both simulation and real data evaluations demonstrate that SPICEMIX markedly improves on the inference of cell types and their spatial patterns compared with existing approaches. By applying to spatial transcriptome data of brain regions in human and mouse acquired by seqFISH+, STARmap and Visium, we show that SPICEMIX can enhance the inference of complex cell identities, reveal interpretable spatial metagenes and uncover differentiation trajectories. SPICEMIX is a generalizable analysis framework for spatial transcriptome data to investigate cell-type composition and spatial organization of cells in complex tissues.

A Multi-Organ Nucleus Segmentation Challenge.

Generalized nucleus segmentation techniques can contribute greatly to reducing the time to develop and validate visual biomarkers for new digital pathology datasets. We summarize the results of MoNuSeg 2018 Challenge whose objective was to develop generalizable nuclei segmentation techniques in digital pathology. The challenge was an official satellite event of the MICCAI 2018 conference in which 32 teams with more than 80 participants from geographically diverse institutes participated. Contestants were given a training set with 30 images from seven organs with annotations of 21,623 individual nuclei. A test dataset with 14 images taken from seven organs, including two organs that did not appear in the training set was released without annotations. Entries were evaluated based on average aggregated Jaccard index (AJI) on the test set to prioritize accurate instance segmentation as opposed to mere semantic segmentation. More than half the teams that completed the challenge outperformed a previous baseline. Among the trends observed that contributed to increased accuracy were the use of color normalization as well as heavy data augmentation. Additionally, fully convolutional networks inspired by variants of U-Net, FCN, and Mask-RCNN were popularly used, typically based on ResNet or VGG base architectures. Watershed segmentation on predicted semantic segmentation maps was a popular post-processing strategy. Several of the top techniques compared favorably to an individual human annotator and can be used with confidence for nuclear morphometrics.

Rotation equivariant and invariant neural networks for microscopy image analysis.

MOTIVATION: Neural networks have been widely used to analyze high-throughput microscopy images. However, the performance of neural networks can be significantly improved by encoding known invariance for particular tasks. Highly relevant to the goal of automated cell phenotyping from microscopy image data is rotation invariance. Here we consider the application of two schemes for encoding rotation equivariance and invariance in a convolutional neural network, namely, the group-equivariant CNN (G-CNN), and a new architecture with simple, efficient conic convolution, for classifying microscopy images. We additionally integrate the 2D-discrete-Fourier transform (2D-DFT) as an effective means for encoding global rotational invariance. We call our new method the Conic Convolution and DFT Network (CFNet). RESULTS: We evaluated the efficacy of CFNet and G-CNN as compared to a standard CNN for several different image classification tasks, including simulated and real microscopy images of subcellular protein localization, and demonstrated improved performance. We believe CFNet has the potential to improve many high-throughput microscopy image analysis applications. AVAILABILITY AND IMPLEMENTATION: Source code of CFNet is available at: https://github.com/bchidest/CFNet. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Ultrasensitive Three-Dimensional Orientation Imaging of Single Molecules on Plasmonic Nanohole Arrays Using Second Harmonic Generation.

Recently, fluorescence-based super-resolution techniques such as stimulated emission depletion (STED) and stochastic optical reconstruction microscopy (STORM) have been developed to achieve near molecular-scale resolution. However, such a super-resolution technique for nonlinear label-free microscopy based on second harmonic generation (SHG) is lacking. Since SHG is label-free and does not involve real-energy level transitions, fluorescence-based super-resolution techniques such as STED cannot be applied to improve the resolution. In addition, due to the coherent and non-isotropic emission nature of SHG, single-molecule localization techniques based on isotropic emission of fluorescent molecule such as STORM will not be appropriate. Single molecule SHG microscopy is largely hindered due to the very weak nonlinear optical scattering cross sections of SHG scattering processes. Thus, enhancing SHG using plasmonic nanostructures and nanoantennas has recently gained much attention owing to the potential of various nanoscale geometries to tightly confine electromagnetic fields into small volumes. This confinement provides substantial enhancement of electromagnetic field in nanoscale regions of interest, which can significantly boost the nonlinear signal produced by molecules located in the plasmonic hotspots. However, to date, plasmon-enhanced SHG has been primarily applied for the measurement of bulk properties of the materials/molecules, and single molecule SHG imaging along with its orientation information has not been realized yet. Herein, we achieved simultaneous visualization and three-dimensional (3D) orientation imaging of individual rhodamine 6G (R6G) molecules in the presence of plasmonic silver nanohole arrays. SHG and two-photon fluorescence microscopy experiments together with finite-difference time-domain (FDTD) simulations revealed a ∼106-fold nonlinear enhancement factor at the hot spots on the plasmonic silver nanohole substrate, enabling detection of single molecules using SHG. The position and 3D orientation of R6G molecules were determined using the template matching algorithm by comparing the experimental data with the calculated dipole emission images. These findings could enable SHG-based single molecule detection and orientation imaging of molecules which could lead to a wide range of applications from nanophotonics to super-resolution SHG imaging of biological cells and tissues.

Discriminative bag-of-cells for imaging-genomics.

Connecting genotypes to image phenotypes is crucial for a comprehensive understanding of cancer. To learn such connections, new machine learning approaches must be developed for the better integration of imaging and genomic data. Here we propose a novel approach called Discriminative Bag-of-Cells (DBC) for predicting genomic markers using imaging features, which addresses the challenge of summarizing histopathological images by representing cells with learned discriminative types, or codewords. We also developed a reliable and efficient patch-based nuclear segmentation scheme using convolutional neural networks from which nuclear and cellular features are extracted. Applying DBC on TCGA breast cancer samples to predict basal subtype status yielded a class-balanced accuracy of 70% on a separate test partition of 213 patients. As data sets of imaging and genomic data become increasingly available, we believe DBC will be a useful approach for screening histopathological images for genomic markers. Source code of nuclear segmentation and DBC are available at: https://github.com/bchidest/DBC.

A quantitative framework for the analysis of multimodal optical microscopy images.

BACKGROUND: Multimodal optical microscopy, a set of imaging techniques based on unique, yet complementary contrast mechanisms and spatially and temporally co-registered data acquisition, has emerged as a powerful biomedical tool. However, the analysis of the dense, high-dimensional datasets acquired by these instruments remains mostly qualitative and restricted to analysis of each modality individually. METHODS: Using a custom-built multimodal nonlinear optical microscope, high dimensional datasets were acquired for automated classification of functional cell states as well as identification of histopathological features in tissues slices. Supervised classification of cell death modes was performed through support vector machines (SVM) and semi-supervised classification of tissue slices was performed through the use of the expectation maximization (EM) algorithm. RESULTS: Applications of these techniques to the automated classification of cell death modes as well as to the identification of tissue components in fixed ex vivo tissue slices are presented. The analysis techniques developed provide a direct link between multimodal image contrast and biological structure and function, resulting in highly accurate classification in both settings. CONCLUSIONS: Quantification of multimodal optical microscopy images through statistical modeling of the high dimensional data acquired gives a strong correlation between biological structure and function and image contrast. These methods are sensitive to the identification of diagnostic, cellular-level features important in a variety of clinical settings.

Efficient hybrid tree-based stereo matching with applications to postcapture image refocusing.

Estimating dense correspondence or depth information from a pair of stereoscopic images is a fundamental problem in computer vision, which finds a range of important applications. Despite intensive past research efforts in this topic, it still remains challenging to recover the depth information both reliably and efficiently, especially when the input images contain weakly textured regions or are captured under uncontrolled, real-life conditions. Striking a desired balance between computational efficiency and estimation quality, a hybrid minimum spanning tree-based stereo matching method is proposed in this paper. Our method performs efficient nonlocal cost aggregation at pixel-level and region-level, and then adaptively fuses the resulting costs together to leverage their respective strength in handling large textureless regions and fine depth discontinuities. Experiments on the standard Middlebury stereo benchmark show that the proposed stereo method outperforms all prior local and nonlocal aggregation-based methods, achieving particularly noticeable improvements for low texture regions. To further demonstrate the effectiveness of the proposed stereo method, also motivated by the increasing desire to generate expressive depth-induced photo effects, this paper is tasked next to address the emerging application of interactive depth-of-field rendering given a real-world stereo image pair. To this end, we propose an accurate thin-lens model for synthetic depth-of-field rendering, which considers the user-stroke placement and camera-specific parameters and performs the pixel-adapted Gaussian blurring in a principled way. Taking ~1.5 s to process a pair of 640×360 images in the off-line step, our system named Scribble2focus allows users to interactively select in-focus regions by simple strokes using the touch screen and returns the synthetically refocused images instantly to the user.

The use of ivermectin to kill ixodes scapularis ticks feeding on humans.

OBJECTIVE: The purpose of this study was to determine whether 400 µg/kg oral ivermectin is able to kill Ixodes scapularis nymphs and adult female ticks feeding on humans. METHODS: Ten study subjects each wore 2 ostomy bags, the one containing 24 I scapularis nymphs, and the other containing 24 I scapularis adult females. Twenty-four hours after the ostomy bags were attached, study subjects received either 400 µg/kg ivermectin or placebo. Thirty hours after the ivermectin or placebo was consumed, the ticks were removed, and mortality determined in a double-blinded manner. RESULTS: Eleven percent of the I scapularis nymphs attached in the ivermectin group compared with 17% in the placebo. Mortality for the I scapularis nymphs that attached at the time of removal was 55% in the ivermectin group and 47% in the placebo group. Mortality for the I scapularis nymphs 5 days after removal was 92% in the ivermectin group and 88% for the placebo. Three percent of the I scapularis adults attached in the ivermectin group compared with 9% in the placebo group. Mortality for I scapularis adults was 0% on day 3 and 33% on day 8 for both the ivermectin and placebo groups. There were statistically insignificant differences in the mortality rates between I scapularis nymphs and adults exposed to ivermectin or placebo. CONCLUSIONS: There were a high number of ticks that died in both groups but the data do not support our hypothesis that ivermectin can kill I scapularis. The study was not designed to determine whether it could prevent the transmission of tick-borne illness.