Temporal shuffling for defending deep action recognition models against adversarial attacks.

Recently, video-based action recognition methods using convolutional neural networks (CNNs) achieve remarkable recognition performance. However, there is still lack of understanding about the generalization mechanism of action recognition models. In this paper, we suggest that action recognition models rely on the motion information less than expected, and thus they are robust to randomization of frame orders. Furthermore, we find that motion monotonicity remaining after randomization also contributes to such robustness. Based on this observation, we develop a novel defense method using temporal shuffling of input videos against adversarial attacks for action recognition models. Another observation enabling our defense method is that adversarial perturbations on videos are sensitive to temporal destruction. To the best of our knowledge, this is the first attempt to design a defense method without additional training for 3D CNN-based video action recognition models.

Phenotyping senescent mesenchymal stromal cells using AI image translation.

Mesenchymal stromal cells (MSCs) offer promising potential in biomedical research, clinical therapeutics, and immunomodulatory therapies due to their ease of isolation and multipotent, immunoprivileged, and immunosuppersive properties. Extensive efforts have focused on optimizing the cell isolation and culture methods to generate scalable, therapeutically-relevant MSCs for clinical applications. However, MSC-based therapies are often hindered by cell heterogeneity and inconsistency of therapeutic function caused, in part, by MSC senescence. As such, noninvasive and molecular-based MSC characterizations play an essential role in assuring the consistency of MSC functions. Here, we demonstrated that AI image translation algorithms can effectively predict immunofluorescence images of MSC senescence markers from phase contrast images. We showed that the expression level of senescence markers including senescence-associated beta-galactosidase (SABG), p16, p21, and p38 are accurately predicted by deep-learning models for Doxorubicin-induced MSC senescence, irradiation-induced MSC senescence, and replicative MSC senescence. Our AI model distinguished the non-senescent and senescent MSC populations and simultaneously captured the cell-to-cell variability within a population. Our microscopy-based phenotyping platform can be integrated with cell culture routines making it an easily accessible tool for MSC engineering and manufacturing.

Stochastic Optimization for Nonconvex Problem With Inexact Hessian Matrix, Gradient, and Function.

Trust region (TR) and adaptive regularization using cubics (ARC) have proven to have some very appealing theoretical properties for nonconvex optimization by concurrently computing function value, gradient, and Hessian matrix to obtain the next search direction and the adjusted parameters. Although stochastic approximations help largely reduce the computational cost, it is challenging to theoretically guarantee the convergence rate. In this article, we explore a family of stochastic TR (STR) and stochastic ARC (SARC) methods that can simultaneously provide inexact computations of the Hessian matrix, gradient, and function values. Our algorithms require much fewer propagations overhead per iteration than TR and ARC. We prove that the iteration complexity to achieve ϵ -approximate second-order optimality is of the same order as the exact computations demonstrated in previous studies. In addition, the mild conditions on inexactness can be met by leveraging a random sampling technology in the finite-sum minimization problem. Numerical experiments with a nonconvex problem support these findings and demonstrate that, with the same or a similar number of iterations, our algorithms require less computational overhead per iteration than current second-order methods.

Trustworthy in silico cell labeling via ensemble-based image translation.

Artificial intelligence (AI) image translation has been a valuable tool for processing image data in biological and medical research. To apply such a tool in mission-critical applications, including drug screening, toxicity study, and clinical diagnostics, it is essential to ensure that the AI prediction is trustworthy. Here, we demonstrate that an ensemble learning method can quantify the uncertainty of AI image translation. We tested the uncertainty evaluation using experimentally acquired images of mesenchymal stromal cells. We find that the ensemble method reports a prediction standard deviation that correlates with the prediction error, estimating the prediction uncertainty. We show that this uncertainty is in agreement with the prediction error and Pearson correlation coefficient. We further show that the ensemble method can detect out-of-distribution input images by reporting increased uncertainty. Altogether, these results suggest that the ensemble-estimated uncertainty can be a useful indicator for identifying erroneous AI image translations.

Stochastically Controlled Compositional Gradient for Composition Problems.

We consider composition problems of the form (1/n)∑i = 1n Fi (1/m)∑j = 1m Gj(x) , which are important for machine learning. Although gradient descent and stochastic gradient descent are straightforward solutions, the essential computation of G (x) = (1/m)∑j = 1mGj(x) in each single iteration is expensive, let alone for large m . In this article, we devise a stochastically controlled compositional gradient algorithm. Specifically, we introduce two variants of stochastically controlled technique to estimate the inner function G(x) and the gradient of the objective function, respectively. The computational cost is largely reduced. However, the natural needs of two stochastic subsets D1 and D2 form direct barriers to guarantee the convergence of the algorithm, especially the theoretical proof of the convergence. To this end, we present a general convergence analysis by proving | D1|=min{1/ϵ,m} and | D2|=min{1/ϵ,n } , through which the proposed method significantly improve composition algorithms under low target accuracy (i.e., 1/ϵ << m or n ) in both strongly convex and nonconvex settings. Comprehensive experiments demonstrate the superiority of the proposed method over existing methods.

Supracellular measurement of spatially varying mechanical heterogeneities in live monolayers.

The mechanical properties of tissues have profound impacts on a wide range of biological processes such as embryo development (1,2), wound healing (3-6), and disease progression (7). Specifically, the spatially varying moduli of cells largely influence the local tissue deformation and intercellular interaction. Despite the importance of characterizing such a heterogeneous mechanical property, it has remained difficult to measure the supracellular modulus field in live cell layers with a high-throughput and minimal perturbation. In this work, we developed a monolayer effective modulus measurement by integrating a custom cell stretcher, light microscopy, and AI-based inference. Our approach first quantifies the heterogeneous deformation of a slightly stretched cell layer and converts the measured strain fields into an effective modulus field using an AI inference. This method allowed us to directly visualize the effective modulus distribution of thousands of cells virtually instantly. We characterized the mean value, SD, and correlation length of the effective cell modulus for epithelial cells and fibroblasts, which are in agreement with previous results. We also observed a mild correlation between cell area and stiffness in jammed epithelia, suggesting the influence of cell modulus on packing. Overall, our reported experimental platform provides a valuable alternative cell mechanics measurement tool that can be integrated with microscopy-based characterizations.

Local Critic Training for Model-Parallel Learning of Deep Neural Networks.

In this article, we propose a novel model-parallel learning method, called local critic training, which trains neural networks using additional modules called local critic networks. The main network is divided into several layer groups, and each layer group is updated through error gradients estimated by the corresponding local critic network. We show that the proposed approach successfully decouples the update process of the layer groups for both convolutional neural networks (CNNs) and recurrent neural networks (RNNs). In addition, we demonstrate that the proposed method is guaranteed to converge to a critical point. We also show that trained networks by the proposed method can be used for structural optimization. Experimental results show that our method achieves satisfactory performance, reduces training time greatly, and decreases memory consumption per machine. Code is available at https://github.com/hjdw2/Local-critic-training.

Investigating heterogeneities of live mesenchymal stromal cells using AI-based label-free imaging.

Mesenchymal stromal cells (MSCs) are multipotent cells that have great potential for regenerative medicine, tissue repair, and immunotherapy. Unfortunately, the outcomes of MSC-based research and therapies can be highly inconsistent and difficult to reproduce, largely due to the inherently significant heterogeneity in MSCs, which has not been well investigated. To quantify cell heterogeneity, a standard approach is to measure marker expression on the protein level via immunochemistry assays. Performing such measurements non-invasively and at scale has remained challenging as conventional methods such as flow cytometry and immunofluorescence microscopy typically require cell fixation and laborious sample preparation. Here, we developed an artificial intelligence (AI)-based method that converts transmitted light microscopy images of MSCs into quantitative measurements of protein expression levels. By training a U-Net+ conditional generative adversarial network (cGAN) model that accurately (mean [Formula: see text] = 0.77) predicts expression of 8 MSC-specific markers, we showed that expression of surface markers provides a heterogeneity characterization that is complementary to conventional cell-level morphological analyses. Using this label-free imaging method, we also observed a multi-marker temporal-spatial fluctuation of protein distributions in live MSCs. These demonstrations suggest that our AI-based microscopy can be utilized to perform quantitative, non-invasive, single-cell, and multi-marker characterizations of heterogeneous live MSC culture. Our method provides a foundational step toward the instant integrative assessment of MSC properties, which is critical for high-throughput screening and quality control in cellular therapies.

Emotional EEG classification using connectivity features and convolutional neural networks.

Convolutional neural networks (CNNs) are widely used to recognize the user's state through electroencephalography (EEG) signals. In the previous studies, the EEG signals are usually fed into the CNNs in the form of high-dimensional raw data. However, this approach makes it difficult to exploit the brain connectivity information that can be effective in describing the functional brain network and estimating the perceptual state of the user. We introduce a new classification system that utilizes brain connectivity with a CNN and validate its effectiveness via the emotional video classification by using three different types of connectivity measures. Furthermore, two data-driven methods to construct the connectivity matrix are proposed to maximize classification performance. Further analysis reveals that the level of concentration of the brain connectivity related to the emotional property of the target video is correlated with classification performance.