Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities.

Recently, zero-shot (or training-free) Neural Architecture Search (NAS) approaches have been proposed to liberate NAS from the expensive training process. The key idea behind zero-shot NAS approaches is to design proxies that can predict the accuracy of some given networks without training the network parameters. The proxies proposed so far are usually inspired by recent progress in theoretical understanding of deep learning and have shown great potential on several datasets and NAS benchmarks. This paper aims to comprehensively review and compare the state-of-the-art (SOTA) zero-shot NAS approaches, with an emphasis on their hardware awareness. To this end, we first review the mainstream zero-shot proxies and discuss their theoretical underpinnings. We then compare these zero-shot proxies through large-scale experiments and demonstrate their effectiveness in both hardware-aware and hardware-oblivious NAS scenarios. Finally, we point out several promising ideas to design better proxies. Our source code and the list of related papers are available on https://github.com/SLDGroup/survey-zero-shot-nas.

MetaNN: accurate classification of host phenotypes from metagenomic data using neural networks.

BACKGROUND: Microbiome profiles in the human body and environment niches have become publicly available due to recent advances in high-throughput sequencing technologies. Indeed, recent studies have already identified different microbiome profiles in healthy and sick individuals for a variety of diseases; this suggests that the microbiome profile can be used as a diagnostic tool in identifying the disease states of an individual. However, the high-dimensional nature of metagenomic data poses a significant challenge to existing machine learning models. Consequently, to enable personalized treatments, an efficient framework that can accurately and robustly differentiate between healthy and sick microbiome profiles is needed. RESULTS: In this paper, we propose MetaNN (i.e., classification of host phenotypes from Metagenomic data using Neural Networks), a neural network framework which utilizes a new data augmentation technique to mitigate the effects of data over-fitting. CONCLUSIONS: We show that MetaNN outperforms existing state-of-the-art models in terms of classification accuracy for both synthetic and real metagenomic data. These results pave the way towards developing personalized treatments for microbiome related diseases.

Weighted Betweenness Preferential Attachment: A New Mechanism Explaining Social Network Formation and Evolution.

The dynamics of social networks is a complex process, as there are many factors which contribute to the formation and evolution of social links. While certain real-world properties are captured by the degree-driven preferential attachment model, it still cannot fully explain social network dynamics. Indeed, important properties such as dynamic community formation, link weight evolution, or degree saturation cannot be completely and simultaneously described by state of the art models. In this paper, we explore the distribution of social network parameters and centralities and argue that node degree is not the main attractor of new social links. Consequently, as node betweenness proves to be paramount to attracting new links - as well as strengthening existing links -, we propose the new Weighted Betweenness Preferential Attachment (WBPA) model, which renders quantitatively robust results on realistic network metrics. Moreover, we support our WBPA model with a socio-psychological interpretation, that offers a deeper understanding of the mechanics behind social network dynamics.

MPLasso: Inferring microbial association networks using prior microbial knowledge.

Due to the recent advances in high-throughput sequencing technologies, it becomes possible to directly analyze microbial communities in human body and environment. To understand how microbial communities adapt, develop, and interact with the human body and the surrounding environment, one of the fundamental challenges is to infer the interactions among different microbes. However, due to the compositional and high-dimensional nature of microbial data, statistical inference cannot offer reliable results. Consequently, new approaches that can accurately and robustly estimate the associations (putative interactions) among microbes are needed to analyze such compositional and high-dimensional data. We propose a novel framework called Microbial Prior Lasso (MPLasso) which integrates graph learning algorithm with microbial co-occurrences and associations obtained from scientific literature by using automated text mining. We show that MPLasso outperforms existing models in terms of accuracy, microbial network recovery rate, and reproducibility. Furthermore, the association networks we obtain from the Human Microbiome Project datasets show credible results when compared against laboratory data.

Multi-fractal characterization of bacterial swimming dynamics: a case study on real and simulated Serratia marcescens.

To add to the current state of knowledge about bacterial swimming dynamics, in this paper, we study the fractal swimming dynamics of populations of Serratia marcescens bacteria both in vitro and in silico, while accounting for realistic conditions like volume exclusion, chemical interactions, obstacles and distribution of chemoattractant in the environment. While previous research has shown that bacterial motion is non-ergodic, we demonstrate that, besides the non-ergodicity, the bacterial swimming dynamics is multi-fractal in nature. Finally, we demonstrate that the multi-fractal characteristic of bacterial dynamics is strongly affected by bacterial density and chemoattractant concentration.

In Silico Evaluation of the Impacts of Quorum Sensing Inhibition (QSI) on Strain Competition and Development of QSI Resistance.

As understanding of bacterial regulatory systems and pathogenesis continues to increase, QSI has been a major focus of research. However, recent studies have shown that mechanisms of resistance to quorum sensing (QS) inhibitors (QSIs) exist, calling into question their clinical value. We propose a computational framework that considers bacteria genotypes relative to QS genes and QS-regulated products including private, quasi-public, and public goods according to their impacts on bacterial fitness. Our results show (1) QSI resistance spreads when QS positively regulates the expression of private or quasi-public goods. (2) Resistance to drugs targeting secreted compounds downstream of QS for a mix of private, public, and quasi-public goods also spreads. (3) Changing the micro-environment during treatment with QSIs may decrease the spread of resistance. At fundamental-level, our simulation framework allows us to directly quantify cell-cell interactions and biofilm dynamics. Practically, the model provides a valuable tool for the study of QSI-based therapies, and the simulations reveal experimental paths that may guide QSI-based therapies in a manner that avoids or decreases the spread of QSI resistance.

Data-Driven Engineering of Social Dynamics: Pattern Matching and Profit Maximization.

In this paper, we define a new problem related to social media, namely, the data-driven engineering of social dynamics. More precisely, given a set of observations from the past, we aim at finding the best short-term intervention that can lead to predefined long-term outcomes. Toward this end, we propose a general formulation that covers two useful engineering tasks as special cases, namely, pattern matching and profit maximization. By incorporating a deep learning model, we derive a solution using convex relaxation and quadratic-programming transformation. Moreover, we propose a data-driven evaluation method in place of the expensive field experiments. Using a Twitter dataset, we demonstrate the effectiveness of our dynamics engineering approach for both pattern matching and profit maximization, and study the multifaceted interplay among several important factors of dynamics engineering, such as solution validity, pattern-matching accuracy, and intervention cost. Finally, the method we propose is general enough to work with multi-dimensional time series, so it can potentially be used in many other applications.

Multi-scale compositionality: identifying the compositional structures of social dynamics using deep learning.

OBJECTIVE: Social media exhibit rich yet distinct temporal dynamics which cover a wide range of different scales. In order to study this complex dynamics, two fundamental questions revolve around (1) the signatures of social dynamics at different time scales, and (2) the way in which these signatures interact and form higher-level meanings. METHOD: In this paper, we propose the Recursive Convolutional Bayesian Model (RCBM) to address both of these fundamental questions. The key idea behind our approach consists of constructing a deep-learning framework using specialized convolution operators that are designed to exploit the inherent heterogeneity of social dynamics. RCBM's runtime and convergence properties are guaranteed by formal analyses. RESULTS: Experimental results show that the proposed method outperforms the state-of-the-art approaches both in terms of solution quality and computational efficiency. Indeed, by applying the proposed method on two social network datasets, Twitter and Yelp, we are able to identify the compositional structures that can accurately characterize the complex social dynamics from these two social media. We further show that identifying these patterns can enable new applications such as anomaly detection and improved social dynamics forecasting. Finally, our analysis offers new insights on understanding and engineering social media dynamics, with direct applications to opinion spreading and online content promotion.

Analytical modeling and experimental characterization of chemotaxis in Serratia marcescens.

This paper presents a modeling and experimental framework to characterize the chemotaxis of Serratia marcescens (S. marcescens) relying on two-dimensional and three-dimensional tracking of individual bacteria. Previous studies mainly characterized bacterial chemotaxis based on population density analysis. Instead, this study focuses on single-cell tracking and measuring the chemotactic drift velocity V(C) from the biased tumble rate of individual bacteria on exposure to a concentration gradient of l-aspartate. The chemotactic response of S. marcescens is quantified over a range of concentration gradients (10^{-3} to 5 mM/mm) and average concentrations (0.5 × 10(-3) to 2.5 mM). Through the analysis of a large number of bacterial swimming trajectories, the tumble rate is found to have a significant bias with respect to the swimming direction. We also verify the relative gradient sensing mechanism in the chemotaxis of S. marcescens by measuring the change of V(C) with the average concentration and the gradient. The applied full pathway model with fitted parameters matches the experimental data. Finally, we show that our measurements based on individual bacteria lead to the determination of the motility coefficient µ (7.25 × 10(-6) cm(2)/s) of a population. The experimental characterization and simulation results for the chemotaxis of this bacterial species contribute towards using S. marcescens in chemically controlled biohybrid systems.

Heterogeneous structure of stem cells dynamics: statistical models and quantitative predictions.

Understanding stem cell (SC) population dynamics is essential for developing models that can be used in basic science and medicine, to aid in predicting cells fate. These models can be used as tools e.g. in studying patho-physiological events at the cellular and tissue level, predicting (mal)functions along the developmental course, and personalized regenerative medicine. Using time-lapsed imaging and statistical tools, we show that the dynamics of SC populations involve a heterogeneous structure consisting of multiple sub-population behaviors. Using non-Gaussian statistical approaches, we identify the co-existence of fast and slow dividing subpopulations, and quiescent cells, in stem cells from three species. The mathematical analysis also shows that, instead of developing independently, SCs exhibit a time-dependent fractal behavior as they interact with each other through molecular and tactile signals. These findings suggest that more sophisticated models of SC dynamics should view SC populations as a collective and avoid the simplifying homogeneity assumption by accounting for the presence of more than one dividing sub-population, and their multi-fractal characteristics.