Causal inference on human behaviour.

Making causal inferences regarding human behaviour is difficult given the complex interplay between countless contributors to behaviour, including factors in the external world and our internal states. We provide a non-technical conceptual overview of challenges and opportunities for causal inference on human behaviour. The challenges include our ambiguous causal language and thinking, statistical under- or over-control, effect heterogeneity, interference, timescales of effects and complex treatments. We explain how methods optimized for addressing one of these challenges frequently exacerbate other problems. We thus argue that clearly specified research questions are key to improving causal inference from data. We suggest a triangulation approach that compares causal estimates from (quasi-)experimental research with causal estimates generated from observational data and theoretical assumptions. This approach allows a systematic investigation of theoretical and methodological factors that might lead estimates to converge or diverge across studies.

Computational kinematics of dance: distinguishing hip hop genres.

Dance plays a vital role in human societies across time and culture, with different communities having invented different systems for artistic expression through movement (genres). Differences between genres can be described by experts in words and movements, but these descriptions can only be appreciated by people with certain background abilities. Existing dance notation schemes could be applied to describe genre-differences, however they fall substantially short of being able to capture the important details of movement across a wide spectrum of genres. Our knowledge and practice around dance would benefit from a general, quantitative and human-understandable method of characterizing meaningful differences between aspects of any dance style; a computational kinematics of dance. Here we introduce and apply a novel system for encoding bodily movement as 17 macroscopic, interpretable features, such as expandedness of the body or the frequency of sharp movements. We use this encoding to analyze Hip Hop Dance genres, in part by building a low-cost machine-learning classifier that distinguishes genre with high accuracy. Our study relies on an open dataset (AIST++) of pose-sequences from dancers instructed to perform one of ten Hip Hop genres, such as Breakdance, Popping, or Krump. For comparison we evaluate moderately experienced human observers at discerning these sequence's genres from movements alone (38% where chance = 10%). The performance of a baseline, Ridge classifier model was fair (48%) and that of the model resulting from our automated machine learning pipeline was strong (76%). This indicates that the selected features represent important dimensions of movement for the expression of the attitudes, stories, and aesthetic values manifested in these dance forms. Our study offers a new window into significant relations of similarity and difference between the genres studied. Given the rich, complex, and culturally shaped nature of these genres, the interpretability of our features, and the lightweight techniques used, our approach has significant potential for generalization to other movement domains and movement-related applications.

Neural signatures of natural behaviour in socializing macaques.

Our understanding of the neurobiology of primate behaviour largely derives from artificial tasks in highly controlled laboratory settings, overlooking most natural behaviours that primate brains evolved to produce1-3. How primates navigate the multidimensional social relationships that structure daily life4 and shape survival and reproductive success5 remains largely unclear at the single-neuron level. Here we combine ethological analysis, computer vision and wireless recording technologies to identify neural signatures of natural behaviour in unrestrained, socially interacting pairs of rhesus macaques. Single-neuron and population activity in the prefrontal and temporal cortex robustly encoded 24 species-typical behaviours, as well as social context. Male-female partners demonstrated near-perfect reciprocity in grooming, a key behavioural mechanism supporting friendships and alliances6, and neural activity maintained a running account of these social investments. Confronted with an aggressive intruder, behavioural and neural population responses reflected empathy and were buffered by the presence of a partner. Our findings reveal a highly distributed neurophysiological ledger of social dynamics, a potential computational foundation supporting communal life in primate societies, including our own.

Downstream network transformations dissociate neural activity from causal functional contributions.

Neuroscientists rely on distributed spatio-temporal patterns of neural activity to understand how neural units contribute to cognitive functions and behavior. However, the extent to which neural activity reliably indicates a unit's causal contribution to the behavior is not well understood. To address this issue, we provide a systematic multi-site perturbation framework that captures time-varying causal contributions of elements to a collectively produced outcome. Applying our framework to intuitive toy examples and artificial neural networks revealed that recorded activity patterns of neural elements may not be generally informative of their causal contribution due to activity transformations within a network. Overall, our findings emphasize the limitations of inferring causal mechanisms from neural activities and offer a rigorous lesioning framework for elucidating causal neural contributions.

Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses.

An ideal vision model accounts for behavior and neurophysiology in both naturalistic conditions and designed lab experiments. Unlike psychological theories, artificial neural networks (ANNs) actually perform visual tasks and generate testable predictions for arbitrary inputs. These advantages enable ANNs to engage the entire spectrum of the evidence. Failures of particular models drive progress in a vibrant ANN research program of human vision.

Inferring causal connectivity from pairwise recordings and optogenetics.

To understand the neural mechanisms underlying brain function, neuroscientists aim to quantify causal interactions between neurons, for instance by perturbing the activity of neuron A and measuring the effect on neuron B. Recently, manipulating neuron activity using light-sensitive opsins, optogenetics, has increased the specificity of neural perturbation. However, using widefield optogenetic interventions, multiple neurons are usually perturbed, producing a confound-any of the stimulated neurons can have affected the postsynaptic neuron making it challenging to discern which neurons produced the causal effect. Here, we show how such confounds produce large biases in interpretations. We explain how confounding can be reduced by combining instrumental variables (IV) and difference in differences (DiD) techniques from econometrics. Combined, these methods can estimate (causal) effective connectivity by exploiting the weak, approximately random signal resulting from the interaction between stimulation and the absolute refractory period of the neuron. In simulated neural networks, we find that estimates using ideas from IV and DiD outperform naïve techniques suggesting that methods from causal inference can be useful to disentangle neural interactions in the brain.

Quantifying interaction with robotic toys in pre-term and full-term infants.

Infants born pre-term are at an increased risk for developmental, behavioral, and motor delay and subsequent disability. When these problems are detected early, clinical intervention can be effective at improving functional outcomes. Current methods of early clinical assessment are resource intensive, require extensive training, and do not always capture infants' behavior in natural play environments. We developed the Play and Neuro Development Assessment (PANDA) Gym, an affordable, mechatronic, sensor-based play environment that can be used outside clinical settings to capture infant visual and motor behavior. Using a set of classification codes developed from the literature, we analyzed videos from 24 pre-term and full-term infants as they played with each of three robotic toys designed to elicit different types of interactions-a lion, an orangutan, and an elephant. We manually coded for frequency and duration of toy interactions such as kicking, grasping, touching, and gazing. Pre-term infants gazed at the toys with similar frequency as full-term infants, but infants born full-term physically engaged more frequently and for longer durations with the robotic toys than infants born pre-term. While we showed we could detect differences between full-term and pre-term infants, further work is needed to determine whether differences seen were primarily due to age, developmental delays, or a combination.

A role for cortical interneurons as adversarial discriminators.

The brain learns representations of sensory information from experience, but the algorithms by which it does so remain unknown. One popular theory formalizes representations as inferred factors in a generative model of sensory stimuli, meaning that learning must improve this generative model and inference procedure. This framework underlies many classic computational theories of sensory learning, such as Boltzmann machines, the Wake/Sleep algorithm, and a more recent proposal that the brain learns with an adversarial algorithm that compares waking and dreaming activity. However, in order for such theories to provide insights into the cellular mechanisms of sensory learning, they must be first linked to the cell types in the brain that mediate them. In this study, we examine whether a subtype of cortical interneurons might mediate sensory learning by serving as discriminators, a crucial component in an adversarial algorithm for representation learning. We describe how such interneurons would be characterized by a plasticity rule that switches from Hebbian plasticity during waking states to anti-Hebbian plasticity in dreaming states. Evaluating the computational advantages and disadvantages of this algorithm, we find that it excels at learning representations in networks with recurrent connections but scales poorly with network size. This limitation can be partially addressed if the network also oscillates between evoked activity and generative samples on faster timescales. Consequently, we propose that an adversarial algorithm with interneurons as discriminators is a plausible and testable strategy for sensory learning in biological systems.

Neural signatures of natural behavior in socializing macaques.

Our understanding of the neurobiology of primate behavior largely derives from artificial tasks in highly-controlled laboratory settings, overlooking most natural behaviors primate brains evolved to produce1. In particular, how primates navigate the multidimensional social relationships that structure daily life and shape survival and reproductive success remains largely unexplored at the single neuron level. Here, we combine ethological analysis with new wireless recording technologies to uncover neural signatures of natural behavior in unrestrained, socially interacting pairs of rhesus macaques within a larger colony. Population decoding of single neuron activity in prefrontal and temporal cortex unveiled robust encoding of 24 species-typical behaviors, which was strongly modulated by the presence and identity of surrounding monkeys. Male-female partners demonstrated near-perfect reciprocity in grooming, a key behavioral mechanism supporting friendships and alliances, and neural activity maintained a running account of these social investments. When confronted with an aggressive intruder, behavioral and neural population responses reflected empathy and were buffered by the presence of a partner. Surprisingly, neural signatures in prefrontal and temporal cortex were largely indistinguishable and irreducible to visual and motor contingencies. By employing an ethological approach to the study of primate neurobiology, we reveal a highly-distributed neurophysiological record of social dynamics, a potential computational foundation supporting communal life in primate societies, including our own.