Classification and analysis of retinal interneurons by computational structure under natural scenes.

Inhibitory neurons are diverse across the brain, but for the visual system we lack the ability to functionally classify these neurons under complex natural stimuli. Here we take the approach of classifying retinal amacrine cell responses to natural scenes using optical recording and an interpretable neural network model. We fit mouse amacrine cell responses to a two-layer convolutional neural network model of a class shown previously to accurately capture salamander ganglion cell responses to natural scenes. Using an approach from interpretable machine learning, we determined for each stimulus the model interneurons that generated each amacrine response, analogous to the set of bipolar cells that target the amacrine population. From this analysis we clustered amacrine cells not by their natural scene responses, but by the model presynaptic neurons that constructed those responses, conservatively finding approximately seven groups by this approach. By analyzing the set of model presynaptic input neurons for each amacrine cluster, we find that distributed rather than dedicated inputs generate natural scene responses for different amacrine cell types. Additional analyses revealed distinct transient and sustained modes exhibited by the network during the response to simple flashes. These results give insight into the computational structure of how the diverse amacrine cell population responds to natural scenes, and generate multiple quantitative hypotheses for how synaptic inputs generate those responses.

HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution.

Genomic (DNA) sequences encode an enormous amount of information for gene regulation and protein synthesis. Similar to natural language models, researchers have proposed foundation models in genomics to learn generalizable features from unlabeled genome data that can then be fine-tuned for downstream tasks such as identifying regulatory elements. Due to the quadratic scaling of attention, previous Transformer-based genomic models have used 512 to 4k tokens as context (<0.001% of the human genome), significantly limiting the modeling of long-range interactions in DNA. In addition, these methods rely on tokenizers or fixed k-mers to aggregate meaningful DNA units, losing single nucleotide resolution where subtle genetic variations can completely alter protein function via single nucleotide polymorphisms (SNPs). Recently, Hyena, a large language model based on implicit convolutions was shown to match attention in quality while allowing longer context lengths and lower time complexity. Leveraging Hyena's new long-range capabilities, we present HyenaDNA, a genomic foundation model pretrained on the human reference genome with context lengths of up to 1 million tokens at the single nucleotide-level - an up to 500x increase over previous dense attention-based models. HyenaDNA scales sub-quadratically in sequence length (training up to 160x faster than Transformer), uses single nucleotide tokens, and has full global context at each layer. We explore what longer context enables - including the first use of in-context learning in genomics. On fine-tuned benchmarks from the Nucleotide Transformer, HyenaDNA reaches state-of-the-art (SotA) on 12 of 18 datasets using a model with orders of magnitude less parameters and pretraining data. On the GenomicBenchmarks, HyenaDNA surpasses SotA on 7 of 8 datasets on average by +10 accuracy points. Code at https://github.com/HazyResearch/hyena-dna.

Interpreting the retinal neural code for natural scenes: From computations to neurons.

Understanding the circuit mechanisms of the visual code for natural scenes is a central goal of sensory neuroscience. We show that a three-layer network model predicts retinal natural scene responses with an accuracy nearing experimental limits. The model's internal structure is interpretable, as interneurons recorded separately and not modeled directly are highly correlated with model interneurons. Models fitted only to natural scenes reproduce a diverse set of phenomena related to motion encoding, adaptation, and predictive coding, establishing their ethological relevance to natural visual computation. A new approach decomposes the computations of model ganglion cells into the contributions of model interneurons, allowing automatic generation of new hypotheses for how interneurons with different spatiotemporal responses are combined to generate retinal computations, including predictive phenomena currently lacking an explanation. Our results demonstrate a unified and general approach to study the circuit mechanisms of ethological retinal computations under natural visual scenes.

Information Geometry of the Retinal Representation Manifold.

The ability for the brain to discriminate among visual stimuli is constrained by their retinal representations. Previous studies of visual discriminability have been limited to either low-dimensional artificial stimuli or pure theoretical considerations without a realistic encoding model. Here we propose a novel framework for understanding stimulus discriminability achieved by retinal representations of naturalistic stimuli with the method of information geometry. To model the joint probability distribution of neural responses conditioned on the stimulus, we created a stochastic encoding model of a population of salamander retinal ganglion cells based on a three-layer convolutional neural network model. This model not only accurately captured the mean response to natural scenes but also a variety of second-order statistics. With the model and the proposed theory, we computed the Fisher information metric over stimuli to study the most discriminable stimulus directions. We found that the most discriminable stimulus varied substantially across stimuli, allowing an examination of the relationship between the most discriminable stimulus and the current stimulus. By examining responses generated by the most discriminable stimuli we further found that the most discriminative response mode is often aligned with the most stochastic mode. This finding carries the important implication that under natural scenes, retinal noise correlations are information-limiting rather than increasing information transmission as has been previously speculated. We additionally observed that sensitivity saturates less in the population than for single cells and that as a function of firing rate, Fisher information varies less than sensitivity. We conclude that under natural scenes, population coding benefits from complementary coding and helps to equalize the information carried by different firing rates, which may facilitate decoding of the stimulus under principles of information maximization.

Synchronous inhibitory pathways create both efficiency and diversity in the retina.

Sensory receptive fields combine features that originate in different neural pathways. Retinal ganglion cell receptive fields compute intensity changes across space and time using a peripheral region known as the surround, a property that improves information transmission about natural scenes. The visual features that construct this fundamental property have not been quantitatively assigned to specific interneurons. Here, we describe a generalizable approach using simultaneous intracellular and multielectrode recording to directly measure and manipulate the sensory feature conveyed by a neural pathway to a downstream neuron. By directly controlling the gain of individual interneurons in the circuit, we show that rather than transmitting different temporal features, inhibitory horizontal cells and linear amacrine cells synchronously create the linear surround at different spatial scales and that these two components fully account for the surround. By analyzing a large population of ganglion cells, we observe substantial diversity in the relative contribution of amacrine and horizontal cell visual features while still allowing individual cells to increase information transmission under the statistics of natural scenes. Established theories of efficient coding have shown that optimal information transmission under natural scenes allows a diverse set of receptive fields. Our results give a mechanism for this theory, showing how distinct neural pathways synthesize a sensory computation and how this architecture both generates computational diversity and achieves the objective of high information transmission.

How inhibitory neurons increase information transmission under threshold modulation.

Modulation of neuronal thresholds is ubiquitous in the brain. Phenomena such as figure-ground segmentation, motion detection, stimulus anticipation, and shifts in attention all involve changes in a neuron's threshold based on signals from larger scales than its primary inputs. However, this modulation reduces the accuracy with which neurons can represent their primary inputs, creating a mystery as to why threshold modulation is so widespread in the brain. We find that modulation is less detrimental than other forms of neuronal variability and that its negative effects can be nearly completely eliminated if modulation is applied selectively to sparsely responding neurons in a circuit by inhibitory neurons. We verify these predictions in the retina where we find that inhibitory amacrine cells selectively deliver modulation signals to sparsely responding ganglion cell types. Our findings elucidate the central role that inhibitory neurons play in maximizing information transmission under modulation.

Adaptation of Inhibition Mediates Retinal Sensitization.

In response to a changing sensory environment, sensory systems adjust their neural code for a number of purposes, including an enhanced sensitivity for novel stimuli, prediction of sensory features, and the maintenance of sensitivity. Retinal sensitization is a form of short-term plasticity that elevates local sensitivity following strong, local, visual stimulation and has been shown to create a prediction of the presence of a nearby localized object. The neural mechanism that generates this elevation in sensitivity remains unknown. Using simultaneous intracellular and multielectrode recording in the salamander retina, we show that a decrease in tonic amacrine transmission is necessary for and is correlated spatially and temporally with ganglion cell sensitization. Furthermore, introducing a decrease in amacrine transmission is sufficient to sensitize nearby ganglion cells. A computational model accounting for adaptive dynamics and nonlinear pathways confirms a decrease in steady inhibitory transmission can cause sensitization. Adaptation of inhibition enhances the sensitivity to the sensory feature conveyed by an inhibitory pathway, creating a prediction of future input.

Radiation Force as a Physical Mechanism for Ultrasonic Neurostimulation of the Ex Vivo Retina.

Focused ultrasound has been shown to be effective at stimulating neurons in many animal models, both in vivo and ex vivo Ultrasonic neuromodulation is the only noninvasive method of stimulation that could reach deep in the brain with high spatial-temporal resolution, and thus has potential for use in clinical applications and basic studies of the nervous system. Understanding the physical mechanism by which energy in a high acoustic frequency wave is delivered to stimulate neurons will be important to optimize this technology. We imaged the isolated salamander retina of either sex during ultrasonic stimuli that drive ganglion cell activity and observed micron scale displacements, consistent with radiation force, the nonlinear delivery of momentum by a propagating wave. We recorded ganglion cell spiking activity and changed the acoustic carrier frequency across a broad range (0.5-43 MHz), finding that increased stimulation occurs at higher acoustic frequencies, ruling out cavitation as an alternative possible mechanism. A quantitative radiation force model can explain retinal responses and could potentially explain previous in vivo results in the mouse, suggesting a new hypothesis to be tested in vivo Finally, we found that neural activity was strongly modulated by the distance between the transducer and the electrode array showing the influence of standing waves on the response. We conclude that radiation force is the dominant physical mechanism underlying ultrasonic neurostimulation in the ex vivo retina and propose that the control of standing waves is a new potential method to modulate these effects.SIGNIFICANCE STATEMENT Ultrasonic neurostimulation is a promising noninvasive technology that has potential for both basic research and clinical applications. The mechanisms of ultrasonic neurostimulation are unknown, making it difficult to optimize in any given application. We studied the physical mechanism by which ultrasound is converted into an effective energy form to cause neurostimulation in the retina and find that ultrasound acts via radiation force leading to a mechanical displacement of tissue. We further show that standing waves have a strong modulatory effect on activity. Our quantitative model by which ultrasound generates radiation force and leads to neural activity will be important in optimizing ultrasonic neurostimulation across a wide range of applications.

From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction.

Recently, deep feedforward neural networks have achieved considerable success in modeling biological sensory processing, in terms of reproducing the input-output map of sensory neurons. However, such models raise profound questions about the very nature of explanation in neuroscience. Are we simply replacing one complex system (a biological circuit) with another (a deep network), without understanding either? Moreover, beyond neural representations, are the deep network's computational mechanisms for generating neural responses the same as those in the brain? Without a systematic approach to extracting and understanding computational mechanisms from deep neural network models, it can be difficult both to assess the degree of utility of deep learning approaches in neuroscience, and to extract experimentally testable hypotheses from deep networks. We develop such a systematic approach by combining dimensionality reduction and modern attribution methods for determining the relative importance of interneurons for specific visual computations. We apply this approach to deep network models of the retina, revealing a conceptual understanding of how the retina acts as a predictive feature extractor that signals deviations from expectations for diverse spatiotemporal stimuli. For each stimulus, our extracted computational mechanisms are consistent with prior scientific literature, and in one case yields a new mechanistic hypothesis. Thus overall, this work not only yields insights into the computational mechanisms underlying the striking predictive capabilities of the retina, but also places the framework of deep networks as neuroscientific models on firmer theoretical foundations, by providing a new roadmap to go beyond comparing neural representations to extracting and understand computational mechanisms.

Adaptive feature detection from differential processing in parallel retinal pathways.

To transmit information efficiently in a changing environment, the retina adapts to visual contrast by adjusting its gain, latency and mean response. Additionally, the temporal frequency selectivity, or bandwidth changes to encode the absolute intensity when the stimulus environment is noisy, and intensity differences when noise is low. We show that the On pathway of On-Off retinal amacrine and ganglion cells is required to change temporal bandwidth but not other adaptive properties. This remarkably specific adaptive mechanism arises from differential effects of contrast on the On and Off pathways. We analyzed a biophysical model fit only to a cell's membrane potential, and verified pharmacologically that it accurately revealed the two pathways. We conclude that changes in bandwidth arise mostly from differences in synaptic threshold in the two pathways, rather than synaptic release dynamics as has previously been proposed to underlie contrast adaptation. Different efficient codes are selected by different thresholds in two independently adapting neural pathways.

Inferring hidden structure in multilayered neural circuits.

A central challenge in sensory neuroscience involves understanding how neural circuits shape computations across cascaded cell layers. Here we attempt to reconstruct the response properties of experimentally unobserved neurons in the interior of a multilayered neural circuit, using cascaded linear-nonlinear (LN-LN) models. We combine non-smooth regularization with proximal consensus algorithms to overcome difficulties in fitting such models that arise from the high dimensionality of their parameter space. We apply this framework to retinal ganglion cell processing, learning LN-LN models of retinal circuitry consisting of thousands of parameters, using 40 minutes of responses to white noise. Our models demonstrate a 53% improvement in predicting ganglion cell spikes over classical linear-nonlinear (LN) models. Internal nonlinear subunits of the model match properties of retinal bipolar cells in both receptive field structure and number. Subunits have consistently high thresholds, supressing all but a small fraction of inputs, leading to sparse activity patterns in which only one subunit drives ganglion cell spiking at any time. From the model's parameters, we predict that the removal of visual redundancies through stimulus decorrelation across space, a central tenet of efficient coding theory, originates primarily from bipolar cell synapses. Furthermore, the composite nonlinear computation performed by retinal circuitry corresponds to a boolean OR function applied to bipolar cell feature detectors. Our methods are statistically and computationally efficient, enabling us to rapidly learn hierarchical non-linear models as well as efficiently compute widely used descriptive statistics such as the spike triggered average (STA) and covariance (STC) for high dimensional stimuli. This general computational framework may aid in extracting principles of nonlinear hierarchical sensory processing across diverse modalities from limited data.

Ultrasound Elicits Behavioral Responses through Mechanical Effects on Neurons and Ion Channels in a Simple Nervous System.

Focused ultrasound has been shown to stimulate excitable cells, but the biophysical mechanisms behind this phenomenon remain poorly understood. To provide additional insight, we devised a behavioral-genetic assay applied to the well-characterized nervous system of Caenorhabditis elegans nematodes. We found that pulsed ultrasound elicits robust reversal behavior in wild-type animals in a pressure-, duration-, and pulse protocol-dependent manner. Responses were preserved in mutants unable to sense thermal fluctuations and absent in mutants lacking neurons required for mechanosensation. Additionally, we found that the worm's response to ultrasound pulses rests on the expression of MEC-4, a DEG/ENaC/ASIC ion channel required for touch sensation. Consistent with prior studies of MEC-4-dependent currents in vivo, the worm's response was optimal for pulses repeated 300-1000 times per second. Based on these findings, we conclude that mechanical, rather than thermal, stimulation accounts for behavioral responses. Further, we propose that acoustic radiation force governs the response to ultrasound in a manner that depends on the touch receptor neurons and MEC-4-dependent ion channels. Our findings illuminate a complete pathway of ultrasound action, from the forces generated by propagating ultrasound to an activation of a specific ion channel. The findings further highlight the importance of optimizing ultrasound pulsing protocols when stimulating neurons via ion channels with mechanosensitive properties.SIGNIFICANCE STATEMENT How ultrasound influences neurons and other excitable cells has remained a mystery for decades. Although it is widely understood that ultrasound can heat tissues and induce mechanical strain, whether or not neuronal activation depends on heat, mechanical force, or both physical factors is not known. We harnessed Caenorhabditis elegans nematodes and their extraordinary sensitivity to thermal and mechanical stimuli to address this question. Whereas thermosensory mutants respond to ultrasound similar to wild-type animals, mechanosensory mutants were insensitive to ultrasound stimulation. Additionally, stimulus parameters that accentuate mechanical effects were more effective than those producing more heat. These findings highlight a mechanical nature of the effect of ultrasound on neurons and suggest specific ways to optimize stimulation protocols in specific tissues.

Optimal Information Transmission by Overlapping Retinal Cell Mosaics.

The retina provides an excellent system for understanding the trade-offs that influence distributed information processing across multiple neuron types. We focus here on the problem faced by the visual system of allocating a limited number neurons to encode different visual features at different spatial locations. The retina needs to solve three competing goals: 1) encode different visual features, 2) maximize spatial resolution for each feature, and 3) maximize accuracy with which each feature is encoded at each location. There is no current understanding of how these goals are optimized together. While information theory provides a platform for theoretically solving these problems, evaluating information provided by the responses of large neuronal arrays is in general challenging. Here we present a solution to this problem in the case where multi-dimensional stimuli can be decomposed into approximately independent components that are subsequently coupled by neural responses. Using this approach we quantify information transmission by multiple overlapping retinal ganglion cell mosaics. In the retina, translation invariance of input signals makes it possible to use Fourier basis as a set of independent components. The results reveal a transition where one high-density mosaic becomes less informative than two or more overlapping lower-density mosaics. The results explain differences in the fractions of multiple cell types, predict the existence of new retinal ganglion cell subtypes, relative distribution of neurons among cell types and differences in their nonlinear and dynamical response properties.

Deep Learning Models of the Retinal Response to Natural Scenes.

A central challenge in sensory neuroscience is to understand neural computations and circuit mechanisms that underlie the encoding of ethologically relevant, natural stimuli. In multilayered neural circuits, nonlinear processes such as synaptic transmission and spiking dynamics present a significant obstacle to the creation of accurate computational models of responses to natural stimuli. Here we demonstrate that deep convolutional neural networks (CNNs) capture retinal responses to natural scenes nearly to within the variability of a cell's response, and are markedly more accurate than linear-nonlinear (LN) models and Generalized Linear Models (GLMs). Moreover, we find two additional surprising properties of CNNs: they are less susceptible to overfitting than their LN counterparts when trained on small amounts of data, and generalize better when tested on stimuli drawn from a different distribution (e.g. between natural scenes and white noise). An examination of the learned CNNs reveals several properties. First, a richer set of feature maps is necessary for predicting the responses to natural scenes compared to white noise. Second, temporally precise responses to slowly varying inputs originate from feedforward inhibition, similar to known retinal mechanisms. Third, the injection of latent noise sources in intermediate layers enables our model to capture the sub-Poisson spiking variability observed in retinal ganglion cells. Fourth, augmenting our CNNs with recurrent lateral connections enables them to capture contrast adaptation as an emergent property of accurately describing retinal responses to natural scenes. These methods can be readily generalized to other sensory modalities and stimulus ensembles. Overall, this work demonstrates that CNNs not only accurately capture sensory circuit responses to natural scenes, but also can yield information about the circuit's internal structure and function.

Synchronized amplification of local information transmission by peripheral retinal input.

Sensory stimuli have varying statistics influenced by both the environment and by active sensing behaviors that rapidly and globally change the sensory input. Consequently, sensory systems often adjust their neural code to the expected statistics of their sensory input to transmit novel sensory information. Here, we show that sudden peripheral motion amplifies and accelerates information transmission in salamander ganglion cells in a 50 ms time window. Underlying this gating of information is a transient increase in adaptation to contrast, enhancing sensitivity to a broader range of stimuli. Using a model and natural images, we show that this effect coincides with an expected increase in information in bipolar cells after a global image shift. Our findings reveal the dynamic allocation of energy resources to increase neural activity at times of expected high information content, a principle of adaptation that balances the competing requirements of conserving spikes and transmitting information.

Critical and maximally informative encoding between neural populations in the retina.

Computation in the brain involves multiple types of neurons, yet the organizing principles for how these neurons work together remain unclear. Information theory has offered explanations for how different types of neurons can maximize the transmitted information by encoding different stimulus features. However, recent experiments indicate that separate neuronal types exist that encode the same filtered version of the stimulus, but then the different cell types signal the presence of that stimulus feature with different thresholds. Here we show that the emergence of these neuronal types can be quantitatively described by the theory of transitions between different phases of matter. The two key parameters that control the separation of neurons into subclasses are the mean and standard deviation (SD) of noise affecting neural responses. The average noise across the neural population plays the role of temperature in the classic theory of phase transitions, whereas the SD is equivalent to pressure or magnetic field, in the case of liquid-gas and magnetic transitions, respectively. Our results account for properties of two recently discovered types of salamander Off retinal ganglion cells, as well as the absence of multiple types of On cells. We further show that, across visual stimulus contrasts, retinal circuits continued to operate near the critical point whose quantitative characteristics matched those expected near a liquid-gas critical point and described by the nearest-neighbor Ising model in three dimensions. By operating near a critical point, neural circuits can maximize information transmission in a given environment while retaining the ability to quickly adapt to a new environment.

Building blocks of temporal filters in retinal synapses.

Sensory systems must be able to extract features of a stimulus to detect and represent properties of the world. Because sensory signals are constantly changing, a critical aspect of this transformation relates to the timing of signals and the ability to filter those signals to select dynamic properties, such as visual motion. At first assessment, one might think that the primary biophysical properties that construct a temporal filter would be dynamic mechanisms such as molecular concentration or membrane electrical properties. However, in the current issue of PLOS Biology, Baden et al. identify a mechanism of temporal filtering in the zebrafish and goldfish retina that is not dynamic but is in fact a structural building block-the physical size of a synapse itself. The authors observe that small, bipolar cell synaptic terminals are fast and highly adaptive, whereas large ones are slower and adapt less. Using a computational model, they conclude that the volume of the synaptic terminal influences the calcium concentration and the number of available vesicles. These results indicate that the size of the presynaptic terminal is an independent control for the dynamics of a synapse and may reveal aspects of synaptic function that can be inferred from anatomical structure.

Insights from the retina into the diverse and general computations of adaptation, detection, and prediction.

The retina performs a diverse set of complex, nonlinear, computations, beyond the simple linear photoreceptor weighting assumed in the classical understanding of ganglion cell receptive fields. Here we attempt to organize these computations and extract rules that correspond to three distinct goals of early sensory systems. First, the retina acts efficiently to transmit information to the higher brain for further processing. We observe that although the retina adapts to a number of complex statistics, many of these may be explained by local adaptation to the mean signal strength at that stage. Second, ganglion cells signal the detection of a diverse set of features. Recent results indicate that feature selectivity arises through the action of specific inhibition, rather than through the convergence of excitation as in classical cortical models. Finally, the retina conveys predictions about the stimulus, a function usually attributed only to the higher brain. We expect that computational and mechanistic rules associated with these classes of functions will be an important guide in the study of other neural circuits.

Spatial segregation of adaptation and predictive sensitization in retinal ganglion cells.

Sensory systems change their sensitivity based on recent stimuli to adjust their response range to the range of inputs and to predict future sensory input. Here, we report the presence of retinal ganglion cells that have antagonistic plasticity, showing central adaptation and peripheral sensitization. Ganglion cell responses were captured by a spatiotemporal model with independently adapting excitatory and inhibitory subunits, and sensitization requires GABAergic inhibition. Using a simple theory of signal detection, we show that the sensitizing surround conforms to an optimal inference model that continually updates the prior signal probability. This indicates that small receptive field regions have dual functionality--to adapt to the local range of signals but sensitize based upon the probability of the presence of that signal. Within this framework, we show that sensitization predicts the location of a nearby object, revealing prediction as a functional role for adapting inhibition in the nervous system.

Transformation of visual signals by inhibitory interneurons in retinal circuits.

One of the largest mysteries of the brain lies in understanding how higher-level computations are implemented by lower-level operations in neurons and synapses. In particular, in many brain regions inhibitory interneurons represent a diverse class of cells, the individual functional roles of which are unknown. We discuss here how the operations of inhibitory interneurons influence the behavior of a circuit, focusing on recent results in the vertebrate retina. A key role in this understanding is played by a common representation of the visual stimulus that can be applied at different stages. By considering how this stimulus representation changes at each location in the circuit, we can understand how neuron-level operations such as thresholds and inhibition yield circuit-level computations such as how stimulus selectivity and gain are controlled by local and peripheral visual stimuli.