Semantic encoding during language comprehension at single-cell resolution.

From sequences of speech sounds1,2 or letters3, humans can extract rich and nuanced meaning through language. This capacity is essential for human communication. Yet, despite a growing understanding of the brain areas that support linguistic and semantic processing4-12, the derivation of linguistic meaning in neural tissue at the cellular level and over the timescale of action potentials remains largely unknown. Here we recorded from single cells in the left language-dominant prefrontal cortex as participants listened to semantically diverse sentences and naturalistic stories. By tracking their activities during natural speech processing, we discover a fine-scale cortical representation of semantic information by individual neurons. These neurons responded selectively to specific word meanings and reliably distinguished words from nonwords. Moreover, rather than responding to the words as fixed memory representations, their activities were highly dynamic, reflecting the words' meanings based on their specific sentence contexts and independent of their phonetic form. Collectively, we show how these cell ensembles accurately predicted the broad semantic categories of the words as they were heard in real time during speech and how they tracked the sentences in which they appeared. We also show how they encoded the hierarchical structure of these meaning representations and how these representations mapped onto the cell population. Together, these findings reveal a finely detailed cortical organization of semantic representations at the neuron scale in humans and begin to illuminate the cellular-level processing of meaning during language comprehension.

Single-neuronal elements of speech production in humans.

Humans are capable of generating extraordinarily diverse articulatory movement combinations to produce meaningful speech. This ability to orchestrate specific phonetic sequences, and their syllabification and inflection over subsecond timescales allows us to produce thousands of word sounds and is a core component of language1,2. The fundamental cellular units and constructs by which we plan and produce words during speech, however, remain largely unknown. Here, using acute ultrahigh-density Neuropixels recordings capable of sampling across the cortical column in humans, we discover neurons in the language-dominant prefrontal cortex that encoded detailed information about the phonetic arrangement and composition of planned words during the production of natural speech. These neurons represented the specific order and structure of articulatory events before utterance and reflected the segmentation of phonetic sequences into distinct syllables. They also accurately predicted the phonetic, syllabic and morphological components of upcoming words and showed a temporally ordered dynamic. Collectively, we show how these mixtures of cells are broadly organized along the cortical column and how their activity patterns transition from articulation planning to production. We also demonstrate how these cells reliably track the detailed composition of consonant and vowel sounds during perception and how they distinguish processes specifically related to speaking from those related to listening. Together, these findings reveal a remarkably structured organization and encoding cascade of phonetic representations by prefrontal neurons in humans and demonstrate a cellular process that can support the production of speech.

DREDge: robust motion correction for high-density extracellular recordings across species.

High-density microelectrode arrays (MEAs) have opened new possibilities for systems neuroscience in human and non-human animals, but brain tissue motion relative to the array poses a challenge for downstream analyses, particularly in human recordings. We introduce DREDge (Decentralized Registration of Electrophysiology Data), a robust algorithm which is well suited for the registration of noisy, nonstationary extracellular electrophysiology recordings. In addition to estimating motion from spikes in the action potential (AP) frequency band, DREDge enables automated tracking of motion at high temporal resolution in the local field potential (LFP) frequency band. In human intraoperative recordings, which often feature fast (period <1s) motion, DREDge correction in the LFP band enabled reliable recovery of evoked potentials, and significantly reduced single-unit spike shape variability and spike sorting error. Applying DREDge to recordings made during deep probe insertions in nonhuman primates demonstrated the possibility of tracking probe motion of centimeters across several brain regions while simultaneously mapping single unit electrophysiological features. DREDge reliably delivered improved motion correction in acute mouse recordings, especially in those made with an recent ultra-high density probe. We also implemented a procedure for applying DREDge to recordings made across tens of days in chronic implantations in mice, reliably yielding stable motion tracking despite changes in neural activity across experimental sessions. Together, these advances enable automated, scalable registration of electrophysiological data across multiple species, probe types, and drift cases, providing a stable foundation for downstream scientific analyses of these rich datasets.

Sector area index: a novel supporting marker for blepharoptosis screening and grading.

PURPOSE: To introduce a new supporting marker for discriminating different grades of ptosis called Sector Area Index (SAI) and a semi-automated technique to calculate it. METHODS: In this cross-sectional comparative case series, a circle enclosing the intercanthal distance was automatically drawn after choosing two points as the medial and lateral canthus and manually selecting the palpebral fissure region. Finally, 15-degree apart sectors are applied to the enclosed circle. SAI was measured automatically by dividing the area of each 15-degree sector marked with the upper eyelid contour by the total area of the sector marked with the edge of the surrounding circle. SAI values and inter-eye SAI differences were compared between patients with different grades of ptosis as well as normal patients. RESULTS: In the current study, 106 eyes were recruited (30, 25, 27, and 24 in the control, mild, moderate, and severe ptosis groups, respectively). Mean values of SAI in all sectors showed a decreasing trend from normal individuals toward patients with severe ptosis. The mean difference values of SAI between study eyes and fellow eyes in all four groups of patients showed a statistically significant difference (p < 0.05). In a pairwise comparison between groups, mean values of SAI in all nasal sectors from 15° to 60° showed a statistically significant difference between all groups (p < 0.05). CONCLUSION: The mean difference of SAI between study eyes and fellow eyes, including eyelid curvature, especially in 15°-60° and 120°-165° sectors, can demonstrate differentiating performance for detecting and discriminating varying grades of ptosis.

Modified Neuropixels probes for recording human neurophysiology in the operating room.

Neuropixels are silicon-based electrophysiology-recording probes with high channel count and recording-site density. These probes offer a turnkey platform for measuring neural activity with single-cell resolution and at a scale that is beyond the capabilities of current clinically approved devices. Our team demonstrated the first-in-human use of these probes during resection surgery for epilepsy or tumors and deep brain stimulation electrode placement in patients with Parkinson's disease. Here, we provide a better understanding of the capabilities and challenges of using Neuropixels as a research tool to study human neurophysiology, with the hope that this information may inform future efforts toward regulatory approval of Neuropixels probes as research devices. In perioperative procedures, the major concerns are the initial sterility of the device, maintaining a sterile field during surgery, having multiple referencing and grounding schemes available to de-noise recordings (if necessary), protecting the silicon probe from accidental contact before insertion and obtaining high-quality action potential and local field potential recordings. The research team ensures that the device is fully operational while coordinating with the surgical team to remove sources of electrical noise that could otherwise substantially affect the signals recorded by the sensitive hardware. Prior preparation using the equipment and training in human clinical research and working in operating rooms maximize effective communication within and between the teams, ensuring high recording quality and minimizing the time added to the surgery. The perioperative procedure requires ~4 h, and the entire protocol requires multiple weeks.

ROBUST ONLINE MULTIBAND DRIFT ESTIMATION IN ELECTROPHYSIOLOGY DATA.

High-density electrophysiology probes have opened new possibilities for systems neuroscience in human and non-human animals, but probe motion poses a challenge for downstream analyses, particularly in human recordings. We improve on the state of the art for tracking this motion with four major contributions. First, we extend previous decentralized methods to use multiband information, leveraging the local field potential (LFP) in addition to spikes. Second, we show that the LFP-based approach enables registration at sub-second temporal resolution. Third, we introduce an efficient online motion tracking algorithm, enabling the method to scale up to longer and higher-resolution recordings, and possibly facilitating real-time applications. Finally, we improve the robustness of the approach by introducing a structure-aware objective and simple methods for adaptive parameter selection. Together, these advances enable fully automated scalable registration of challenging datasets from human and mouse.

Reduced sociability and social agency encoding in adult Shank3-mutant mice are restored through gene re-expression in real time.

Despite a growing understanding of the molecular and developmental basis of autism spectrum disorder (ASD), how the neuronal encoding of social information is disrupted in ASD and whether it contributes to abnormal social behavior remains unclear. Here, we disrupted and then restored expression of the ASD-associated gene Shank3 in adult male mice while tracking the encoding dynamics of neurons in the medial prefrontal cortex (mPFC) over weeks. We find that Shank3 disruption led to a reduction of neurons encoding the experience of other mice and an increase in neurons encoding the animal's own experience. This shift was associated with a loss of ability by neurons to distinguish other from self and, therefore, the inability to encode social agency. Restoration of Shank3 expression in the mPFC reversed this encoding imbalance and increased sociability over 5-8 weeks. These findings reveal a neuronal-encoding process that is necessary for social behavior and that may be disrupted in ASD.

Single-neuronal predictions of others' beliefs in humans.

Human social behaviour crucially depends on our ability to reason about others. This capacity for theory of mind has a vital role in social cognition because it enables us not only to form a detailed understanding of the hidden thoughts and beliefs of other individuals but also to understand that they may differ from our own1-3. Although a number of areas in the human brain have been linked to social reasoning4,5 and its disruption across a variety of psychosocial disorders6-8, the basic cellular mechanisms that underlie human theory of mind remain undefined. Here, using recordings from single cells in the human dorsomedial prefrontal cortex, we identify neurons that reliably encode information about others' beliefs across richly varying scenarios and that distinguish self- from other-belief-related representations. By further following their encoding dynamics, we show how these cells represent the contents of the others' beliefs and accurately predict whether they are true or false. We also show how they track inferred beliefs from another's specific perspective and how their activities relate to behavioural performance. Together, these findings reveal a detailed cellular process in the human dorsomedial prefrontal cortex for representing another's beliefs and identify candidate neurons that could support theory of mind.

Cerebellar Prediction of the Dynamic Sensory Consequences of Gravity.

As we go about our everyday activities, our brain computes accurate estimates of both our motion relative to the world and our orientation relative to gravity. However, how the brain then accounts for gravity as we actively move and interact with our environment is not yet known. Here, we provide evidence that, although during passive movements, individual cerebellar output neurons encode representations of head motion and orientation relative to gravity, these gravity-driven responses are cancelled when head movement is a consequence of voluntary generated movement. In contrast, the gravity-driven responses of primary otolith and semicircular canal afferents remain intact during both active and passive self-motion, indicating the attenuated responses of central neurons are not inherited from afferent inputs. Taken together, our results are consistent with the view that the cerebellum builds a dynamic prediction (e.g., internal model) of the sensory consequences of gravity during active self-motion, which in turn enables the preferential encoding of unexpected motion to ensure postural and perceptual stability.

Coding strategies in the otolith system differ for translational head motion vs. static orientation relative to gravity.

The detection of gravito-inertial forces by the otolith system is essential for our sense of balance and accurate perception. To date, however, how this system encodes the self-motion stimuli that are experienced during everyday activities remains unknown. Here, we addressed this fundamental question directly by recording from single otolith afferents in monkeys during naturalistic translational self-motion and changes in static head orientation. Otolith afferents with higher intrinsic variability transmitted more information overall about translational self-motion than their regular counterparts, owing to stronger nonlinearities that enabled precise spike timing including phase locking. By contrast, more regular afferents better discriminated between different static head orientations relative to gravity. Using computational methods, we further demonstrated that coupled increases in intrinsic variability and sensitivity accounted for the observed functional differences between afferent classes. Together, our results indicate that irregular and regular otolith afferents use different strategies to encode naturalistic self-motion and static head orientation relative to gravity.

Dorsolateral prefrontal neurons mediate subjective decisions and their variation in humans.

Subjective decisions play a vital role in human behavior because, while often grounded in fact, they are inherently based on personal beliefs that can vary broadly within and between individuals. While these properties set subjective decisions apart from many other sensorimotor processes and are of wide sociological impact, their single-neuronal basis in humans is unknown. Here we find cells in the dorsolateral prefrontal cortex (dlPFC) that reflect variations in the subjective decisions of humans when performing opinion-based tasks. These neurons changed their activities gradually as the participants transitioned between choice options but also reflected their unique point of conversion at equipoise. Focal disruption of the dlPFC, by contrast, diminished gradation between opposing decisions but had little effect on sensory perceptual choices or their motor report. These findings suggest that the human dlPFC plays an important role in subjective decisions and propose a mechanism for mediating their variation during opinion formation.

Decoding unconstrained arm movements in primates using high-density electrocorticography signals for brain-machine interface use.

Motor deficit is among the most debilitating aspects of injury to the central nervous system. Despite ongoing progress in brain-machine interface (BMI) development and in the functional electrical stimulation of muscles and nerves, little is understood about how neural signals in the brain may be used to potentially control movement in one's own unconstrained paralyzed limb. We recorded from high-density electrocorticography (ECoG) electrode arrays in the ventral premotor cortex (PMv) of a rhesus macaque and used real-time motion tracking techniques to correlate spatial-temporal changes in neural activity with arm movements made towards objects in three-dimensional space at millisecond precision. We found that neural activity from a small number of electrodes within the PMv can be used to accurately predict reach-return movement onset and directionality. Also, whereas higher gamma frequency field activity was more predictive about movement direction during performance, mid-band (beta and low gamma) activity was more predictive of movement prior to onset. We speculate these dual spatiotemporal signals may be used to optimize both planning and execution of movement during natural reaching, with prospective relevance to the future development of neural prosthetics aimed at restoring motor control over one's own paralyzed limb.

Envelope statistics of self-motion signals experienced by human subjects during everyday activities: Implications for vestibular processing.

There is accumulating evidence that the brain's neural coding strategies are constrained by natural stimulus statistics. Here we investigated the statistics of the time varying envelope (i.e. a second-order stimulus attribute that is related to variance) of rotational and translational self-motion signals experienced by human subjects during everyday activities. We found that envelopes can reach large values across all six motion dimensions (~450 deg/s for rotations and ~4 G for translations). Unlike results obtained in other sensory modalities, the spectral power of envelope signals decreased slowly for low (< 2 Hz) and more sharply for high (>2 Hz) temporal frequencies and thus was not well-fit by a power law. We next compared the spectral properties of envelope signals resulting from active and passive self-motion, as well as those resulting from signals obtained when the subject is absent (i.e. external stimuli). Our data suggest that different mechanisms underlie deviation from scale invariance in rotational and translational self-motion envelopes. Specifically, active self-motion and filtering by the human body cause deviation from scale invariance primarily for translational and rotational envelope signals, respectively. Finally, we used well-established models in order to predict the responses of peripheral vestibular afferents to natural envelope stimuli. We found that irregular afferents responded more strongly to envelopes than their regular counterparts. Our findings have important consequences for understanding the coding strategies used by the vestibular system to process natural second-order self-motion signals.

The statistics of the vestibular input experienced during natural self-motion differ between rodents and primates.

KEY POINTS: In order to understand how the brain's coding strategies are adapted to the statistics of the sensory stimuli experienced during everyday life, the use of animal models is essential. Mice and non-human primates have become common models for furthering our knowledge of the neuronal coding of natural stimuli, but differences in their natural environments and behavioural repertoire may impact optimal coding strategies. Here we investigated the structure and statistics of the vestibular input experienced by mice versus non-human primates during natural behaviours, and found important differences. Our data establish that the structure and statistics of natural signals in non-human primates more closely resemble those observed previously in humans, suggesting similar coding strategies for incoming vestibular input. These results help us understand how the effects of active sensing and biomechanics will differentially shape the statistics of vestibular stimuli across species, and have important implications for sensory coding in other systems. ABSTRACT: It is widely believed that sensory systems are adapted to the statistical structure of natural stimuli, thereby optimizing coding. Recent evidence suggests that this is also the case for the vestibular system, which senses self-motion and in turn contributes to essential brain functions ranging from the most automatic reflexes to spatial perception and motor coordination. However, little is known about the statistics of self-motion stimuli actually experienced by freely moving animals in their natural environments. Accordingly, here we examined the natural self-motion signals experienced by mice and monkeys: two species commonly used to study vestibular neural coding. First, we found that probability distributions for all six dimensions of motion (three rotations, three translations) in both species deviated from normality due to long tails. Interestingly, the power spectra of natural rotational stimuli displayed similar structure for both species and were not well fitted by power laws. This result contrasts with reports that the natural spectra of other sensory modalities (i.e. vision, auditory and tactile) instead show a power-law relationship with frequency, which indicates scale invariance. Analysis of natural translational stimuli revealed important species differences as power spectra deviated from scale invariance for monkeys but not for mice. By comparing our results to previously published data for humans, we found the statistical structure of natural self-motion stimuli in monkeys and humans more closely resemble one another. Our results thus predict that, overall, neural coding strategies used by vestibular pathways to encode natural self-motion stimuli are fundamentally different in rodents and primates.

Self-motion evokes precise spike timing in the primate vestibular system.

The accurate representation of self-motion requires the efficient processing of sensory input by the vestibular system. Conventional wisdom is that vestibular information is exclusively transmitted through changes in firing rate, yet under this assumption vestibular neurons display relatively poor detection and information transmission. Here, we carry out an analysis of the system's coding capabilities by recording neuronal responses to repeated presentations of naturalistic stimuli. We find that afferents with greater intrinsic variability reliably discriminate between different stimulus waveforms through differential patterns of precise (∼6 ms) spike timing, while those with minimal intrinsic variability do not. A simple mathematical model provides an explanation for this result. Postsynaptic central neurons also demonstrate precise spike timing, suggesting that higher brain areas also represent self-motion using temporally precise firing. These findings demonstrate that two distinct sensory channels represent vestibular information: one using rate coding and the other that takes advantage of precise spike timing.

Rapid adaptation of multisensory integration in vestibular pathways.

Sensing gravity is vital for our perception of spatial orientation, the control of upright posture, and generation of our everyday activities. When an astronaut transitions to microgravity or returns to earth, the vestibular input arising from self-motion will not match the brain's expectation. Our recent neurophysiological studies have provided insight into how the nervous system rapidly reorganizes when vestibular input becomes unreliable by both (1) updating its internal model of the sensory consequences of motion and (2) up-weighting more reliable extra-vestibular information. These neural strategies, in turn, are linked to improvements in sensorimotor performance (e.g., gaze and postural stability, locomotion, orienting) and perception characterized by similar time courses. We suggest that furthering our understanding of the neural mechanisms that underlie sensorimotor adaptation will have important implications for optimizing training programs for astronauts before and after space exploration missions and for the design of goal-oriented rehabilitation for patients.

Coding of envelopes by correlated but not single-neuron activity requires neural variability.

Understanding how the brain processes sensory information is often complicated by the fact that neurons exhibit trial-to-trial variability in their responses to stimuli. Indeed, the role of variability in sensory coding is still highly debated. Here, we examined how variability influences neural responses to naturalistic stimuli consisting of a fast time-varying waveform (i.e., carrier or first order) whose amplitude (i.e., envelope or second order) varies more slowly. Recordings were made from fish electrosensory and monkey vestibular sensory neurons. In both systems, we show that correlated but not single-neuron activity can provide detailed information about second-order stimulus features. Using a simple mathematical model, we made the strong prediction that such correlation-based coding of envelopes requires neural variability. Strikingly, the performance of correlated activity at predicting the envelope was similarly optimally tuned to a nonzero level of variability in both systems, thereby confirming this prediction. Finally, we show that second-order sensory information can only be decoded if one takes into account joint statistics when combining neural activities. Our results thus show that correlated but not single-neural activity can transmit information about the envelope, that such transmission requires neural variability, and that this information can be decoded. We suggest that envelope coding by correlated activity is a general feature of sensory processing that will be found across species and systems.

The increased sensitivity of irregular peripheral canal and otolith vestibular afferents optimizes their encoding of natural stimuli.

Efficient processing of incoming sensory input is essential for an organism's survival. A growing body of evidence suggests that sensory systems have developed coding strategies that are constrained by the statistics of the natural environment. Consequently, it is necessary to first characterize neural responses to natural stimuli to uncover the coding strategies used by a given sensory system. Here we report for the first time the statistics of vestibular rotational and translational stimuli experienced by rhesus monkeys during natural (e.g., walking, grooming) behaviors. We find that these stimuli can reach intensities as high as 1500 deg/s and 8 G. Recordings from afferents during naturalistic rotational and linear motion further revealed strongly nonlinear responses in the form of rectification and saturation, which could not be accurately predicted by traditional linear models of vestibular processing. Accordingly, we used linear-nonlinear cascade models and found that these could accurately predict responses to naturalistic stimuli. Finally, we tested whether the statistics of natural vestibular signals constrain the neural coding strategies used by peripheral afferents. We found that both irregular otolith and semicircular canal afferents, because of their higher sensitivities, were more optimized for processing natural vestibular stimuli as compared with their regular counterparts. Our results therefore provide the first evidence supporting the hypothesis that the neural coding strategies used by the vestibular system are matched to the statistics of natural stimuli.

Integration of canal and otolith inputs by central vestibular neurons is subadditive for both active and passive self-motion: implication for perception.

Traditionally, the neural encoding of vestibular information is studied by applying either passive rotations or translations in isolation. However, natural vestibular stimuli are typically more complex. During everyday life, our self-motion is generally not restricted to one dimension, but rather comprises both rotational and translational motion that will simultaneously stimulate receptors in the semicircular canals and otoliths. In addition, natural self-motion is the result of self-generated and externally generated movements. However, to date, it remains unknown how information about rotational and translational components of self-motion is integrated by vestibular pathways during active and/or passive motion. Accordingly, here, we compared the responses of neurons at the first central stage of vestibular processing to rotation, translation, and combined motion. Recordings were made in alert macaques from neurons in the vestibular nuclei involved in postural control and self-motion perception. In response to passive stimulation, neurons did not combine canal and otolith afferent information linearly. Instead, inputs were subadditively integrated with a weighting that was frequency dependent. Although canal inputs were more heavily weighted at low frequencies, the weighting of otolith input increased with frequency. In response to active stimulation, neuronal modulation was significantly attenuated (∼ 70%) relative to passive stimulation for rotations and translations and even more profoundly attenuated for combined motion due to subadditive input integration. Together, these findings provide insights into neural computations underlying the integration of semicircular canal and otolith inputs required for accurate posture and motor control, as well as perceptual stability, during everyday life.