Modeling responses of macaque and human retinal ganglion cells to natural images using a convolutional neural network.

Linear-nonlinear (LN) cascade models provide a simple way to capture retinal ganglion cell (RGC) responses to artificial stimuli such as white noise, but their ability to model responses to natural images is limited. Recently, convolutional neural network (CNN) models have been shown to produce light response predictions that were substantially more accurate than those of a LN model. However, this modeling approach has not yet been applied to responses of macaque or human RGCs to natural images. Here, we train and test a CNN model on responses to natural images of the four numerically dominant RGC types in the macaque and human retina - ON parasol, OFF parasol, ON midget and OFF midget cells. Compared with the LN model, the CNN model provided substantially more accurate response predictions. Linear reconstructions of the visual stimulus were more accurate for CNN compared to LN model-generated responses, relative to reconstructions obtained from the recorded data. These findings demonstrate the effectiveness of a CNN model in capturing light responses of major RGC types in the macaque and human retinas in natural conditions.

Decomposition of retinal ganglion cell electrical images for cell type and functional inference.

Identifying neuronal cell types and their biophysical properties based on their extracellular electrical features is a major challenge for experimental neuroscience and the development of high-resolution brain-machine interfaces. One example is identification of retinal ganglion cell (RGC) types and their visual response properties, which is fundamental for developing future electronic implants that can restore vision. The electrical image (EI) of a RGC, or the mean spatio-temporal voltage footprint of its recorded spikes on a high-density electrode array, contains substantial information about its anatomical, morphological, and functional properties. However, the analysis of these properties is complex because of the high-dimensional nature of the EI. We present a novel optimization-based algorithm to decompose electrical image into a low-dimensional, biophysically-based representation: the temporally-shifted superposition of three learned basis waveforms corresponding to spike waveforms produced in the somatic, dendritic and axonal cellular compartments. Large-scale multi-electrode recordings from the macaque retina were used to test the effectiveness of the decomposition. The decomposition accurately localized the somatic and dendritic compartments of the cell. The imputed dendritic fields of RGCs correctly predicted the location and shape of their visual receptive fields. The inferred waveform amplitudes and shapes accurately identified the four major primate RGC types (ON and OFF midget and parasol cells), a substantial advance. Together, these findings may contribute to more accurate inference of RGC types and their original light responses in the degenerated retina, with possible implications for other electrical imaging applications.

Fixational Eye Movements Enhance the Precision of Visual Information Transmitted by the Primate Retina.

Fixational eye movements alter the number and timing of spikes transmitted from the retina to the brain, but whether these changes enhance or degrade the visual signal is unclear. To quantify this, we developed a Bayesian method for reconstructing natural images from the recorded spikes of hundreds of macaque retinal ganglion cells (RGCs) of the major cell types, combining a likelihood model for RGC light responses with the natural image prior implicitly embedded in an artificial neural network optimized for denoising. The method matched or surpassed the performance of previous reconstruction algorithms, and provided an interpretable framework for characterizing the retinal signal. Reconstructions were improved with artificial stimulus jitter that emulated fixational eye movements, even when the jitter trajectory was inferred from retinal spikes. Reconstructions were degraded by small artificial perturbations of spike times, revealing more precise temporal encoding than suggested by previous studies. Finally, reconstructions were substantially degraded when derived from a model that ignored cell-to-cell interactions, indicating the importance of stimulus-evoked correlations. Thus, fixational eye movements enhance the precision of the retinal representation.

Individual variability of neural computations in the primate retina.

Variation in the neural code contributes to making each individual unique. We probed neural code variation using ∼100 population recordings from major ganglion cell types in the macaque retina, combined with an interpretable computational representation of individual variability. This representation captured variation and covariation in properties such as nonlinearity, temporal dynamics, and spatial receptive field size and preserved invariances such as asymmetries between On and Off cells. The covariation of response properties in different cell types was associated with the proximity of lamination of their synaptic input. Surprisingly, male retinas exhibited higher firing rates and faster temporal integration than female retinas. Exploiting data from previously recorded retinas enabled efficient characterization of a new macaque retina, and of a human retina. Simulations indicated that combining a large dataset of retinal recordings with behavioral feedback could reveal the neural code in a living human and thus improve vision restoration with retinal implants.

Reconstruction of natural images from responses of primate retinal ganglion cells.

The visual message conveyed by a retinal ganglion cell (RGC) is often summarized by its spatial receptive field, but in principle also depends on the responses of other RGCs and natural image statistics. This possibility was explored by linear reconstruction of natural images from responses of the four numerically-dominant macaque RGC types. Reconstructions were highly consistent across retinas. The optimal reconstruction filter for each RGC - its visual message - reflected natural image statistics, and resembled the receptive field only when nearby, same-type cells were included. ON and OFF cells conveyed largely independent, complementary representations, and parasol and midget cells conveyed distinct features. Correlated activity and nonlinearities had statistically significant but minor effects on reconstruction. Simulated reconstructions, using linear-nonlinear cascade models of RGC light responses that incorporated measured spatial properties and nonlinearities, produced similar results. Spatiotemporal reconstructions exhibited similar spatial properties, suggesting that the results are relevant for natural vision.

Vision begins in the retina, the layer of tissue that lines the back of the eye. Light-sensitive cells called rods and cones absorb incoming light and convert it into electrical signals. They pass these signals to neurons called retinal ganglion cells (RGCs), which convert them into electrical signals called spikes. Spikes from RGCs then travel along the optic nerve to the brain. They are the only source of visual information that the brain receives. From this information, the brain constructs our entire visual world. The primate retina contains roughly 20 types of RGCs. Each encodes a different visual feature, such as the presence of bright spots of a certain size, or information about texture and movement. But exactly what input each RGC sends to the brain, and how the brain uses this information, is unclear. Brackbill et al. set out to answer these questions by measuring and analyzing the electrical activity in isolated retinas from macaque monkeys. Studying the macaque retina was important because the primate visual system differs from that of other species in several ways. These include the numbers and types of RGCs present in the retina. These primates are also similar to humans in their high-resolution central vision and trichromatic color vision. Using electrode arrays to monitor hundreds of RGCs at the same time, Brackbill et al. recorded the responses of macaque retinas to real-life images of landscapes, objects, animals or people. Based on these recordings, plus existing knowledge about RGC responses, Brackbill et al. then attempted to reconstruct the original images using just the electrical activity recorded. The resulting reconstructions were similar across all retinas tested. Moreover, they showed a striking resemblance to the original images. These results made it possible to comprehend how the light-response properties of each cell represent visual information that can be used by the brain. Understanding how macaque retinas work in natural conditions is critical to decoding how our own retinas process and convey information. A better knowledge of how the brain uses this input to generate images could ultimately make it possible to design artificial retinas to restore vision in patients with certain forms of blindness.

Inference of nonlinear receptive field subunits with spike-triggered clustering.

Responses of sensory neurons are often modeled using a weighted combination of rectified linear subunits. Since these subunits often cannot be measured directly, a flexible method is needed to infer their properties from the responses of downstream neurons. We present a method for maximum likelihood estimation of subunits by soft-clustering spike-triggered stimuli, and demonstrate its effectiveness in visual neurons. For parasol retinal ganglion cells in macaque retina, estimated subunits partitioned the receptive field into compact regions, likely representing aggregated bipolar cell inputs. Joint clustering revealed shared subunits between neighboring cells, producing a parsimonious population model. Closed-loop validation, using stimuli lying in the null space of the linear receptive field, revealed stronger nonlinearities in OFF cells than ON cells. Responses to natural images, jittered to emulate fixational eye movements, were accurately predicted by the subunit model. Finally, the generality of the approach was demonstrated in macaque V1 neurons.

Unusual Physiological Properties of Smooth Monostratified Ganglion Cell Types in Primate Retina.

The functions of the diverse retinal ganglion cell types in primates and the parallel visual pathways they initiate remain poorly understood. Here, unusual physiological and computational properties of the ON and OFF smooth monostratified ganglion cells are explored. Large-scale multi-electrode recordings from 48 macaque retinas revealed that these cells exhibit irregular receptive field structure composed of spatially segregated hotspots, quite different from the classic center-surround model of retinal receptive fields. Surprisingly, visual stimulation of different hotspots in the same cell produced spikes with subtly different spatiotemporal voltage signatures, consistent with a dendritic contribution to hotspot structure. Targeted visual stimulation and computational inference demonstrated strong nonlinear subunit properties associated with each hotspot, supporting a model in which the hotspots apply nonlinearities at a larger spatial scale than bipolar cells. These findings reveal a previously unreported nonlinear mechanism in the output of the primate retina that contributes to signaling spatial information.

Design Rationale and Performance Evaluation of the Wavelet Health Wristband: Benchtop Validation of a Wrist-Worn Physiological Signal Recorder.

BACKGROUND: Wearable and connected health devices along with the recent advances in mobile and cloud computing provide a continuous, convenient-to-patient, and scalable way to collect personal health data remotely. The Wavelet Health platform and the Wavelet wristband have been developed to capture multiple physiological signals and to derive biometrics from these signals, including resting heart rate (HR), heart rate variability (HRV), and respiration rate (RR). OBJECTIVE: This study aimed to evaluate the accuracy of the biometric estimates and signal quality of the wristband. METHODS: Measurements collected from 35 subjects using the Wavelet wristband were compared with simultaneously recorded electrocardiogram and spirometry measurements. RESULTS: The HR, HRV SD of normal-to-normal intervals, HRV root mean square of successive differences, and RR estimates matched within 0.7 beats per minute (SD 0.9), 7 milliseconds (SD 10), 11 milliseconds (SD 12), and 1 breaths per minute (SD 1) mean absolute deviation of the reference measurements, respectively. The quality of the raw plethysmography signal collected by the wristband, as determined by the harmonic-to-noise ratio, was comparable with that obtained from measurements from a finger-clip plethysmography device. CONCLUSIONS: The accuracy of the biometric estimates and high signal quality indicate that the wristband photoplethysmography device is suitable for performing pulse wave analysis and measuring vital signs.

IpsiHand Bravo: an improved EEG-based brain-computer interface for hand motor control rehabilitation.

Stroke and other nervous system injuries can damage or destroy hand motor control and greatly upset daily activities. Brain computer interfaces (BCIs) represent an emerging technology that can bypass damaged nerves to restore basic motor function and provide more effective rehabilitation. A wireless BCI system was implemented to realize these goals using electroencephalographic brain signals, machine learning techniques, and a custom designed orthosis. The IpsiHand Bravo BCI system is designed to reach a large demographic by using non-traditional brain signals and improving on past BCI system pitfalls.