Experimental Demonstration of Reservoir Computing with Self-Assembled Percolating Networks of Nanoparticles.

The complex self-assembled network of neurons and synapses that comprises the biological brain enables natural information processing with remarkable efficiency. Percolating networks of nanoparticles (PNNs) are complex self-assembled nanoscale systems that have been shown to possess many promising brain-like attributes and which are therefore appealing systems for neuromorphic computation. Here experiments are performed that show that PNNs can be utilized as physical reservoirs within a nanoelectronic reservoir computing framework and demonstrate successful computation for several benchmark tasks (chaotic time series prediction, nonlinear transformation, and memory capacity). For each task, relevant literature results are compiled and it is shown that the performance of the PNNs compares favorably to that previously reported from nanoelectronic reservoirs. It is then demonstrated experimentally that PNNs can be used for spoken digit recognition with state-of-the-art accuracy. Finally, a parallel reservoir architecture is emulated, which increases the dimensionality and richness of the reservoir outputs and results in further improvements in performance across all tasks.

Computation via Neuron-like Spiking in Percolating Networks of Nanoparticles.

The biological brain is a highly efficient computational system in which information processing is performed via electrical spikes. Neuromorphic computing systems that work on similar principles could support the development of the next generation of artificial intelligence and, in particular, enable low-power edge computing. Percolating networks of nanoparticles (PNNs) have previously been shown to exhibit critical spiking behavior, with promise for highly efficient natural computation. Here we employ a rate coding scheme to show that PNNs can perform Boolean operations and image classification. Near perfect accuracy is achieved in both tasks by manipulating the spiking activity using certain control voltages. We demonstrate that the key to successful computation is that nanoscale tunnel gaps within the percolating networks transform input data through a powerful modulus-like nonlinearity. These results provide a basis for implementation of further computational schemes that exploit the brain-like criticality of these networks.

Neuromorphic behaviour in discontinuous metal films.

Physical systems that exhibit brain-like behaviour are currently under intense investigation as platforms for neuromorphic computing. We show that discontinuous metal films, comprising irregular flat islands on a substrate and formed using simple evaporation processes, exhibit correlated avalanches of electrical signals that mimic those observed in the cortex. We further demonstrate that these signals meet established criteria for criticality. We perform a detailed experimental investigation of the atomic-scale switching processes that are responsible for these signals, and show that they mimic the integrate-and-fire mechanism of biological neurons. Using numerical simulations and a simple circuit model, we show that the characteristic features of the switching events are dependent on the network state and the local position of the switch within the complex network. We conclude that discontinuous films provide an interesting potential platform for brain-inspired computing.

Geometric frustration in the myosin superlattice of vertebrate muscle.

Geometric frustration results from an incompatibility between minimum energy arrangements and the geometry of a system, and gives rise to interesting and novel phenomena. Here, we report geometric frustration in a native biological macromolecular system---vertebrate muscle. We analyse the disorder in the myosin filament rotations in the myofibrils of vertebrate striated (skeletal and cardiac) muscle, as seen in thin-section electron micrographs, and show that the distribution of rotations corresponds to an archetypical geometrically frustrated system---the triangular Ising antiferromagnet. Spatial correlations are evident out to at least six lattice spacings. The results demonstrate that geometric frustration can drive the development of structure in complex biological systems, and may have implications for the nature of the actin--myosin interactions involved in muscle contraction. Identification of the distribution of myosin filament rotations with an Ising model allows the extensive results on the latter to be applied to this system. It shows how local interactions (between adjacent myosin filaments) can determine long-range order and, conversely, how observations of long-range order (such as patterns seen in electron micrographs) can be used to estimate the energetics of these local interactions. Furthermore, since diffraction by a disordered system is a function of the second-order statistics, the derived correlations allow more accurate diffraction calculations, which can aid in interpretation of X-ray diffraction data from muscle specimens for structural analysis.

Stochastic Spiking Behavior in Neuromorphic Networks Enables True Random Number Generation.

There is currently a great deal of interest in the use of nanoscale devices to emulate the behaviors of neurons and synapses and to facilitate brain-inspired computation. Here, it is shown that percolating networks of nanoparticles exhibit stochastic spiking behavior that is strikingly similar to that observed in biological neurons. The spiking rate can be controlled by the input stimulus, similar to "rate coding" in biology, and the distributions of times between events are log-normal, providing insights into the atomic-scale spiking mechanism. The stochasticity of the spiking behavior is then used for true random number generation, and the high quality of the generated random bit-streams is demonstrated, opening up promising routes toward integration of neuromorphic computing with secure information processing.

Atomic Scale Dynamics Drive Brain-like Avalanches in Percolating Nanostructured Networks.

Self-assembled networks of nanoparticles and nanowires have recently emerged as promising systems for brain-like computation. Here, we focus on percolating networks of nanoparticles which exhibit brain-like dynamics. We use a combination of experiments and simulations to show that the brain-like network dynamics emerge from atomic-scale switching dynamics inside tunnel gaps that are distributed throughout the network. The atomic-scale dynamics emulate leaky integrate and fire (LIF) mechanisms in biological neurons, leading to the generation of critical avalanches of signals. These avalanches are quantitatively the same as those observed in cortical tissue and are signatures of the correlations that are required for computation. We show that the avalanches are associated with dynamical restructuring of the networks which self-tune to balanced states consistent with self-organized criticality. Our simulations allow visualization of the network states and detailed mechanisms of signal propagation.

A software framework for real-time multi-modal detection of microsleeps.

A software framework is described which was designed to process EEG, video of one eye, and head movement in real time, towards achieving early detection of microsleeps for prevention of fatal accidents, particularly in transport sectors. The framework is based around a pipeline structure with user-replaceable signal processing modules. This structure can encapsulate a wide variety of feature extraction and classification techniques and can be applied to detecting a variety of aspects of cognitive state. Users of the framework can implement signal processing plugins in C++ or Python. The framework also provides a graphical user interface and the ability to save and load data to and from arbitrary file formats. Two small studies are reported which demonstrate the capabilities of the framework in typical applications: monitoring eye closure and detecting simulated microsleeps. While specifically designed for microsleep detection/prediction, the software framework can be just as appropriately applied to (i) other measures of cognitive state and (ii) development of biomedical instruments for multi-modal real-time physiological monitoring and event detection in intensive care, anaesthesiology, cardiology, neurosurgery, etc. The software framework has been made freely available for researchers to use and modify under an open source licence.

Losing the struggle to stay awake: divergent thalamic and cortical activity during microsleeps.

Maintaining alertness is critical for safe and successful performance of most human activities. Consequently, microsleeps during continuous visuomotor tasks, such as driving, can be very serious, not only disrupting performance but sometimes leading to injury or death due to accidents. We have investigated the neural activity underlying behavioral microsleeps--brief (0.5-15 s) episodes of complete failure to respond accompanied by slow eye-closures--and EEG theta activity during drowsiness in a continuous task. Twenty healthy normally-rested participants performed a 50-min continuous tracking task while fMRI, EEG, eye-video, and responses were simultaneously recorded. Visual rating of performance and eye-video revealed that 70% of the participants had frequent microsleeps. fMRI analysis revealed a transient decrease in thalamic, posterior cingulate, and occipital cortex activity and an increase in frontal, posterior parietal, and parahippocampal activity during microsleeps. The transient activity was modulated by the duration of the microsleep. In subjects with frequent microsleeps, power in the post-central EEG theta was positively correlated with the BOLD signal in the thalamus, basal forebrain, and visual, posterior parietal, and prefrontal cortices. These results provide evidence for distinct neural changes associated with microsleeps and with EEG theta activity during drowsiness in a continuous task. They also suggest that the occurrence of microsleeps during an active task is not a global deactivation process but involves localized activation of fronto-parietal cortex, which, despite a transient loss of arousal, may constitute a mechanism by which these regions try to restore responsiveness.

Optimal EEG feature selection from average distance between events and non-events.

Biosignal classification systems often have to deal with extraneous features, highly imbalanced datasets, and a low SNR. A robust feature selection/reduction method is a crucial step in this process. Sets of artificial data were generated to test a prototype EEG-based microsleep detection system, consisting of a combination of EEG and 2-s bursts of 15-Hz sinusoids of varied signal-to-noise ratios (SNRs) ranging from 16 to 0.03. The balance between events and non-events was varied between evenly balanced and highly imbalanced (e.g., events occurring only 2% of the time). Features were spectral estimates of various EEG bands (e.g., alpha band power) or ratios between them. A total of 34 features for each of the 16 channels yielded a total of 544 features. Five minutes of EEG from a total of eight subjects were used in the generation of the artificial data. Several feature reduction and classifier structures were investigated. Taking only a single feature corresponding to the maximum of average distance between events and non-events (ADEN) on unbalanced data yielded a phi correlation of 0.94 on the mock data with an SNR of 0.3, compared with a phi coefficient of 0.00 for principal component analysis (PCA). ADEN consistently outperformed alternative system configurations, independent of the classifier utilized. While ADEN's high performance may be due to the nature of the artificial dataset, this simulation has demonstrated strong potential compared to other feature selection/reduction methods.

A wearable device for measuring eye dynamics in real-world conditions.

Drowsiness and lapses of responsiveness have the potential to cause fatalities in many occupations. One subsystem of a prototype device which aims to detect these lapses as they occur is described. A head-mounted camera measures several features of the eye that are known to correlate with drowsiness. The system was tested with eight combinations of eye colour, ambient lighting, and eye glasses to simulate typical real-world input conditions. A task was completed for each set of conditions to simulate a range of eye movement-saccades, tracking, and eye closure. Our image processing software correctly classified 99.3% of video frames as open/closed/partly closed, and the error rate was not affected by the combinations of input conditions. Most errors occurred during eyelid movement. The accuracy of the pupil localisation was also not influenced by input conditions, with the possible exception of one subject's glasses.

Detection of lapses in responsiveness from the EEG.

A system capable of reliably detecting lapses in responsiveness ('lapses') has the potential to increase safety in many occupations. We have developed an approach for detecting the state of lapsing with second-scale temporal resolution using data from 15 subjects performing a one-dimensional (1D) visuomotor tracking task for two 1 h sessions while their electroencephalogram (EEG), facial video, and tracking performances were recorded. Lapses identified using a combination of facial video and tracking behaviour were used to train the classification models. Linear discriminant analysis was used to form detection models based on individual subject data and stacked generalization was utilized to combine the outputs of multiple classifiers to obtain the final prediction. The performance of detectors estimating the lapse/not-lapse state at 1 Hz based on power spectral features, approximate entropy, fractal dimension, and Lempel-Ziv complexity of the EEG was compared. Best lapse state estimation performance was achieved using the detector model created using power spectral features with an area under the curve from receiver operating characteristic analysis of 0.86 ± 0.03 (mean±SE) and an area under the precision-recall curve of 0.43 ± 0.09. A novel technique was developed to provide improved estimation of accuracy of detection of variable-duration events. Via this approach, we were able to show that the detection of lapse events from spectral power features was of moderate accuracy (sensitivity = 73.5%, selectivity = 25.5%).

Prior estimate-based compressed sensing in parallel MRI.

Two improved compressed sensing (CS)-based image reconstruction methods for MRI are proposed: prior estimate-based compressed sensing (PECS) and sensitivity encoding-based compressed sensing (SENSECS). PECS allows prior knowledge of the underlying image to be intrinsically incorporated in the image recovery process, extending the use of data sorting as first proposed by Adluru and DiBella (Int J Biomed Imaging 2008: 341648). It does so by rearranging the elements in the underlying image based on the magnitude information gathered from a prior image estimate, so that the underlying image can be recovered in a new form that exhibits a higher level of sparsity. SENSECS is an application of PECS in parallel imaging. In SENSECS, image reconstruction is carried out in two stages: SENSE and PECS, with the SENSE reconstruction being used as a image prior estimate in the following PECS reconstruction. SENSECS bypasses the conflict of sampling pattern design in directly applying CS recovery in multicoil data sets and exploits the complementary characteristics of SENSE-type and CS-type reconstructions, hence achieving better image reconstructions than using SENSE or CS alone. The characteristics of PECS and SENSECS are investigated using experimental data.

The relationship between behavioural microsleeps, visuomotor performance and EEG theta.

Visuomotor performance and responsiveness deteriorates with time-on-task due to drowsiness and increased propensity to sleep. Frequent episodes of behavioural microsleep (BM) are also common during extended and monotonous tasks. In this study, simultaneous recording of EEG, eye-video, and continuous visuomotor response is used to investigate visuomotor performance and EEG activity during tonic drowsiness and phasic BMs. The data were collected from 20 healthy volunteers while they performed a continuous 2-D pursuit tracking task for 50 min. We identified episodes of BMs by expert visual rating of eye-video and visuomotor response using a set of pre-defined criteria. Visuomotor performance and EEG activity were correlated with and without BM events. A moderate correlation was observed between visuomotor error and theta activity in EEG at a posterior channel (Pz) before the removal of BMs. However, when BMs were removed from the data, the correlation dropped in most subjects. Furthermore, most of the large fluctuations in performance observed during the visuomotor task disappeared after the removal of BMs. This suggests that episodic behaviours such as BMs contribute substantially to fluctuations in performance and to EEG theta activity during an extended task, and that they should be taken into account when studying tonic drowsiness.

Automated video-based measurement of eye closure for detecting behavioral microsleep.

A device capable of continuously monitoring an individual's levels of alertness in real-time is highly desirable for preventing drowsiness and microsleep related accidents. This paper presents a development of non-intrusive and light-insensitive video-based system that uses computer-vision methods to measure facial metric for identifying visible facial signs of drowsiness and behavioral microsleep. The developed system uses a remotely placed camera with a near-infrared illumination to acquire the video. The computer-vision methods are then applied to sequentially localize face, eyes, and eyelids positions to measure ratio of eye closure. The system was evaluated in frontal images of nine subjects with varying facial structures and exhibiting several ratio of eye closure and eye gaze under fully dark and ambient lighting conditions. The preliminary results showed promising results with sufficient accuracy to distinguish between fully closed, half closed, and fully open eyes.

Lapses of responsiveness: Characteristics, detection, and underlying mechanisms.

Lapses in responsiveness ('lapses'), particularly microsleeps and attention lapses, are complete disruptions in performance from approximately 0.5-15 s. They are of particular importance in the transport sector in which there is a need to maintain sustained attention for extended periods and in which lapses can lead to multiple-fatality accidents.

Measurement of BOLD changes due to cued eye-closure and stopping during a continuous visuomotor task via model-based and model-free approaches.

As a precursor for investigation of changes in neural activity underlying lapses of responsiveness, we set up a system to simultaneously record functional magnetic resonance imaging (fMRI), eye-video, EOG, and continuous visuomotor response inside an MRI scanner. The BOLD fMRI signal was acquired during a novel 2-D tracking task in which participants (10 males, 10 females) were cued to either briefly stop tracking and close their eyes (Stop +Close) or to briefly stop tracking (Stop) only. The onset and duration of eye-closure and stopping were identified post hoc from eye-video, EOG, and visuomotor response. fMRI data were analyzed using a general linear model (GLM) and tensorial independent component analysis (TICA). The GLM-based analysis identified predominantly increased blood oxygenation level dependent (BOLD) activity during eye-closure and stopping in multisensory areas, sensory-motor integration areas, and default-mode regions. Stopping during tracking elicited increased activity in visual processing areas, sensory-motor integration areas, and premotor areas. TICA separated the spatio-temporal pattern of activity into multiple task-related networks including the 1) occipito-medial frontal eye-movement network, 2) sensory areas, 3) left-lateralized visuomotor network, and 4) fronto-parietal visuomotor network, which were modulated differently by Stop +Close and Stop. The results demonstrate the merits of using simultaneous fMRI, behavioral, and physiological recordings to investigate the mechanisms underlying complex human behaviors in the human brain. Furthermore, knowledge of widespread modulations in brain activity due to voluntary eye-closure or stopping during a continuous visuomotor task is important for studies of the brain mechanisms underlying involuntary behaviors, such as microsleeps and attention lapses, which are often accompanied by brief eye-closure and/or response failures.

fMRI correlates of behavioural microsleeps during a continuous visuomotor task.

Behavioural microsleeps (BMs) are brief episodes of absent responsiveness accompanied by slow-eye-closure. They frequently occur as a consequence of sleep-deprivation, an extended monotonous task, and are modulated by the circadian rhythm and sleep homeostatic pressure. In this paper, a multimodal method to investigate the neural correlates of BMs using simultaneous recording of fMRI, eye-video, VEOG, and continuous visuomotor response is presented. The data were collected from 20 healthy volunteers while they performed a continuous visuomotor tracking task inside an MRI scanner for 50 min. The BMs were identified post-hoc by expert visual rating of eye-video and visuomotor response using a set of pre-defined criteria. fMRI analysis of BMs revealed changes in haemodynamic activity in several cortical and sub-cortical regions associated with visuomotor control and arousal.

Determination of myosin filament orientations in electron micrographs of muscle cross sections.

An automated image analysis system for determining myosin filament azimuthal rotations, or orientations, in electron micrographs of muscle cross sections is described. The micrographs of thin sections intersect the myosin filaments which lie on a triangular lattice. The myosin filament profiles are variable and noisy, and the images exhibit a variable contrast and background. Filament positions are determined by filtering with a point spread function that incorporates the local symmetry of the lattice. Filament orientations are determined by correlation with a template that incorporates the salient filament characteristics, and the orientations are classified using a Gaussian mixture model. The precision of the technique is assessed by application to a variety of micrographs and comparison with manual classification of the orientations. The system provides a convenient, robust, and rapid means of analysing micrographs containing many filaments to study the distribution of filament orientations.

Improved matrix inversion in image plane parallel MRI.

A new 3D parallel magnetic resonance imaging (MRI) method named Generalized Unaliasing Incorporating Support constraint and sensitivity Encoding (GUISE) is presented. GUISE allows direct image recovery from arbitrary Cartesian k-space trajectories. However, periodic k-space sampling patterns are considered for reconstruction efficiency. Image recovery methods such as 2D SENSE (SENSitivity Encoding) and 2D CAIPIRINHA (Controlled Aliasing In Parallel Imaging Results IN Higher Acceleration) are special instances of GUISE where specific restrictions are placed on the k-space sampling patterns used. It is shown that the sampling pattern has large impacts on the image reconstruction error due to noise. An efficient sampling pattern design method that incorporates prior knowledge of object support and coil sensitivity profile is proposed. It requires no experimental trials and could be used in clinical imaging. Comparison of the proposed sampling pattern design method with 2D SENSE and 2D CAIPIRINHA are made based on both simulation and experiment results. It is seen that this new adaptive sampling pattern design method results in a lower noise level in reconstructions due to better exploitation of the coil sensitivity variation and object support constraint. In addition, elimination of the non-object region from reconstruction potentially allows an acceleration factor higher than the number of receiver coils used.