Compositional Processing Emerges in Neural Networks Solving Math Problems.

A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition. Humans can infer the structured relationships (e.g., grammatical rules) implicit in their sensory observations (e.g., auditory speech), and use this knowledge to guide the composition of simpler meanings into complex wholes. Recent progress in artificial neural networks has shown that when large models are trained on enough linguistic data, grammatical structure emerges in their representations. We extend this work to the domain of mathematical reasoning, where it is possible to formulate precise hypotheses about how meanings (e.g., the quantities corresponding to numerals) should be composed according to structured rules (e.g., order of operations). Our work shows that neural networks are not only able to infer something about the structured relationships implicit in their training data, but can also deploy this knowledge to guide the composition of individual meanings into composite wholes.

Optimized deep neural network architecture for robust detection of epileptic seizures using EEG signals.

OBJECTIVE: Automatic detection of epileptic seizures based on deep learning methods received much attention last year. However, the potential of deep neural networks in seizure detection has not been fully exploited in terms of the optimal design of the model architecture and the detection power of the time-series brain data. In this work, a deep neural network architecture is introduced to learn the temporal dependencies in Electroencephalogram (EEG) data for robust detection of epileptic seizures. METHODS: A deep Long Short-Term Memory (LSTM) network is first used to learn the high-level representations of different EEG patterns. Then, a Fully Connected (FC) layer is adopted to extract the most robust EEG features relevant to epileptic seizures. Finally, these features are supplied to a softmax layer to output predicted labels. RESULTS: The results on a benchmark clinical dataset reveal the prevalence of the proposed approach over the baseline techniques; achieving 100% classification accuracy, 100% sensitivity, and 100% specificity. Our approach is additionally shown to be robust in noisy and real-life conditions. It maintains high detection performance in the existence of common EEG artifacts (muscle activities and eye movement) as well as background noise. CONCLUSIONS: We demonstrate the clinical feasibility of our seizure detection approach achieving superior performance over the cutting-edge techniques in terms of seizure detection performance and robustness. SIGNIFICANCE: Our seizure detection approach can contribute to accurate and robust detection of epileptic seizures in ideal and real-life situations.