Evaluation of Swin Transformer and knowledge transfer for denoising of super-resolution structured illumination microscopy data.

BACKGROUND: Convolutional neural network (CNN)-based methods have shown excellent performance in denoising and reconstruction of super-resolved structured illumination microscopy (SR-SIM) data. Therefore, CNN-based architectures have been the focus of existing studies. However, Swin Transformer, an alternative and recently proposed deep learning-based image restoration architecture, has not been fully investigated for denoising SR-SIM images. Furthermore, it has not been fully explored how well transfer learning strategies work for denoising SR-SIM images with different noise characteristics and recorded cell structures for these different types of deep learning-based methods. Currently, the scarcity of publicly available SR-SIM datasets limits the exploration of the performance and generalization capabilities of deep learning methods. RESULTS: In this work, we present SwinT-fairSIM, a novel method based on the Swin Transformer for restoring SR-SIM images with a low signal-to-noise ratio. The experimental results show that SwinT-fairSIM outperforms previous CNN-based denoising methods. Furthermore, as a second contribution, two types of transfer learning-namely, direct transfer and fine-tuning-were benchmarked in combination with SwinT-fairSIM and CNN-based methods for denoising SR-SIM data. Direct transfer did not prove to be a viable strategy, but fine-tuning produced results comparable to conventional training from scratch while saving computational time and potentially reducing the amount of training data required. As a third contribution, we publish four datasets of raw SIM images and already reconstructed SR-SIM images. These datasets cover two different types of cell structures, tubulin filaments and vesicle structures. Different noise levels are available for the tubulin filaments. CONCLUSION: The SwinT-fairSIM method is well suited for denoising SR-SIM images. By fine-tuning, already trained models can be easily adapted to different noise characteristics and cell structures. Furthermore, the provided datasets are structured in a way that the research community can readily use them for research on denoising, super-resolution, and transfer learning strategies.

Active Learning for Handling Missing Data.

Recently, the massive growth of IoT devices and Internet data, which are widely used in many applications, including industry and healthcare, has dramatically increased the amount of free unlabeled data collected. However, this unlabeled data is useless if we want to learn supervised machine learning models. The expensive and time-consuming cost of labeling makes the problem even more challenging. Here, the active learning (AL) technique provides a solution by labeling small but highly informative and representative data, which guarantees a high degree of generalizability over space and improves classification performance with data we have never seen before. The task is more difficult when the active learner has no predefined knowledge, such as initial training data, and when the obtained data is incomplete (i.e., contains missing values). In previous studies, the missing data should first be imputed. Then, the active learner selects from the available unlabeled data, regardless of whether the points were originally observed or imputed. However, selecting inaccurate imputed data points would negatively affect the active learner and prevent it from selecting informative and/or representative points, thus reducing the overall classification performance of the prediction models. This motivated us to introduce a novel query selection strategy that accounts for imputation uncertainty when querying new points. For this purpose, we first introduce a novel multiple imputation method that considers feature importance in selecting the most promising feature groups for missing values estimation. This multiple imputation method provides the ability to quantify the imputation uncertainty of each imputed data point. Furthermore, in each of the two phases of the proposed active learner (exploration and exploitation), imputation uncertainty is taken into account to reduce the probability of selecting points with high imputation uncertainty. We tested the effectiveness of the proposed active learner on different binary and multiclass datasets with different missing rates.

sEMG-based prediction of human forearm movements utilizing a biomechanical model based on individual anatomical/ physiological measures and a reduced set of optimization parameters.

For assistive devices such as active orthoses, exoskeletons or other close-to-body robotic-systems, the immediate prediction of biological limb movements based on biosignals in the respective control system can be used to enable intuitive operation also by untrained users e.g. in healthcare, rehabilitation or industrial scenarios. Surface electromyography (sEMG) signals from the muscles that drive the limbs can be measured before the actual movement occurs and, hence, can be used as source for predicting limb movements. The aim of this work was to create a model that can be adapted to a new user or movement scenario with little measurement and computing effort. Therefore, a biomechanical model is presented that predicts limb movements of the human forearm based on easy to measure sEMG signals of the main muscles involved in forearm actuation (lateral and long head of triceps and short and long head of biceps). The model has 42 internal parameters of which 37 were attributed to 8 individually measured physiological measures (location of acromion at the shoulder, medial/lateral epicondyles as well as olecranon at the elbow, and styloid processes of radius/ulna at the wrist; maximum muscle forces of biceps and triceps). The remaining 5 parameters are adapted to specific movement conditions in an optimization process. The model was tested in an experimental study with 31 subjects in which the prediction quality of the model was assessed. The quality of the movement prediction was evaluated by using the normalized mean absolute error (nMAE) for two arm postures (lower, upper), two load conditions (2 kg, 4 kg) and two movement velocities (slow, fast). For the resulting 8 experimental combinations the nMAE varied between nMAE = 0.16 and nMAE = 0.21 (lower numbers better). An additional quality score (QS) was introduced that allows direct comparison between different movements. This score ranged from QS = 0.25 to QS = 0.40 (higher numbers better) for the experimental combinations. The above formulated aim was achieved with good prediction quality by using only 8 individual measurements (easy to collect body dimensions) and the subsequent optimization of only 5 parameters. At the same time, just easily accessible sEMG measurement locations are used to enable simple integration, e.g. in exoskeletons. This biomechanical model does not compete with models that measure all sEMG signals of the muscle heads involved in order to achieve the highest possible prediction quality.

Using Artificial Intelligence for Assistance Systems to Bring Motor Learning Principles into Real World Motor Tasks.

Humans learn movements naturally, but it takes a lot of time and training to achieve expert performance in motor skills. In this review, we show how modern technologies can support people in learning new motor skills. First, we introduce important concepts in motor control, motor learning and motor skill learning. We also give an overview about the rapid expansion of machine learning algorithms and sensor technologies for human motion analysis. The integration between motor learning principles, machine learning algorithms and recent sensor technologies has the potential to develop AI-guided assistance systems for motor skill training. We give our perspective on this integration of different fields to transition from motor learning research in laboratory settings to real world environments and real world motor tasks and propose a stepwise approach to facilitate this transition.

Adaptive dimensionality reduction for neural network-based online principal component analysis.

"Principal Component Analysis" (PCA) is an established linear technique for dimensionality reduction. It performs an orthonormal transformation to replace possibly correlated variables with a smaller set of linearly independent variables, the so-called principal components, which capture a large portion of the data variance. The problem of finding the optimal number of principal components has been widely studied for offline PCA. However, when working with streaming data, the optimal number changes continuously. This requires to update both the principal components and the dimensionality in every timestep. While the continuous update of the principal components is widely studied, the available algorithms for dimensionality adjustment are limited to an increment of one in neural network-based and incremental PCA. Therefore, existing approaches cannot account for abrupt changes in the presented data. The contribution of this work is to enable in neural network-based PCA the continuous dimensionality adjustment by an arbitrary number without the necessity to learn all principal components. A novel algorithm is presented that utilizes several PCA characteristics to adaptivly update the optimal number of principal components for neural network-based PCA. A precise estimation of the required dimensionality reduces the computational effort while ensuring that the desired amount of variance is kept. The computational complexity of the proposed algorithm is investigated and it is benchmarked in an experimental study against other neural network-based and incremental PCA approaches where it produces highly competitive results.

The NEST Dry-Run Mode: Efficient Dynamic Analysis of Neuronal Network Simulation Code.

NEST is a simulator for spiking neuronal networks that commits to a general purpose approach: It allows for high flexibility in the design of network models, and its applications range from small-scale simulations on laptops to brain-scale simulations on supercomputers. Hence, developers need to test their code for various use cases and ensure that changes to code do not impair scalability. However, running a full set of benchmarks on a supercomputer takes up precious compute-time resources and can entail long queuing times. Here, we present the NEST dry-run mode, which enables comprehensive dynamic code analysis without requiring access to high-performance computing facilities. A dry-run simulation is carried out by a single process, which performs all simulation steps except communication as if it was part of a parallel environment with many processes. We show that measurements of memory usage and runtime of neuronal network simulations closely match the corresponding dry-run data. Furthermore, we demonstrate the successful application of the dry-run mode in the areas of profiling and performance modeling.

Coupled singular value decomposition of a cross-covariance matrix.

We derive coupled on-line learning rules for the singular value decomposition (SVD) of a cross-covariance matrix. In coupled SVD rules, the singular value is estimated alongside the singular vectors, and the effective learning rates for the singular vector rules are influenced by the singular value estimates. In addition, we use a first-order approximation of Gram-Schmidt orthonormalization as decorrelation method for the estimation of multiple singular vectors and singular values. Experiments on synthetic data show that coupled learning rules converge faster than Hebbian learning rules and that the first-order approximation of Gram-Schmidt orthonormalization produces more precise estimates and better orthonormality than the standard deflation method.

Bootstrapping cognition from behavior-a computerized thought experiment.

We show that simple perceptual competences can emerge from an internal simulation of action effects and are thus grounded in behavior. A simulated agent learns to distinguish between dead ends and corridors without the necessity to represent these concepts in the sensory domain. Initially, the agent is only endowed with a simple value system and the means to extract low-level features from an image. In the interaction with the environment, it acquires a visuo-tactile forward model that allows the agent to predict how the visual input is changing under its movements, and whether movements will lead to a collision. From short-term predictions based on the forward model, the agent learns an inverse model. The inverse model in turn produces suggestions about which actions should be simulated in long-term predictions, and long-term predictions eventually give rise to the perceptual ability.

Emotion and memory: Event-related potential indices predictive for subsequent successful memory depend on the emotional mood state.

The present research investigated the influencesof emotional mood states on cognitive processes and neural circuits during long-term memory encoding using event-related potentials (ERPs). We assessed whether the subsequent memory effect (SME), an electrophysiological index of successful memory encoding, varies as a function of participants' current mood state. ERPs were recorded while participants in good or bad mood states were presented with words that had to be memorized for subsequent recall. In contrast to participants in bad mood, participants in good mood most frequently applied elaborative encoding styles. At the neurophysiological level, ERP analyses showed that potentials to subsequently recalled words were more positive than to forgotten words at central electrodes in the time interval of 500-650 ms after stimulus onset (SME). At fronto-central electrodes, a polarity-reversed SME was obtained. The strongest modulations of the SME by participants' mood state were obtained at fronto-temporal electrodes. These differences in the scalp topography of the SME suggest that successful recall relies on partially separable neural circuits for good and bad mood states. The results are consistent with theoretical accounts of the interface between emotion and cognition that propose mood-dependent cognitive styles.

Mood states modulate activity in semantic brain areas during emotional word encoding.

It is controversially discussed whether or not mood-congruent recall (i.e., superior recall for mood-congruent material) reflects memory encoding processes or reduces to processes during retrieval. We therefore investigated the neurophysiological correlates of mood-dependent memory during emotional word encoding. Event-related potentials (ERPs) were recorded while participants in good or bad mood states encoded words of positive and negative valence. Words were either complete or had to be generated from fragments. Participants had to memorize words for subsequent recall. Mood-congruent recall tended to be largest in good mood for generated words. Starting at 200 ms, mood-congruent ERP effects of word valence were obtained in good, but not in bad mood. Only for good mood, source analysis revealed valence-related activity in ventral temporal cortex and for generated words also in prefrontal cortex. These areas are known to be involved in semantic processing. Our findings are consistent with the view that mood-congruent recall depends on the activation of mood-congruent semantic knowledge during encoding. Incoming stimuli are more readily transformed according to stored knowledge structures in good mood particularly during generative encoding tasks. The present results therefore show that mood-congruent memory originates already during encoding and cannot be reduced to strategic processes during retrieval.

Spectral contrasts for landmark navigation.

Visual robot navigation in outdoor environments would benefit from an illumination-independent representation of images. We explore how such a representation, comprising a black skyline of objects in front of a white sky, can be obtained from dual-channel spectral contrast measures. Light from sky and natural objects under different conditions of illumination was analyzed by five spectral channels: ultraviolet, blue, green, red, and near infrared. Linear discriminant analysis was applied to determine the optimal linear separation between sky and object points. A statistical comparison shows that contrasts with large differences in the wavelength of the two channels, specifically ultraviolet-infrared, blue-infrared, and ultraviolet-red, yield the best separation. Within a single channel, the best separation was obtained for ultraviolet light. The gain in separation quality when all five channels were included is relatively small.

Learning visuomotor transformations for gaze-control and grasping.

For reaching to and grasping of an object, visual information about the object must be transformed into motor or postural commands for the arm and hand. In this paper, we present a robot model for visually guided reaching and grasping. The model mimics two alternative processing pathways for grasping, which are also likely to coexist in the human brain. The first pathway directly uses the retinal activation to encode the target position. In the second pathway, a saccade controller makes the eyes (cameras) focus on the target, and the gaze direction is used instead as positional input. For both pathways, an arm controller transforms information on the target's position and orientation into an arm posture suitable for grasping. For the training of the saccade controller, we suggest a novel staged learning method which does not require a teacher that provides the necessary motor commands. The arm controller uses unsupervised learning: it is based on a density model of the sensor and the motor data. Using this density, a mapping is achieved by completing a partially given sensorimotor pattern. The controller can cope with the ambiguity in having a set of redundant arm postures for a given target. The combined model of saccade and arm controller was able to fixate and grasp an elongated object with arbitrary orientation and at arbitrary position on a table in 94% of trials.

Priming trait inferences through pictures and moving pictures: the impact of open and closed mindsets.

A newly developed paradigm for studying spontaneous trait inferences (STI) was applied in 3 experiments. The authors primed dyadic stimulus behaviors involving a subject (S) and an object (O) person through degraded pictures or movies. An encoding task called for the verification of either a graphical feature or a semantic interpretation, which either fit or did not fit the primed behavior. Next, participants had to identify a trait word that appeared gradually behind a mask and that either matched or did not match the primed behavior. STI effects, defined as shorter identification latencies for matching than nonmatching traits, were stronger for S than for O traits, after graphical rather than semantic encoding decisions and after encoding failures. These findings can be explained by assuming that trait inferences are facilitated by open versus closed mindsets supposed to result from distracting (graphical) encoding tasks or encoding failures (involving nonfitting interpretations).