Generalization of stochastic-resonance-based threshold networks with Tikhonov regularization.

Injecting artificial noise into a feedforward threshold neural network allows it to become trainable by gradient-based methods and also enlarges the parameter space as well as the range of synaptic weights. This configuration constitutes a stochastic-resonance-based threshold neural network, where the noise level can adaptively converge to a nonzero optimal value for finding a local minimum of the loss criterion. We prove theoretically that the injected noise plays the role of a generalized Tikhonov regularizer for training the designed threshold network. Experiments on regression and classification problems demonstrate that the generalization of the stochastic-resonance-based threshold network is improved by the injection of noise. The feasibility of injecting noise into the threshold neural network opens up the potential for adaptive stochastic resonance in machine learning.

Signal estimation and filtering from quantized observations via adaptive stochastic resonance.

Using a gradient-based algorithm, we investigate signal estimation and filtering in a large-scale summing network of single-bit quantizers. Besides adjusting weights, the proposed learning algorithm also adaptively updates the level of added noise components that are intentionally injected into quantizers. Experimental results show that minimization of the mean-squared error requires a nonzero optimal level of the added noise. The process adaptively achieves in this way a form of stochastic resonance or noise-aided signal processing. This adaptive optimization method of the level of added noise extends the application of adaptive stochastic resonance to some complex nonlinear signal processing tasks.

Study of vibrational resonance in nonlinear signal processing.

Vibrational resonance (VR) intentionally applies high-frequency periodic vibrations to a nonlinear system, in order to obtain enhanced efficiency for a number of information processing tasks. Note that VR is analogous to stochastic resonance where enhanced processing is sought via purposeful addition of a random noise instead of deterministic high-frequency vibrations. Comparatively, due to its ease of implementation, VR provides a valuable approach for nonlinear signal processing, through detailed modalities that are still under investigation. In this paper, VR is investigated in arrays of nonlinear processing devices, where a range of high-frequency sinusoidal vibrations of the same amplitude at different frequencies are injected and shown capable of enhancing the efficiency for estimating unknown signal parameters or for detecting weak signals in noise. In addition, it is observed that high-frequency vibrations with differing frequencies can be considered, at the sampling times, as independent random variables. This property allows us here to develop a probabilistic analysis-much like in stochastic resonance-and to obtain a theoretical basis for the VR effect and its optimization for signal processing. These results provide additional insight for controlling the capabilities of VR for nonlinear signal processing. This article is part of the theme issue 'Vibrational and stochastic resonance in driven nonlinear systems (part 1)'.

Adaptive recursive algorithm for optimal weighted suprathreshold stochastic resonance.

Suprathreshold stochastic resonance (SSR) is a distinct form of stochastic resonance, which occurs in multilevel parallel threshold arrays with no requirements on signal strength. In the generic SSR model, an optimal weighted decoding scheme shows its superiority in minimizing the mean square error (MSE). In this study, we extend the proposed optimal weighted decoding scheme to more general input characteristics by combining a Kalman filter and a least mean square (LMS) recursive algorithm, wherein the weighted coefficients can be adaptively adjusted so as to minimize the MSE without complete knowledge of input statistics. We demonstrate that the optimal weighted decoding scheme based on the Kalman-LMS recursive algorithm is able to robustly decode the outputs from the system in which SSR is observed, even for complex situations where the signal and noise vary over time.

Exploiting vibrational resonance in weak-signal detection.

In this paper, we investigate the first exploitation of the vibrational resonance (VR) effect to detect weak signals in the presence of strong background noise. By injecting a series of sinusoidal interference signals of the same amplitude but with different frequencies into a generalized correlation detector, we show that the detection probability can be maximized at an appropriate interference amplitude. Based on a dual-Dirac probability density model, we compare the VR method with the stochastic resonance approach via adding dichotomous noise. The compared results indicate that the VR method can achieve a higher detection probability for a wider variety of noise distributions.

Capacity of very noisy communication channels based on Fisher information.

We generalize the asymptotic capacity expression for very noisy communication channels to now include coloured noise. For the practical scenario of a non-optimal receiver, we consider the common case of a correlation receiver. Due to the central limit theorem and the cumulative characteristic of a correlation receiver, we model this channel noise as additive Gaussian noise. Then, the channel capacity proves to be directly related to the Fisher information of the noise distribution and the weak signal energy. The conditions for occurrence of a noise-enhanced capacity effect are discussed, and the capacity difference between this noisy communication channel and other nonlinear channels is clarified.

Double-maximum enhancement of signal-to-noise ratio gain via stochastic resonance and vibrational resonance.

This paper studies the signal-to-noise ratio (SNR) gain of a parallel array of nonlinear elements that transmits a common input composed of a periodic signal and external noise. Aiming to further enhance the SNR gain, each element is injected with internal noise components or high-frequency sinusoidal vibrations. We report that the SNR gain exhibits two maxima at different values of the internal noise level or of the sinusoidal vibration amplitude. For the addition of internal noise to an array of threshold-based elements, the condition for occurrence of stochastic resonance is analytically investigated in the limit of weak signals. Interestingly, when the internal noise components are replaced by high-frequency sinusoidal vibrations, the SNR gain displays the vibrational multiresonance phenomenon. In both considered cases, there are certain regions of the internal noise intensity or the sinusoidal vibration amplitude wherein the achieved maximal SNR gain can be considerably beyond unity for a weak signal buried in non-Gaussian external noise. Due to the easy implementation of sinusoidal vibration modulation, this approach is potentially useful for improving the output SNR in an array of nonlinear devices.

Stochastic resonance with colored noise for neural signal detection.

We analyze signal detection with nonlinear test statistics in the presence of colored noise. In the limits of small signal and weak noise correlation, the optimal test statistic and its performance are derived under general conditions, especially concerning the type of noise. We also analyze, for a threshold nonlinearity-a key component of a neural model, the conditions for noise-enhanced performance, establishing that colored noise is superior to white noise for detection. For a parallel array of nonlinear elements, approximating neurons, we demonstrate even broader conditions allowing noise-enhanced detection, via a form of suprathreshold stochastic resonance.

Weak-periodic stochastic resonance in a parallel array of static nonlinearities.

This paper studies the output-input signal-to-noise ratio (SNR) gain of an uncoupled parallel array of static, yet arbitrary, nonlinear elements for transmitting a weak periodic signal in additive white noise. In the small-signal limit, an explicit expression for the SNR gain is derived. It serves to prove that the SNR gain is always a monotonically increasing function of the array size for any given nonlinearity and noisy environment. It also determines the SNR gain maximized by the locally optimal nonlinearity as the upper bound of the SNR gain achieved by an array of static nonlinear elements. With locally optimal nonlinearity, it is demonstrated that stochastic resonance cannot occur, i.e. adding internal noise into the array never improves the SNR gain. However, in an array of suboptimal but easily implemented threshold nonlinearities, we show the feasibility of situations where stochastic resonance occurs, and also the possibility of the SNR gain exceeding unity for a wide range of input noise distributions.

Fisher information as a metric of locally optimal processing and stochastic resonance.

The origins of Fisher information are in its use as a performance measure for parametric estimation. We augment this and show that the Fisher information can characterize the performance in several other significant signal processing operations. For processing of a weak signal in additive white noise, we demonstrate that the Fisher information determines (i) the maximum output signal-to-noise ratio for a periodic signal; (ii) the optimum asymptotic efficacy for signal detection; (iii) the best cross-correlation coefficient for signal transmission; and (iv) the minimum mean square error of an unbiased estimator. This unifying picture, via inequalities on the Fisher information, is used to establish conditions where improvement by noise through stochastic resonance is feasible or not.

Fisher-information condition for enhanced signal detection via stochastic resonance.

Various situations where a signal is enhanced by noise through stochastic resonance are now known. This paper contributes to determining general conditions under which improvement by noise can be a priori decided as feasible or not. We focus on the detection of a known signal in additive white noise. Under the assumptions of a weak signal and a sufficiently large sample size, it is proved, with an inequality based on the Fisher information, that improvement by adding noise is never possible, generically, in these conditions. However, under less restrictive conditions, an example of signal detection is shown with favorable action of adding noise.

Synaptic signal transduction aided by noise in a dynamical saturating model.

A generic dynamical model with saturation for neural signal transduction at the synaptic stage is presented. Analysis of this model of a synaptic pathway demonstrates its ability to give rise to stochastic resonance or improvement by noise, at this stage of signal transmission. Beyond the case of the intrinsic threshold nonlinearity of the neuron response, the results extend the feasibility of stochastic resonance to neural saturating dynamics at the synaptic stage. The present results also constitute the exposition of a new type of nonlinear (saturating) dynamics capable of stochastic resonance.

Comparison of aperiodic stochastic resonance in a bistable system realized by adding noise and by tuning system parameters.

Two methods of realizing aperiodic stochastic resonance (ASR) by adding noise and tuning system parameters in a bistable system, after a scale transformation, can be compared in a real parameter space. In this space, the resonance point of ASR via adding noise denotes the extremum of a line segment, whereas the method of tuning system parameters presents the extrema of a parameter plane. We demonstrate that, in terms of the system performance, the method of tuning system parameters takes the precedence of the approach of adding noise for an adjustable bistable system. Besides, adding noise can be viewed as a specific case of tuning system parameters. Further research shows that the optimal system found by tuning system parameters may be subthreshold or suprathreshold, and the conventional ASR effects might not occur in some suprathreshold optimal systems.

Residual aperiodic stochastic resonance in a bistable dynamic system transmitting a suprathreshold binary signal.

Conventional stochastic resonance can be viewed as an amplitude effect, in which a small (subthreshold) input signal receives assistance from noise to trigger a stronger response from a nonlinear system. We demonstrate another mechanism of improvement by the noise, which is more of a temporal effect. An intrinsically slow system has difficulty to respond to a fast (suprathreshold) input, and the noise plays a constructive role by spurring the system for a more efficient response. The possibility of this form of stochastic resonance is established and studied here in a double-well bistable dynamic system, driven by a suprathreshold random binary signal, with the noise accelerating the switching between wells.

Suprathreshold stochastic resonance and noise-enhanced Fisher information in arrays of threshold devices.

We analyze the parametric estimation that can be performed on a signal buried in noise based on the parsimonious representation provided by a parallel array of threshold devices. The Fisher information contained in the array output about the input parameter is used as the measure of performance in the estimation task. For estimation on a suprathreshold input signal, we establish that enhancement of the Fisher information can be obtained by addition of independent noises to the thresholds in the array. Similar improvement by noise is also shown to be possible for the estimation error of the maximum likelihood estimator. These results extend the applicability of the recently introduced nonlinear phenomenon of suprathreshold stochastic resonance.