Performance-guaranteed regularization in maximum likelihood method: Gauge symmetry in Kullback-Leibler divergence.

The maximum likelihood method is the best-known method for estimating the probabilities behind the data. However, the conventional method obtains the probability model closest to the empirical distribution, resulting in overfitting. Then regularization methods prevent the model from being excessively close to the wrong probability, but little is known systematically about their performance. The idea of regularization is similar to error-correcting codes, which obtain optimal decoding by mixing suboptimal solutions with an incorrectly received code. The optimal decoding in error-correcting codes is achieved based on gauge symmetry. We propose a theoretically guaranteed regularization in the maximum likelihood method by focusing on a gauge symmetry in Kullback-Leibler divergence. In our approach, we obtain the optimal model without the need to search for hyperparameters frequently appearing in regularization.

Diversity of dynamical behaviors due to initial conditions: Extension of the Ott-Antonsen ansatz for identical Kuramoto-Sakaguchi phase oscillators.

The Ott-Antonsen ansatz is a powerful tool to extract the behaviors of coupled phase oscillators, but it imposes a strong restriction on the initial condition. Herein, an extension of the Ott-Antonsen ansatz is proposed to relax the restriction, enabling the systematic approximation of the behavior of a globally coupled phase oscillator system with an arbitrary initial condition. The proposed method is validated on the Kuramoto-Sakaguchi model of identical phase oscillators. The method yields cluster and chimera-like solutions that are not obtained by the conventional ansatz.

Langevin dynamics neglecting detailed balance condition.

An improved method for driving a system into a desired distribution, for example, the Gibbs-Boltzmann distribution, is proposed, which makes use of an artificial relaxation process. The standard techniques for achieving the Gibbs-Boltzmann distribution involve numerical simulations under the detailed balance condition. In contrast, in the present study we formulate the Langevin dynamics, for which the corresponding Fokker-Planck operator includes an asymmetric component violating the detailed balance condition. This leads to shifts in the eigenvalues and results in the acceleration of the relaxation toward the steady state. The numerical implementation demonstrates faster convergence and shorter correlation time, and the technique of biased event sampling, Nemoto-Sasa theory, further highlights the efficacy of our method.

Full-order fluctuation-dissipation relation for a class of nonequilibrium steady states.

Acceleration of relaxation toward a fixed stationary distribution via violation of detailed balance was reported in the context of a Markov chain Monte Carlo method recently. Inspired by this result, systematic methods to violate detailed balance in Langevin dynamics were formulated by using exponential and rotational nonconservative forces. In the present paper, we accentuate that such specific nonconservative forces relate to the large deviation of total heat in an equilibrium state. The response to these nonconservative forces can be described by the intrinsic fluctuation of the total heat in the equilibrium state. Consequently, the fluctuation-dissipation relation for nonequilibrium steady states is derived without recourse to a linear response approximation.

Violation of detailed balance accelerates relaxation.

Recent studies have experienced the acceleration of convergence in Markov chain Monte Carlo methods implemented by the systems without detailed balance condition (DBC). However, such advantage of the violation of DBC has not been confirmed in general. We investigate the effect of the absence of DBC on the convergence toward equilibrium. Surprisingly, it is shown that the DBC violation always makes the relaxation faster. Our result implies the existence of a kind of thermodynamic inequality that connects the nonequilibrium process relaxing toward steady state with the relaxation process which has the same probability distribution as its equilibrium state.

Singular probability distribution of shot-noise driven systems.

We study the stationary probability distribution of a system driven by shot noise. We find that both in the overdamped and underdamped regime, the coordinate distribution displays power-law singularities in its central part. For sufficiently low rate of noise pulses they correspond to distribution peaks. We find the positions of the peaks and the corresponding exponents. In the underdamped regime the peak positions are given by a geometric progression. The energy distribution in this case also displays multiple peaks with positions given by a geometric progression. Such structure is a signature of the shot-noise induced fluctuations. The analytical results are in excellent agreement with numerical simulations.

Relation between optimal nonlinearity and non-Gaussian noise: enhancing a weak signal in a nonlinear system.

In the study of stochastic resonance, it is often mentioned that nonlinearity can enhance a weak signal embedded in noise. In order to give a systematic proof of the signal enhancement in nonlinear systems, we derive an optimal nonlinearity that maximizes a signal-to-noise ratio (SNR). The obtained optimal nonlinearity yields the maximum unbiased signal estimation performance, which is known in the context of information theory. It is found that a linear system is optimal for a Gaussian noise, but for a non-Gaussian noise, there exist nonlinear systems that can achieve an SNR higher than that obtained from linear systems. This analysis refers to a system subjected to an additive non-Gaussian noise with a small signal input.

Design and characterization of nonlinear functions for the transmission of a small signal with non-Gaussian noise.

We design nonlinear functions for the transmission of a small signal with non-Gaussian noise and perform experiments to characterize their responses. Using statistical design theory [A. Ichiki and Y. Tadokoro, Phys. Rev. E 87, 012124 (2013)], a static nonlinear function is estimated from the probability density function of the given noise in order to maximize the signal-to-noise ratio of the output. Using an electronic system that implements the optimized nonlinear function, we confirm the recovery of a small signal from a signal with non-Gaussian noise. In our experiment, the non-Gaussian noise is a mixture of Gaussian noises. A similar technique is also applied to the optimization of the threshold value of the function. We find that, for non-Gaussian noise, the response of the optimized nonlinear systems is better than that of the linear system.

Singular response of bistable systems driven by telegraph noise.

We show that weak periodic driving can exponentially strongly change the rate of escape from a potential well of a system driven by telegraph noise. The analysis refers to an overdamped system, where escape requires that the noise amplitude θ exceed a critical value θ(c). For θ close to θ(c), the exponent of the escape rate displays a nonanalytic dependence on the amplitude of an additional low-frequency modulation. This leads to giant nonlinearity of the response of a bistable system to periodic modulation. Also studied is the linear response to periodic modulation far from θ(c). We analyze the scaling of the logarithm of the escape rate with the distance to the saddle-node and pitchfork bifurcation points. The analytical results are in excellent agreement with numerical simulations.

Temperature-modulated synchronization transition in coupled neuronal oscillators.

We study two firing properties to characterize the activities of a neuron: frequency-current (f-I) curves and phase response curves (PRCs), with variation in the intrinsic temperature scaling parameter (µ) controlling the opening and closing of ionic channels. We show a peak of the firing frequency for small µ in a class I neuron with the I value immediately after the saddle-node bifurcation, which is entirely different from previous experimental reports as well as model studies. The PRC takes a type II form on a logarithmic f-I curve when µ is small. Then, we analyze the synchronization phenomena in a two-neuron network using the phase-reduction method. We find common µ-dependent transition and bifurcation of synchronizations, regardless of the values of I. Such results give us helpful insight into synchronizations tuned with a sinusoidal-wave temperature modulation on neurons.

Thouless-Anderson-Palmer equation and self-consistent signal-to-noise analysis for the Hopfield model with three-body interaction.

The self-consistent signal-to-noise analysis (SCSNA) is an alternative to the replica method for deriving the set of order parameter equations for associative memory neural network models and is closely related with the Thouless-Anderson-Palmer equation (TAP) approach. In the recent paper by Shiino and Yamana the Onsager reaction term of the TAP equation has been found to be obtained from the SCSNA for Hopfield neural networks with two-body interaction. We study the TAP equation for an associative memory stochastic analog neural network with three-body interaction to investigate the structure of the Onsager reaction term, in connection with the term proportional to the output characteristic to the SCSNA. We report on the SCSNA framework for analog networks with three-body interactions as well as provide a recipe based on the cavity concept that involves two cavities and the hybrid use of the SCSNA to obtain the TAP equation.