Multistability and fixed-time multisynchronization of switched neural networks with state-dependent switching rules.

This paper presents theoretical results on the multistability and fixed-time synchronization of switched neural networks with multiple almost-periodic solutions and state-dependent switching rules. It is shown herein that the number, location, and stability of the almost-periodic solutions of the switched neural networks can be characterized by making use of the state-space partition. Two sets of sufficient conditions are derived to ascertain the existence of 3n exponentially stable almost-periodic solutions. Subsequently, this paper introduces the novel concept of fixed-time multisynchronization in switched neural networks associated with a range of almost-periodic parameters within multiple stable equilibrium states for the first time. Based on the multistability results, it is demonstrated that there are 3n synchronization manifolds, wherein n is the number of neurons. Additionally, an estimation for the settling time required for drive-response switched neural networks to achieve synchronization is provided. It should be noted that this paper considers stable equilibrium points (static multisynchronization), stable almost-periodic orbits (dynamical multisynchronization), and hybrid stable equilibrium states (hybrid multisynchronization) as special cases of multistability (multisynchronization). Two numerical examples are elaborated to substantiate the theoretical results.

Distributed Algorithms for Linear Equations Over General Directed Networks.

This article deals with linear equations of the form Ax = b . By reformulating the original problem as an unconstrained optimization problem, we first provide a gradient-based distributed continuous-time algorithm over weight-balanced directed graphs, in which each agent only knows partial rows of the augmented matrix (A b) . The algorithm is also applicable to time-varying networks. By estimating a right-eigenvector corresponding to 0 eigenvalue of the out-Laplacian matrix in finite time, we further propose a distributed algorithm over weight-unbalanced communication networks. It is proved that each solution of the designed algorithms converges exponentially to an equilibrium point. Moreover, the convergence rate is given out clearly. For linear equations without solution, these algorithms are used to obtain a least-squares solution in approximate sense. These theoretical results are illustrated by four numerical examples.

Input-to-state stability of delayed memristor-based inertial neural networks via non-reduced order method.

This paper is concerned with the input-to-state stability (ISS) for a kind of delayed memristor-based inertial neural networks (DMINNs). Based on the nonsmooth analysis and stability theory, novel delay-dependent and delay-independent criteria on the ISS of DMINNs are obtained by constructing different Lyapunov functions. Moreover, compared with the reduced order approach used in the previous works, this paper consider the ISS of DMINNs via non-reduced order approach. Directly analysis the model of DMINNs can better maintain its physical backgrounds, which reduces the complexity of calculations and is more rigorous in practical application. Additionally, the novel proposed results on the ISS of DMINNs here incorporate and complement the existing studies on memristive neural network dynamical systems. Lastly, a numerical example is provided to show that the obtained criteria are reliable.

Unified analysis on multistablity of fraction-order multidimensional-valued memristive neural networks.

This article provides a unified analysis of the multistability of fraction-order multidimensional-valued memristive neural networks (FOMVMNNs) with unbounded time-varying delays. Firstly, based on the knowledge of fractional differentiation and memristors, a unified model is established. This model is a unified form of real-valued, complex-valued, and quaternion-valued systems. Then, based on a unified method, the number of equilibrium points for FOMVMNNs is discussed. The sufficient conditions for determining the number of equilibrium points have been obtained. By using 1-norm to construct Lyapunov functions, the unified criteria for multistability of FOMVMNNs are obtained, these criteria are less conservative and easier to verify. Moreover, the attraction basins of the stable equilibrium points are estimated. Finally, two numerical simulation examples are provided to verify the correctness of the results.

Finite-Time Stabilization of Inertial Memristive Neural Networks via Nonreduced Order Method.

This article investigates the finite-time stabilization problem of inertial memristive neural networks (IMNNs) with bounded and unbounded time-varying delays, respectively. To simplify the theoretical derivation, the nonreduced order method is utilized for constructing appropriate comparison functions and designing a discontinuous state feedback controller. Then, based on the controller, the state of IMNNs can directly converge to 0 in finite time. Several criteria for finite-time stabilization of IMNNs are obtained and the setting time is estimated. Compared with previous studies, the requirement of differentiability of time delay is eliminated. Finally, numerical examples illustrate the usefulness of the analysis results in this article.

Neural Network Circuits for Bionic Associative Memory and Temporal Order Memory Based on DNA Strand Displacement.

Pavlovian associative memory plays an important role in our daily life and work. The realization of Pavlovian associative memory at the deoxyribonucleic acid (DNA) molecular level will promote the development of biological computing and broaden the application scenarios of neural networks. In this article, bionic associative memory and temporal order memory circuits are constructed by DNA strand displacement (DSD) reactions. First, a temporal logic gate is constructed on the basis of DSD circuit and extended to a three-input temporal logic gate. The output of temporal logic gate is used for the weight species of associative memory. Second, the forgetting module and output module based on the DSD circuit are constructed to realize some functions of associative memory, including associative memory with simultaneous stimulus, associative memory with interstimulus interval effect, and the facilitation by intermittent stimulus. In addition, the coding, storage, and retrieval modules are designed based on the analysis and memory capabilities of temporal logic gate for temporal information. The temporal order memory circuit is constructed, demonstrating the temporal order memory ability of DNA circuit. Finally, the reliability of the circuit is verified through Visual DSD software simulation. Our work provides ideas and inspiration to construct more complex DNA bionic circuits and intelligent circuits by using DSD technology.

Residual Temporal Convolutional Network With Dual Attention Mechanism for Multilead-Time Interpretable Runoff Forecasting.

As a pivotal subfield within the domain of time series forecasting, runoff forecasting plays a crucial role in water resource management and scheduling. Recent advancements in the application of artificial neural networks (ANNs) and attention mechanisms have markedly enhanced the accuracy of runoff forecasting models. This article introduces an innovative hybrid model, ResTCN-DAM, which synergizes the strengths of deep residual network (ResNet), temporal convolutional networks (TCNs), and dual attention mechanisms (DAMs). The proposed ResTCN-DAM is designed to leverage the unique attributes of these three modules: TCN has outstanding capability to process time series data in parallel. By combining with modified ResNet, multiple TCN layers can be densely stacked to capture more hidden information in the temporal dimension. DAM module adeptly captures the interdependencies within both temporal and feature dimensions, adeptly accentuating relevant time steps/features while diminishing less significant ones with minimal computational cost. Furthermore, the snapshot ensemble method is able to obtain the effect of training multiple models through one single training process, which ensures the accuracy and robustness of the forecasts. The deep integration and collaborative cooperation of these modules comprehensively enhance the model's forecasting capability from various perspectives. Ablation studies conducted validate the efficacy of each module, and through multiple sets of comparative experiments, it is shown that the proposed ResTCN-DAM has exceptional and consistent performance across varying lead times. We also employ visualization techniques to display heatmaps of the model's weights, thereby enhancing the interpretability of the model. When compared with the prevailing neural network-based runoff forecasting models, ResTCN-DAM exhibits state-of-the-art accuracy, temporal robustness, and interpretability, positioning it at the forefront of contemporary research.

Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions.

Activation functions have a significant effect on the dynamics of neural networks (NNs). This study proposes new nonmonotonic wave-type activation functions and examines the complete stability of delayed recurrent NNs (DRNNs) with these activation functions. Using the geometrical properties of the wave-type activation function and subsequent iteration scheme, sufficient conditions are provided to ensure that a DRNN with n neurons has exactly (2m + 3)n equilibria, where (m + 2)n equilibria are locally exponentially stable, the remainder (2m + 3)n - (m + 2)n equilibria are unstable, and a positive integer m is related to wave-type activation functions. Furthermore, the DRNN with the proposed activation function is completely stable. Compared with the previous literature, the total number of equilibria and the stable equilibria significantly increase, thereby enhancing the memory storage capacity of DRNN. Finally, several examples are presented to demonstrate our proposed results.

Operant Conditioning Neuromorphic Circuit With Addictiveness and Time Memory for Automatic Learning.

Most operant conditioning circuits predominantly focus on simple feedback process, few studies consider the intricacies of feedback outcomes and the uncertainty of feedback time. This paper proposes a neuromorphic circuit based on operant conditioning with addictiveness and time memory for automatic learning. The circuit is mainly composed of hunger output module, neuron module, excitement output module, memristor-based decision module, and memory and feedback generation module. In the circuit, the process of output excitement and addiction in stochastic feedback is achieved. The memory of interval between the two rewards is formed. The circuit can adapt to complex scenarios with these functions. In addition, hunger and satiety are introduced to realize the interaction between biological behavior and exploration desire, which enables the circuit to continuously reshape its memories and actions. The process of operant conditioning theory for automatic learning is accomplished. The study of operant conditioning can serve as a reference for more intelligent brain-inspired neural systems.

Self-paced regularized adaptive multi-view unsupervised feature selection.

Multi-view unsupervised feature selection (MUFS) is an efficient approach for dimensional reduction of heterogeneous data. However, existing MUFS approaches mostly assign the samples the same weight, thus the diversity of samples is not utilized efficiently. Additionally, due to the presence of various regularizations, the resulting MUFS problems are often non-convex, making it difficult to find the optimal solutions. To address this issue, a novel MUFS method named Self-paced Regularized Adaptive Multi-view Unsupervised Feature Selection (SPAMUFS) is proposed. Specifically, the proposed approach firstly trains the MUFS model with simple samples, and gradually learns complex samples by using self-paced regularizer. l2,p-norm (0

A memristive all-inclusive hypernetwork for parallel analog deployment of full search space architectures.

In recent years, there has been a significant advancement in memristor-based neural networks, positioning them as a pivotal processing-in-memory deployment architecture for a wide array of deep learning applications. Within this realm of progress, the emerging parallel analog memristive platforms are prominent for their ability to generate multiple feature maps in a single processing cycle. However, a notable limitation is that they are specifically tailored for neural networks with fixed structures. As an orthogonal direction, recent research reveals that neural architecture should be specialized for tasks and deployment platforms. Building upon this, the neural architecture search (NAS) methods effectively explore promising architectures in a large design space. However, these NAS-based architectures are generally heterogeneous and diversified, making it challenging for deployment on current single-prototype, customized, parallel analog memristive hardware circuits. Therefore, investigating memristive analog deployment that overrides the full search space is a promising and challenging problem. Inspired by this, and beginning with the DARTS search space, we study the memristive hardware design of primitive operations and propose the memristive all-inclusive hypernetwork that covers 2×1025 network architectures. Our computational simulation results on 3 representative architectures (DARTS-V1, DARTS-V2, PDARTS) show that our memristive all-inclusive hypernetwork achieves promising results on the CIFAR10 dataset (89.2% of PDARTS with 8-bit quantization precision), and is compatible with all architectures in the DARTS full-space. The hardware performance simulation indicates that the memristive all-inclusive hypernetwork costs slightly more resource consumption (nearly the same in power, 22%∼25% increase in Latency, 1.5× in Area) relative to the individual deployment, which is reasonable and may reach a tolerable trade-off deployment scheme for industrial scenarios.

Fault-Tolerant Synchronization for Memristive Neural Networks With Multiple Actuator Failures.

By using the fault-tolerant control method, the synchronization of memristive neural networks (MNNs) subjected to multiple actuator failures is investigated in this article. The considered actuator failures include the effectiveness failure and the lock-in-place failure, which are different from previous results. First of all, the mathematical expression of the control inputs in the considered system is given by introducing the models of the above two types of actuator failures. Following, two classes of synchronization strategies, which are state feedback control strategies and event-triggered control strategies, are proposed by using some inequality techniques and Lyapunov stability theories. The designed controllers can, respectively, guarantee the realization of synchronizations of the global exponential, the finite-time and the fixed-time for the MNNs by selecting different parameter conditions. Then the estimations of settling times of provided synchronization schemes are computed and the Zeno phenomenon of proposed event-triggered strategies is explicitly excluded. Finally, two experiments are conducted to confirm the availability of given synchronization strategies.

Pruning and quantization algorithm with applications in memristor-based convolutional neural network.

The human brain's ultra-low power consumption and highly parallel computational capabilities can be accomplished by memristor-based convolutional neural networks. However, with the rapid development of memristor-based convolutional neural networks in various fields, more complex applications and heavier computations lead to the need for a large number of memristors, which makes power consumption increase significantly and the network model larger. To mitigate this problem, this paper proposes an SBT-memristor-based convolutional neural network architecture and a hybrid optimization method combining pruning and quantization. Firstly, SBT-memristor-based convolutional neural network is constructed by using the good thresholding property of the SBT memristor. The memristive in-memory computing unit, activation unit and max-pooling unit are designed. Then, the hybrid optimization method combining pruning and quantization is used to improve the SBT-memristor-based convolutional neural network architecture. This hybrid method can simplify the memristor-based neural network and represent the weights at the memristive synapses better. Finally, the results show that the SBT-memristor-based convolutional neural network reduces a large number of memristors, decreases the power consumption and compresses the network model at the expense of a little precision loss. The SBT-memristor-based convolutional neural network obtains faster recognition speed and lower power consumption in MNIST recognition. It provides new insights for the complex application of convolutional neural networks.

Fixed-time synchronization of complex-valued neural networks for image protection and 3D point cloud information protection.

This paper studies the fixed-time synchronization (FDTS) of complex-valued neural networks (CVNNs) based on quantized intermittent control (QIC) and applies it to image protection and 3D point cloud information protection. A new controller was designed which achieved FDTS of the CVNNs, with the estimation of the convergence time not dependent on the initial state. Our approach divides the neural network into two real-valued systems and then combines the framework of the Lyapunov method to give criteria for FDTS. Applying synchronization to image protection, the image will be encrypted with a drive system sequence and decrypted with a response system sequence. The quality of image encryption and decryption depends on the synchronization error. Meanwhile, the depth image of the object is encrypted and then the 3D point cloud is reconstructed based on the decrypted depth image. This means that the 3D point cloud information is protected. Finally, simulation examples verify the efficacy of the controller and the synchronization criterion, giving results for applications in image protection and 3D point cloud information protection.

Saturation function-based continuous control on fixed-time synchronization of competitive neural networks.

Currently, through proposing discontinuous control strategies with the signum function and discussing separately short-term memory (STM) and long-term memory (LTM) of competitive artificial neural networks (ANNs), the fixed-time (FXT) synchronization of competitive ANNs has been explored. Note that the method of separate analysis usually leads to complicated theoretical derivation and synchronization conditions, and the signum function inevitably causes the chattering to reduce the performance of the control schemes. To try to solve these challenging problems, the FXT synchronization issue is concerned in this paper for competitive ANNs by establishing a theorem of FXT stability with switching type and developing continuous control schemes based on a kind of saturation functions. Firstly, different from the traditional method of studying separately STM and LTM of competitive ANNs, the models of STM and LTM are compressed into a high-dimensional system so as to reduce the complexity of theoretical analysis. Additionally, as an important theoretical preliminary, a FXT stability theorem with switching differential conditions is established and some high-precision estimates for the convergence time are explicitly presented by means of several special functions. To achieve FXT synchronization of the addressed competitive ANNs, a type of continuous pure power-law control scheme is developed via introducing the saturation function instead of the signum function, and some synchronization criteria are further derived by the established FXT stability theorem. These theoretical results are further illustrated lastly via a numerical example and are applied to image encryption.

A Distributed k-Winners-Take-All Model With Binary Consensus Protocols.

This article concentrates on solving the k -winners-take-all (k WTA) problem with large-scale inputs in a distributed setting. We propose a multiagent system with a relatively simple structure, in which each agent is equipped with a 1-D system and interacts with others via binary consensus protocols. That is, only the signs of the relative state information between neighbors are required. By virtue of differential inclusion theory, we prove that the system converges from arbitrary initial states. In addition, we derive the convergence rate as O(1/t) . Furthermore, in comparison to the existing models, we introduce a novel comparison filter to eliminate the resolution ratio requirement on the input signal, that is, the difference between the k th and (k+1) th largest inputs must be larger than a positive threshold. As a result, the proposed distributed k WTA model is capable of solving the k WTA problem, even when more than two elements of the input signal share the same value. Finally, we validate the effectiveness of the theoretical results through two simulation examples.

Multistability and Robustness of Competitive Neural Networks With Time-Varying Delays.

This article is devoted to analyzing the multistability and robustness of competitive neural networks (NNs) with time-varying delays. Based on the geometrical structure of activation functions, some sufficient conditions are proposed to ascertain the coexistence of ∏i=1n(2Ri+1) equilibrium points, ∏i=1n(Ri+1) of them are locally exponentially stable, where n represents a dimension of system and Ri is the parameter related to activation functions. The derived stability results not only involve exponential stability but also include power stability and logarithmical stability. In addition, the robustness of ∏i=1n(Ri+1) stable equilibrium points is discussed in the presence of perturbations. Compared with previous papers, the conclusions proposed in this article are easy to verify and enrich the existing stability theories of competitive NNs. Finally, numerical examples are provided to support theoretical results.

Synchronization of coupled switched neural networks subject to hybrid stochastic disturbances.

In this paper, the theoretical analysis on exponential synchronization of a class of coupled switched neural networks suffering from stochastic disturbances and impulses is presented. A control law is developed and two sets of sufficient conditions are derived for the synchronization of coupled switched neural networks. First, for desynchronizing stochastic impulses, the synchronization of coupled switched neural networks is analyzed by Lyapunov function method, the comparison principle and a impulsive delay differential inequality. Then, for general stochastic impulses, by partitioning impulse interval and using the convex combination technique, a set of sufficient condition on the basis of linear matrix inequalities (LMIs) is derived for the synchronization of coupled switched neural networks. Eventually, two numerical examples and a practical application are elaborated to illustrate the effectiveness of the theoretical results.

AutoGMap: Learning to Map Large-Scale Sparse Graphs on Memristive Crossbars.

The sparse representation of graphs has shown great potential for accelerating the computation of graph applications (e.g., social networks and knowledge graphs) on traditional computing architectures (CPU, GPU, or TPU). But, the exploration of large-scale sparse graph computing on processing-in-memory (PIM) platforms (typically with memristive crossbars) is still in its infancy. To implement the computation or storage of large-scale or batch graphs on memristive crossbars, a natural assumption is that a large-scale crossbar is demanded, but with low utilization. Some recent works question this assumption; to avoid the waste of storage and computational resource, the fixed-size or progressively scheduled "block partition" schemes are proposed. However, these methods are coarse-grained or static and are not effectively sparsity-aware. This work proposes the dynamic sparsity-aware mapping scheme generating method that models the problem with a sequential decision-making model, and optimizes it by reinforcement learning (RL) algorithm (REINFORCE). Our generating model long short-term memory (LSTM), combined with the dynamic-fill scheme generates remarkable mapping performance on the small-scale graph/matrix data (complete mapping costs 43% area of the original matrix) and two large-scale matrix data (costing 22.5% area on qh882 and 17.1% area on qh1484). Our method may be extended to sparse graph computing on other PIM architectures, not limited to the memristive device-based platforms.

Generalized-Type Multistability of Almost Periodic Solutions for Memristive Cohen-Grossberg Neural Networks.

This article investigates a generalized type of multistability about almost periodic solutions for memristive Cohen-Grossberg neural networks (MCGNNs). As the inevitable disturbances in biological neurons, almost periodic solutions are more common in nature than equilibrium points (EPs). They are also generalizations of EPs in mathematics. According to the concepts of almost periodic solutions and Ψ -type stability, this article presents a generalized-type multistability definition of almost periodic solutions. The results show that (K+1)n generalized stable almost periodic solutions can coexist in a MCGNN with n neurons, where K is a parameter of the activation functions. The enlarged attraction basins are also estimated based on the original state space partition method. Some comparisons and convincing simulations are given to verify the theoretical results at the end of this article.