A two-level hamming network for high performance associative memory.

This paper presents an analysis of a two-level decoupled Hamming network, which is a high performance discrete-time/discrete-state associative memory model. The two-level Hamming memory generalizes the Hamming memory by providing for local Hamming distance computations in the first level and a voting mechanism in the second level. In this paper, we study the effect of system dimension, window size, and noise on the capacity and error correction capability of the two-level Hamming memory. Simulation results are given for both random images and human face images.

Recurrent neural nets as dynamical Boolean systems with application to associative memory.

Discrete-time/discrete-state recurrent neural networks are analyzed from a dynamical Boolean systems point of view in order to devise new analytic and design methods for the class of both single and multilayer recurrent artificial neural networks. With the proposed dynamical Boolean systems analysis, we are able to formulate necessary and sufficient conditions for network stability which are more general than the well-known but restrictive conditions for the class of single layer networks: (1) symmetric weight matrix with (2) positive diagonal and (3) asynchronous update. In terms of design, we use a dynamical Boolean systems analysis to construct a high performance associative memory. With this Boolean memory, we can guarantee that all fundamental memories are stored, and also guarantee the size of the basin of attraction for each fundamental memory.

A coupled gradient network approach for static and temporal mixed-integer optimization.

Utilizes the ideas of artificial neural networks to propose new solution methods for a class of constrained mixed-integer optimization problems. These new solution methods are more suitable to parallel implementation than the usual sequential methods of mathematical programming. Another attractive feature of the proposed approach is that some global search mechanisms may be easily incorporated into the computation, producing results which are more globally optimal. To formulate the solution method proposed in this paper, a penalty function approach is used to define a coupled gradient-type network with an appropriate architecture, energy function and dynamics such that high-quality solutions may be obtained upon convergence of the dynamics. Finally, it is shown how the coupled gradient net may be extended to handle temporal mixed-integer optimization problems, and simulations are presented which demonstrate the effectiveness of the approach.

Neurocontrollers trained with rules extracted by a genetic assisted reinforcement learning system.

This paper proposes a novel system for rule extraction of temporal control problems and presents a new way of designing neurocontrollers. The system employs a hybrid genetic search and reinforcement learning strategy for extracting the rules. The learning strategy requires no supervision and no reference model. The extracted rules are weighted micro rules that operate on small neighborhoods of the admissable control space. A further refinement of the extracted rules is achieved by applying additional genetic search and reinforcement to reduce the number of extracted micro rules. This process results in a smaller set of macro rules which can be used to train a feedforward multilayer perceptron neurocontroller. The micro rules or the macro rules may also be utilized directly in a table look-up controller. As an example of the macro rules-based neurocontroller, we chose four benchmarks. In the first application we verify the capability of our system to learn optimal linear control strategies. The other three applications involve engine idle speed control, bioreactor control, and stabilizing two poles on a moving cart. These problems are highly nonlinear, unstable, and may include noise and delays in the plant dynamics. In terms of retrievals; the neurocontrollers generally outperform the controllers using a table look-up method. Both controllers, though, show robustness against noise disturbances and plant parameter variations.

NNERVE: neural network extraction of repetitive vectors for electromyography--Part I: Algorithm.

Artificial neural network (ANN) based signal processing methods have been shown to have significant robustness in processing complex, degraded, noisy, and unstable signals. A novel approach to automated electromyogram (EMG) signal decomposition, using an ANN processing architecture, is presented in this paper. Due to the lack of a priori knowledge of motor unit action potential (MUAP) morphology, the EMG decomposition must be performed in an unsupervised manner. An ANN classifier, consisting of a multilayer perceptron neural network and employing a novel unsupervised training strategy, is proposed. The ANN learns repetitive appearances of MUAP waveforms from their suspected occurrences in a filtered EMG signal in an autoassociative learning task. The same training waveforms are fed into the trained ANN and the output of the ANN is fed back to its input, giving rise to a dynamic retrieval net classifier. For each waveform in the data, the network discovers a feature vector associated with that waveform. For each waveform, classification is achieved by comparing its feature vector with those of the other waveforms. Firing information of each MUAP is further used to refine the classification results of the ANN classifier. Then, individual MUAP waveform shapes are derived and their firing tables are created.

NNERVE: neural network extraction of repetitive vectors for electromyography--Part II: Performance analysis.

We have presented a new method for the decomposition of clinical electromyographic signals, NNERVE, which utilizes a novel "pseudo-unsupervised" neural network approach to signal decomposition. In this paper, we present a detailed performance analysis. We present definitions for quantitative performance criteria. NNERVE is shown to be highly reliable over a wide range of neural network architectures. It is also minimally sensitive to learning parameters. The degradations of performance over a wide range of signals and parameters are shown to be gradual, slight and graceful. These characteristics are shown to translate directly into a high degree of robustness over widely varying signals. Real signals obtained from the entire range of patients encountered in clinical situations are shown to be correctly handled without any modifications or adjustments of any parameters. This neural network method is then directly compared to a prior traditional signal processing method and is shown quantitatively to have consistently superior performance on both simulated and real signals. Clinically acceptable performance over a wide range of signals, recorded using standard clinical methodology, and the lack of a need for user interaction, will facilitate the use of motor unit quantitation in routine clinical electromyography.

Quantitative computer analysis of the sounds of isolated motor unit action potentials.

To develop quantitative measures of motor unit action potential (MUAP) sounds, we correlated Fourier domain features of isolated, individual MUAPs with classic time domain measurements. There were moderate correlations between amplitude or duration measurements, and absolute low-frequency (39 to 234 Hz) energy and amplitude of the peak frequency, and high correlations between the composite time domain feature of amplitude x duration, and total energy, amplitude of the peak frequency, and absolute low-frequency energy. Polyphasic potentials have multiple peaks in the magnitude component of the Fourier transform. Phase information appears to convey the "crisp" sound of MUAPs close to the recording electrode. The clinical description of "large" or "small" MUAPs by sound is likely based on absolute low-frequency energy, and incorporates both amplitude and duration information. We conclude that features of isolated MUAPs may be analyzed in the Fourier domain, and that they correlate closely with traditionally used measures of known diagnostic significance. The sound of the EMG used by clinical electromyographers is amenable to quantitative analysis.

Adaptive Ho-Kashyap rules for perceptron training.

Three adaptive versions of the Ho-Kashyap perceptron training algorithm are derived based on gradient descent strategies. These adaptive Ho-Kashyap (AHK) training rules are comparable in their complexity to the LMS and perceptron training rules and are capable of adaptively forming linear discriminant surfaces that guarantee linear separability and of positioning such surfaces for maximal classification robustness. In particular, a derived version called AHK II is capable of adaptively identifying critical input vectors lying close to class boundaries in linearly separable problems. The authors extend this algorithm as AHK III, which adds the capability of fast convergence to linear discriminant surfaces which are good approximations for nonlinearly separable problems. This is achieved by a simple built-in unsupervised strategy which allows for the adaptive grading and discarding of input vectors causing nonseparability. Performance comparisons with LMS and perceptron training are presented.

Exact associative neural memory dynamics utilizing Boolean matrices.

The exact dynamics of shallow loaded associative neural memories are generated and characterized. The Boolean matrix analysis approach is employed for the efficient generation of all possible state transition trajectories for parallel updated binary-state dynamic associative memories (DAMs). General expressions for the size of the basin of attraction of fundamental and oscillatory memories and the number of oscillatory and stable states are derived for discrete synchronous Hopfield DAMs loaded with one, two, or three even-dimensionality bipolar memory vectors having the same mutual Hamming distances between them. Spurious memories are shown to occur only if the number of stored patterns exceeds two in an even-dimensionality Hopfield memory. The effects of odd- versus even-dimensionality memory vectors on DAM dynamics and the effects of memory pattern encoding on DAM performance are tested. An extension of the Boolean matrix dynamics characterization technique to other, more complex DAMs is presented.

Optical threshold logic elements for digital computation.

A hybrid integrated optical threshold logic element is proposed for fast digital computation on a chip level. Each input and output optical beam occupies a separate channel and is coupled directly to the source. This suggests the possibility of achieving reliable optical digital operation in a complex network. Such a network is expected to be particularly compact because one or several threshold logic elements may replace many simple optical logic elements. The devices are also capable of high-speed programmability, so the same set of elements can serve different functions. Applications to conventional arithmetic computation and to high-speed residue arithmetic computation are presented.