Artificial neural networks: opening the black box.

Artificial neural networks now are used in many fields. They have become well established as viable, multipurpose, robust computational methodologies with solid theoretic support and with strong potential to be effective in any discipline, especially medicine. For example, neural networks can extract new medical information from raw data, build computer models that are useful for medical decision-making, and aid in the distribution of medical expertise. Because many important neural network applications currently are emerging, the authors have prepared this article to bring a clearer understanding of these biologically inspired computing paradigms to anyone interested in exploring their use in medicine. They discuss the historical development of neural networks and provide the basic operational mathematics for the popular multilayered perceptron. The authors also describe good training, validation, and testing techniques, and discuss measurements of performance and reliability, including the use of bootstrap methods to obtain confidence intervals. Because it is possible to predict outcomes for individual patients with a neural network, the authors discuss the paradigm shift that is taking place from previous "bin-model" approaches, in which patient outcome and management is assumed from the statistical groups in which the patient fits. The authors explain that with neural networks it is possible to mediate predictions for individual patients with prevalence and misclassification cost considerations using receiver operating characteristic methodology. The authors illustrate their findings with examples that include prostate carcinoma detection, coronary heart disease risk prediction, and medication dosing. The authors identify and discuss obstacles to success, including the need for expanded databases and the need to establish multidisciplinary teams. The authors believe that these obstacles can be overcome and that neural networks have a very important role in future medical decision support and the patient management systems employed in routine medical practice.

Synchrony detection in neural assemblies.

The identification of synchronously active neural assemblies in simultaneous recordings of neuron activities is an important research issue and a difficult algorithmic problem. A gravitational analysis method has been developed to detect and identify groups of neurons that tend to generate action potentials in near-synchrony from among a larger population of simultaneously recorded units. In this paper, an improved algorithm is used for the gravitational clustering method and its performance is characterized. Whereas the original algorithm ran in n3 time (n = the number of neurons), the new algorithm runs in n2 time. Neurons are represented as particles in n-space that 'gravitate' towards one another whenever near-synchronous electrical activity occurs. Ensembles of neurons that tend to fire together then become clustered together. The gravitational technique not only identifies the synchronous groups present but also shows graphically the changing activity patterns and changing synchronies.

Cytoskeletal logic: a model for molecular computation via Boolean operations in microtubules and microtubule-associated proteins.

Adaptive behaviors and dynamic activities within living cells are organized by the cytoskeleton: intracellular networks of interconnected protein polymers which include microtubules (MTs), actin, intermediate filaments, microtubule associated proteins (MAPs) and other protein structures. Cooperative interactions among cytoskeletal protein subunit conformational states have been used to model signal transmission and information processing. In the present work we present a theoretical model for molecular computing in which Boolean logic is implemented in parallel networks of individual MTs interconnected by MAPs. Conformational signals propagate on MTs as in data buses and in the model MAPs are considered as Boolean operators, either as bit-lines (like MTs) where a signal can be transported unchanged between MTs ('BUS-MAP'), or as bit-lines where a Boolean operation is performed in one of the two MAP-MT attachments ('LOGIC-MAP'). Three logic MAPs have been defined ('NOT-MAP, 'AND-MAP', 'XOR-MAP') and used to demonstrate addition, subtraction and other arithmetic operations. Although our choice of Boolean logic is arbitrary, the simulations demonstrate symbolic manipulation in a connectionist system and suggest that MT-MAP networks can perform computation in living cells and are candidates for future molecular computing devices.

Toward modeling a dynamic biological neural network.

Mammalian macular endorgans are linear bioaccelerometers located in the vestibular membranous labyrinth of the inner ear. In this paper, the organization of the endorgan is interpreted on physical and engineering principles. This is a necessary prerequisite to mathematical and symbolic modeling of information processing by the macular neural network. Mathematical notations that describe the functioning system were used to produce a novel, symbolic model. The model is six-tiered and is constructed to mimic the neural system. Initial simulations show that the network functions best when some of the detecting elements (type I hair cells) are excitatory and others (type II hair cells) are weakly inhibitory. The simulations also illustrate the importance of disinhibition of receptors located in the third tier in shaping nerve discharge patterns at the sixth tier in the model system.

Cooperative firing activity in simultaneously recorded populations of neurons: detection and measurement.

Recent advances in techniques for chronic recording from multiple extracellular microelectrodes allow simultaneous observation of firings of substantial populations of neurons. We describe a new conceptual representation of cooperative behavior within the observed neuronal population. This representation leads to a new technique for detecting and studying functional neuronal assemblies that are characterized by temporally related firing patterns. The representation may be applied to both dynamic and long-term aspects of cooperativity. The basic idea is to map activity of neurons into motions of particles in a multidimensional Euclidean space. Each neuron is represented by a point particle located in this space. In the simplest version of the mapping, each nerve impulse results in an increment in a "charge" associated with that particle; between firings the charges decay. The force exerted by any such particle on any other is, by analogy with some physical forces, proportional to the product of their charges and may depend on the Euclidean distance separating them. The force on a particle directly affects its velocity rather than its acceleration, as with actual particles in a viscous medium. These forces result in aggregation of those particles that correspond to neurons tending to fire together; separate clusters represent independent cooperative groups. Modification of the charges and forces permits inclusion of inhibitory interactions. Identification, measurement, and display of the resulting clusters can be performed with any of a number of algorithms. We illustrate the application of this approach to populations of computer-simulated neurons having both direct and indirect excitatory coupling.

Favored patterns in spike trains. I. Detection.

Traditional spike-train analysis methods cannot identify patterns of firing that occur frequently but at arbitrary times. It is appropriate to search for recurring patterns because such patterns could be used for information transfer. In this paper, we present two methods for identifying "favored patterns" --patterns that occur more often than is reasonably expected at random. The quantized Monte Carlo method identifies and establishes significance for favored patterns whose detailed timing may vary but that do not have extra or missing spikes. The template method identifies favored patterns whose occurrences may have extra or missing spikes. This method is useful when employed after the results of the first method are known. Studies with simulated spike trains containing known interpolated patterns are used to establish the sensitivity and accuracy of the quantized Monte Carlo method. Certain trends with regard to parameters of the detected patterns and of the analysis methods are described. Application of these methods to neurophysiological data has shown that a large proportion of spike trains have favored patterns. These findings are described in the accompanying paper (3).

Favored patterns in spike trains. II. Application.

In this paper we apply the two methods described in the companion paper (4) to experimentally recorded spike trains from two preparations, the crayfish claw and the cat striate cortex. Neurons in the crayfish claw control system produced favored patterns in 23 of 30 spike trains under a variety of experimental conditions. Favored patterns generally consisted of 3-7 spikes and were found to be in excess by both quantized and template methods. Spike trains from area 17 of the lightly anesthetized cat showed favored patterns in 16 of 27 cases (in quantized form). Some patterns were also found to be favored in template form; these were not as abundant in the cat data as in the crayfish data. Most firing of the cat neurons occurred at times near stimulation, and the observed patterns may represent stimulus information. Favored patterns generally contained up to 7 spikes. No obvious correlations between identified neurons or experimental conditions and the generation of favored patterns were apparent from these data in either preparation. This work adds to the existing evidence that pattern codes are available for use by the nervous system. The potential biological significance of pattern codes is discussed.