The extraction of information and knowledge from trained neural networks.

In the past, neural networks were viewed as classification and regression systems whose internal representations were incomprehensible. It is now becoming apparent that algorithms can be designed that extract comprehensible representations from trained neural networks, enabling them to be used for data mining and knowledge discovery, that is, the discovery and explanation of previously unknown relationships present in data. This chapter reviews existing algorithms for extracting comprehensible representations from neural networks and outlines research to generalize and extend the capabilities of one of these algorithms, TREPAN. This algorithm has been generalized for application to bioinformatics data sets, including the prediction of splice junctions in human DNA sequences, and cheminformatics. The results generated on these data sets are compared with those generated by a conventional data mining technique (C5) and appropriate conclusions are drawn.

Automatic relevance determination for identifying thalamic regions implicated in schizophrenia.

There have been many theories about and computational models of the schizophrenic disease state. Brain imaging techniques have suggested that abnormalities of the thalamus may contribute to the pathophysiology of schizophrenia. Several studies have found the thalamus to be altered in schizophrenia, and the thalamus has connections with other brain structures implicated in the disorder. This paper describes an experiment examining thalamic levels of the metabolite N-acetylaspartate (NAA), taken from schizophrenics and controls using in vivo proton magnetic resonance spectroscopic imaging. Automatic relevance determination was performed on neural networks trained on this data, identifying NAA group differences in the pulvinar and mediodorsal nucleus, underscoring the importance of examining thalamic subregions in schizophrenia.

Representation and extrapolation in multilayer perceptrons.

To give an adequate explanation of cognition and perform certain practical tasks, connectionist systems must be able to extrapolate. This work explores the relationship between input representation and extrapolation, using simulations of multilayer perceptrons trained to model the identity function. It has been discovered that representation has a marked effect on extrapolation.

Detecting systematic structure in distributed representations.

A novel method of analysing the systematic structure formed inside a feedforward neural network is developed. This method is applied to the task of understanding how a network is performing unification on distributed representations. Systematic structure is detected by examining inter-representational distances, rather than by attempting to detect the vectorial similarities sought by previously described techniques. The method described can be applied to the detection of systematic structure in other networks using distributed representations.