Solving the linear interval tolerance problem for weight initialization of neural networks.

Determining good initial conditions for an algorithm used to train a neural network is considered a parameter estimation problem dealing with uncertainty about the initial weights. Interval analysis approaches model uncertainty in parameter estimation problems using intervals and formulating tolerance problems. Solving a tolerance problem is defining lower and upper bounds of the intervals so that the system functionality is guaranteed within predefined limits. The aim of this paper is to show how the problem of determining the initial weight intervals of a neural network can be defined in terms of solving a linear interval tolerance problem. The proposed linear interval tolerance approach copes with uncertainty about the initial weights without any previous knowledge or specific assumptions on the input data as required by approaches such as fuzzy sets or rough sets. The proposed method is tested on a number of well known benchmarks for neural networks trained with the back-propagation family of algorithms. Its efficiency is evaluated with regards to standard performance measures and the results obtained are compared against results of a number of well known and established initialization methods. These results provide credible evidence that the proposed method outperforms classical weight initialization methods.

Efficient information theoretic strategies for classifier combination, feature extraction and performance evaluation in improving false positives and false negatives for spam e-mail filtering.

Spam emails are considered as a serious privacy-related violation, besides being a costly, unsolicited communication. Various spam filtering techniques have been so far proposed, mainly based on Naïve Bayesian algorithms. Other Machine Learning algorithms like Boosting trees, or Support Vector Machines (SVM) have already been used with success. However, the number of False Positives (FP) and False Negatives (FN) resulting through applying various spam e-mail filters still remains too high and the problem of spam e-mail categorization cannot be solved completely from a practical viewpoint. In this paper, we propose a novel approach for spam e-mail filtering based on efficient information theoretic techniques for integrating classifiers, for extracting improved features and for properly evaluating categorization accuracy in terms of FP and FN. The goal of the presented methodology is to empirically but explicitly minimize these FP and FN numbers by combining high-performance FP filters with high-performance FN filters emerging from a previous work of the authors [Zorkadis, V., Panayotou, M., & Karras, D. A. (2005). Improved spam e-mail filtering based on committee machines and information theoretic feature extraction. Proceedings of the International Joint Conference on Neural Networks, July 31-August 4, 2005, Montreal, Canada]. To this end, Random Committee-based filters along with ADTree-based ones are efficiently combined through information theory, respectively. The experiments conducted are of the most extensive ones so far in the literature, exploiting widely accepted benchmarking e-mail data sets and comparing the proposed methodology with the Naive Bayes spam filter as well as with the Boosting tree methodology, the classification via regression and other machine learning models. It is illustrated by means of novel information theoretic measures of FP & FN filtering performance that the proposed approach is very favorably compared to the other rival methods. Finally, it is found that the proposed information theoretic Boolean features present a remarkably high spam categorization performance.

On neural network techniques in the secure management of communication systems through improving and quality assessing pseudorandom stream generators.

Random components play an especially important role in the management of secure communication systems, with emphasis on the key management of cryptographic protocols. For this reason, the existence of strong pseudo random number generators is highly required. This paper presents novel techniques, which rely on Artificial Neural Network (ANN) architectures, to strengthen traditional generators such as IDEA and ANSI X.9 based on 3DES and IDEA. Additionally, this paper proposes a non-linear test method for the quality assessment of the required non-predictability property, which relies on feedforward neural networks. This non-predictability test method along with commonly used empirical tests based on statistics is proposed as a methodology for quality assessing strong pseudorandom stream generators. By means of this methodology, traditional and Neural Network based pseudorandom stream generators are evaluated. The results show that the proposed generators behave significantly better than the traditional ones, in particular, in terms of non-predictability.

CoLD: a versatile detection system for colorectal lesions in endoscopy video-frames.

In this paper, we present CoLD (colorectal lesions detector) an innovative detection system to support colorectal cancer diagnosis and detection of pre-cancerous polyps, by processing endoscopy images or video frame sequences acquired during colonoscopy. It utilizes second-order statistical features that are calculated on the wavelet transformation of each image to discriminate amongst regions of normal or abnormal tissue. An artificial neural network performs the classification of the features. CoLD integrates the feature extraction and classification algorithms under a graphical user interface, which allows both novice and expert users to utilize effectively all system's functions. It has been developed in close cooperation with gastroenterology specialists and has been tested on various colonoscopy videos. The detection accuracy of the proposed system has been estimated to be more than 95%. As it has been resulted, it can be used as a supplementary diagnostic tool for colorectal lesions.

Improved statistical classification methods in computerized psychiatric diagnosis.

BACKGROUND: Mainstream psychiatric diagnosis involves mainly sequential, expert-system-derived, logical decision rules. Among the few statistical classification methods that have been sporadically evaluated are Bayes, k-nearest neighbor, and discriminant analysis classifiers. METHODS: A statistical classification method based on artificial neural networks (ANN) with task-specific constrained architectures was applied to a sample of 796 clinical interviews, where the symptom evaluation and the diagnostic judgments were made using the Psychiatric State Examination (PSE) system. The proposed constrained ANN (CANN) method was compared with other statistical classification methods. RESULTS: CANN was found to be superior to all other considered methods, having an overall "correct" classification rate of 80% when applied to test data. Similarly, the concordance coefficients of agreement with the PSE diagnostic categories were all very high. Among the other used methods, discriminant analysis had slightly inferior performance but better generalization capability. CONCLUSIONS: The proposed CANN method has a definite utility in psychiatric diagnosis and requires further evaluation, perhaps alongside other standard classification systems and/or with larger samples.

An efficient constrained training algorithm for feedforward networks.

A novel algorithm is presented which supplements the training phase in feedforward networks with various forms of information about desired learning properties. This information is represented by conditions which must be satisfied in addition to the demand for minimization of the usual mean square error cost function. The purpose of these conditions is to improve convergence, learning speed, and generalization properties through prompt activation of the hidden units, optimal alignment of successive weight vector offsets, elimination of excessive hidden nodes, and regulation of the magnitude of search steps in the weight space. The algorithm is applied to several small- and large-scale binary benchmark training tasks, to test its convergence ability and learning speed, as well as to a large-scale OCR problem, to test its generalization capability. Its performance in terms of percentage of local minima, learning speed, and generalization ability is evaluated and found superior to the performance of the backpropagation algorithm and variants thereof taking especially into account the statistical significance of the results.