An analysis of training and generalization errors in shallow and deep networks.

This paper is motivated by an open problem around deep networks, namely, the apparent absence of over-fitting despite large over-parametrization which allows perfect fitting of the training data. In this paper, we analyze this phenomenon in the case of regression problems when each unit evaluates a periodic activation function. We argue that the minimal expected value of the square loss is inappropriate to measure the generalization error in approximation of compositional functions in order to take full advantage of the compositional structure. Instead, we measure the generalization error in the sense of maximum loss, and sometimes, as a pointwise error. We give estimates on exactly how many parameters ensure both zero training error as well as a good generalization error. We prove that a solution of a regularization problem is guaranteed to yield a good training error as well as a good generalization error and estimate how much error to expect at which test data.

Serology and longevity of immunity against Echinococcus granulosus in sheep and llama induced by an oil-based EG95 vaccine.

An oil-based formulation of the EG95 vaccine to protect grazing animals against infection with Echinococcus granulosus was formulated in Argentina. The efficacy of the vaccine was monitored by serology in sheep and llama (Lama glama) and was compared to the serology in sheep previously published using a QuilA-adjuvanted vaccine. Long-term efficacy was also tested in sheep by challenging with E. granulosus eggs of the G1 strain 4 years after the beginning of the trial. The serological results for both sheep and llama were similar to those described previously, except that there was a more rapid response after the first vaccination. A third vaccination given after 1 year resulted in a transient boost in serology that lasted for about 12 months, which was similar to results previously described. Sheep challenged after 4 years with three vaccinations presented 84·2% reduction of live cysts counts compared with control group, and after a fourth vaccination prior to challenge, this reduction was 94·7%. The oil-based vaccine appeared to be bio-equivalent to the QuilA vaccine.

Multiclass cancer diagnosis using tumor gene expression signatures.

The optimal treatment of patients with cancer depends on establishing accurate diagnoses by using a complex combination of clinical and histopathological data. In some instances, this task is difficult or impossible because of atypical clinical presentation or histopathology. To determine whether the diagnosis of multiple common adult malignancies could be achieved purely by molecular classification, we subjected 218 tumor samples, spanning 14 common tumor types, and 90 normal tissue samples to oligonucleotide microarray gene expression analysis. The expression levels of 16,063 genes and expressed sequence tags were used to evaluate the accuracy of a multiclass classifier based on a support vector machine algorithm. Overall classification accuracy was 78%, far exceeding the accuracy of random classification (9%). Poorly differentiated cancers resulted in low-confidence predictions and could not be accurately classified according to their tissue of origin, indicating that they are molecularly distinct entities with dramatically different gene expression patterns compared with their well differentiated counterparts. Taken together, these results demonstrate the feasibility of accurate, multiclass molecular cancer classification and suggest a strategy for future clinical implementation of molecular cancer diagnostics.

Categorical representation of visual stimuli in the primate prefrontal cortex.

The ability to group stimuli into meaningful categories is a fundamental cognitive process. To explore its neural basis, we trained monkeys to categorize computer-generated stimuli as "cats" and "dogs." A morphing system was used to systematically vary stimulus shape and precisely define the category boundary. Neural activity in the lateral prefrontal cortex reflected the category of visual stimuli, even when a monkey was retrained with the stimuli assigned to new categories.

Models of object recognition.

Understanding how biological visual systems recognize objects is one of the ultimate goals in computational neuroscience. From the computational viewpoint of learning, different recognition tasks, such as categorization and identification, are similar, representing different trade-offs between specificity and invariance. Thus, the different tasks do not require different classes of models. We briefly review some recent trends in computational vision and then focus on feedforward, view-based models that are supported by psychophysical and physiological data.

Learning in brains and machines.

The problem of learning is arguably at the very core of the problem of intelligence, both biological and artificial. In this paper we sketch some of our work over the last ten years in the area of supervised learning, focusing on three interlinked directions of research: theory, engineering applications (that is, making intelligent software) and neuroscience (that is, understanding the brain's mechanisms of learning).

Hierarchical models of object recognition in cortex.

Visual processing in cortex is classically modeled as a hierarchy of increasingly sophisticated representations, naturally extending the model of simple to complex cells of Hubel and Wiesel. Surprisingly, little quantitative modeling has been done to explore the biological feasibility of this class of models to explain aspects of higher-level visual processing such as object recognition. We describe a new hierarchical model consistent with physiological data from inferotemporal cortex that accounts for this complex visual task and makes testable predictions. The model is based on a MAX-like operation applied to inputs to certain cortical neurons that may have a general role in cortical function.

A Sparse Representation for Function Approximation.

We derive a new general representation for a function as a linear combination of local correlation kernels at optimal sparse locations (and scales) and characterize its relation to principal component analysis, regularization, sparsity principles, and support vector machines.

Microbiology of acute arthropathies among children in Argentina: Mycoplasma pneumoniae and hominis and Ureaplasma urealyticum.

OBJECTIVE: To describe the isolation of mycoplasmas and ureaplasmas from synovial fluid in pediatric patients with joint disorders. METHODS: During 1 year 45 samples of synovial fluid, blood and urine were collected from 33 hospitalized pediatric patients up to 17 years old who had joint disorders. Mycoplasmas and ureaplasmas were isolated in joint fluid by culture methods. RESULTS: Of the 33 patients 12 (36%) had joint disorders associated with pathogens (bacteria, Mycoplasma/Ureaplasma, Chlamydia) present at the site of inflammation. Mycoplasma hominis and Ureaplasma urealyticum were isolated from 3 and 1% of joint fluid samples, respectively. M. pneumoniae was isolated from nasopharyngeal secretion in a patient with evidence of a reactive arthritis. CONCLUSION: Our results raise the question of the possible role of Mycoplasma as a cofactor in the triggering of inflammatory joint disease, as well as the hypothesis that arthropathies may be caused by chronic local infection. These findings may contribute to early diagnosis of the disease and initiation of specific treatment.

Top-down learning of low-level vision tasks.

Perceptual tasks such as edge detection, image segmentation, lightness computation and estimation of three-dimensional structure are considered to be low-level or mid-level vision problems and are traditionally approached in a bottom-up, generic and hard-wired way. An alternative to this would be to take a top-down, object-class-specific and example-based approach. In this paper, we present a simple computational model implementing the latter approach. The results generated by our model when tested on edge-detection and view-prediction tasks for three-dimensional objects are consistent with human perceptual expectations. The model's performance is highly tolerant to the problems of sensor noise and incomplete input image information. Results obtained with conventional bottom-up strategies show much less immunity to these problems. We interpret the encouraging performance of our computational model as evidence in support of the hypothesis that the human visual system may learn to perform supposedly low-level perceptual tasks in a top-down fashion.

Role of learning in three-dimensional form perception.

One of the most remarkable characteristics of the human visual system is its ability to perceive specific three-dimensional forms in single two-dimensional contour images. This has often been attributed to a few general purpose and possibly innately specified shape biases, such as those favouring symmetry and other structural regularities (Fig. 1). An alternative approach proposed by the early empiricists and since tested suggests that this ability may also be acquired from visual experience, with the three-dimensional percept being the manifestation of a learned association between specific two-dimensional projections and the correlated three-dimensional structures. These studies of shape learning have been considered inconclusive, however, because their results can potentially be accounted for as cognitive decisions that might have little to do with shape perception per se. Here we present an experimental system that enables objective verification of the role of learning in shape perception by rendering the learning to be perceptually manifest. We show that the human visual system can learn associations between arbitrarily paired two-dimensional pictures and (projectionally consistent) three-dimensional structures. These results implicate high-level recognition processes in the task of shape perception.

Image representations for visual learning.

Computer vision researchers are developing new approaches to object recognition and detection that are based almost directly on images and avoid the use of intermediate three-dimensional models. Many of these techniques depend on a representation of images that induce a linear vector space structure and in principle requires dense feature correspondence. This image representation allows the use of learning techniques for the analysis of images (for computer vision) as well as for the synthesis of images (for computer graphics).

Fast perceptual learning in hyperacuity.

We investigated fast improvement of visual performance in several hyperacuity tasks such as vernier acuity and stereoscopic depth perception in almost 100 observers. Results indicate that the fast phase of perceptual learning, occurring within less than 1 hr of training, is specific for the visual field position and for the particular hyperacuity task, but is only partly specific for the eye trained and for the offset tested. Learning occurs without feedback. We conjecture that the site of learning may be quite early in the visual pathway.

Shape representation in the inferior temporal cortex of monkeys.

BACKGROUND: The inferior temporal cortex (IT) of the monkey has long been known to play an essential role in visual object recognition. Damage to this area results in severe deficits in perceptual learning and object recognition, without significantly affecting basic visual capacities. Consistent with these ablation studies is the discovery of IT neurons that respond to complex two-dimensional visual patterns, or objects such as faces or body parts. What is the role of these neurons in object recognition? Is such a complex configurational selectivity specific to biologically meaningful objects, or does it develop as a result of extensive exposure to any objects whose identification relies on subtle shape differences? If so, would IT neurons respond selectively to recently learned views of features of novel objects? The present study addresses this question by using combined psychophysical and electrophysiological experiments, in which monkeys learned to classify and recognize computer-generated three-dimensional objects. RESULTS: A population of IT neurons was found that responded selectively to views of previously unfamiliar objects. The cells discharged maximally to one view of an object, and their response declined gradually as the object was rotated away from this preferred view. No selective responses were ever encountered for views that the animal systematically failed to recognize. Most neurons also exhibited orientation-dependent responses during view-plane rotations. Some neurons were found to be tuned around two views of the same object, and a very small number of cells responded in a view-invariant manner. For the five different objects that were used extensively during the training of the animals, and for which behavioral performance became view-independent, multiple cells were found that were tuned around different views of the same object. A number of view-selective units showed response invariance for changes in the size of the object or the position of its image within the parafovea. CONCLUSION: Our results suggest that IT neurons can develop a complex receptive field organization as a consequence of extensive training in the discrimination and recognition of objects. None of these objects had any prior meaning for the animal, nor did they resemble anything familiar in the monkey's environment. Simple geometric features did not appear to account for the neurons' selective responses. These findings support the idea that a population of neurons--each tuned to a different object aspect, and each showing a certain degree of invariance to image transformations--may, as an ensemble, encode at least some types of complex three-dimensional objects. In such a system, several neurons may be active for any given vantage point, with a single unit acting like a blurred template for a limited neighborhood of a single view.

View-based models of 3D object recognition: invariance to imaging transformations.

This report describes the main features of a view-based model of object recognition. The model does not attempt to account for specific cortical structures; it tries to capture general properties to be expected in a biological architecture for object recognition. The basic module is a regularization network (RBF-like; see Poggio and Girosi, 1989; Poggio, 1990) in which each of the hidden units is broadly tuned to a specific view of the object to be recognized. The network output, which may be largely view independent, is first described in terms of some simple simulations. The following refinements and details of the basic module are then discussed: (1) some of the units may represent only components of views of the object--the optimal stimulus for the unit, its "center," is effectively a complex feature; (2) the units' properties are consistent with the usual description of cortical neurons as tuned to multidimensional optimal stimuli and may be realized in terms of plausible biophysical mechanisms; (3) in learning to recognize new objects, preexisting centers may be used and modified, but also new centers may be created incrementally so as to provide maximal view invariance; (4) modules are part of a hierarchical structure--the output of a network may be used as one of the inputs to another, in this way synthesizing increasingly complex features and templates; (5) in several recognition tasks, in particular at the basic level, a single center using view-invariant features may be sufficient.