Data pre-processing to improve the mining of large feed databases.

The information stored in animal feed databases is highly variable, in terms of both provenance and quality; therefore, data pre-processing is essential to ensure reliable results. Yet, pre-processing at best tends to be unsystematic; at worst, it may even be wholly ignored. This paper sought to develop a systematic approach to the various stages involved in pre-processing to improve feed database outputs. The database used contained analytical and nutritional data on roughly 20 000 alfalfa samples. A range of techniques were examined for integrating data from different sources, for detecting duplicates and, particularly, for detecting outliers. Special attention was paid to the comparison of univariate and multivariate solutions. Major issues relating to the heterogeneous nature of data contained in this database were explored, the observed outliers were characterized and ad hoc routines were designed for error control. Finally, a heuristic diagram was designed to systematize the various aspects involved in the detection and management of outliers and errors.

Handling of missing data to improve the mining of large feed databases.

Feed databases often have missing data. Despite their potentially major effect on data analysis (e.g., as a source of biased results and loss of statistical power), database managers and nutrition researchers have paid little attention to missing data. This study evaluated various methods of handling missing data using mining outputs from a database containing data on chemical composition and nutritive value for 18,864 alfalfa samples. A complete reference dataset was obtained comprising the 2,303 cases with no missing data for the attributes CP, crude fiber (CF), NDF, ADF and ADL. This dataset was used to simulate 2 types of missing data (at random and not at random), each with 2 loss intensities (33 and 66%), thus yielding a total of 4 incomplete datasets. Missing data from these datasets were handled using 2 deletion methods and 4 imputation methods, and outputs in terms of the identification and typing of alfalfa (using ANOVA and descriptive statistics) and of correlations between attributes (using regressions) were compared with outputs from the complete dataset. Imputation methods, particularly model-based versions, were found to perform better than deletion methods in terms of maximizing information use and minimizing bias although the extent of differences between methods depended on the type of missing data. The best approximation to the uncertainty value was provided by multiple imputation methods. It was concluded that the choice of the most suitable method for handling missing data depended both on the type of missing data and on the purpose of data analysis.

Building a metadata framework for sharing feed information in Spain.

Information about the nutritional aspects and uses of feed is of widespread interest, hence systematic efforts of laboratories to obtain it. The way this information is currently being handled leaves something to be desired, underscoring the need to use computerized systems and statistical techniques that allow the management of large volumes of heterogeneous information. This project seeks to develop a structure that will facilitate the exchange and exploitation of information on feeds produced in Spain. To this end, metadata and data mining techniques have been adopted by the Feed Information Service at the University of Cordoba. The structure has been designed to work on the basis of a server-client architecture, in which information is stored on local software (Califa) by its own creators so that it can subsequently be incorporated into a database server where it can be accessed online. Various aspects of the structure are described in this paper: organization (participants and data shared), format (physical features), logistics (data description), quality (reliability of information), legality (correct use of data), and financing (revenue and expenditure). An indication is given of the amount of information accumulated to date, now exceeding 200,000 numerical data and associated metadata, arranged in several thematic databases. The activities carried out highlight the heterogeneous nature of the information produced, as well as the large number of errors and ambiguities that slip through the normal filters and reach the end-user of the data.

Nutritive evaluation of olive tree leaves by near-infrared spectroscopy: effect of soil contamination and correction with spectral pretreatments.

Olive leaves obtained as a byproduct in the Mediterranean region could play an important role in the nutrition of extensive ruminant systems. However, the reported variation in their nutritive value, among other reasons due to discrepancies in mineral content, is considered an important obstacle for their common use. Near-infrared spectroscopy (NIRS) could fulfill the requirements of these productive systems, providing analytical information in a rapid and economic way. In this work, the effect of soil contamination on NIR spectra has been studied, as well as its correction with some of the most commonly used spectral pretreatments (derivatives, multiplicative scatter correction, auto scaling, detrending, and a combination of the last two transforms). Effects were evaluated by visual inspection of the transformed spectra and comparison of the calibration statistics obtained to estimate acid insoluble ash and total ash contents and in vitro pepsin cellulase digestibility of organic and dry matter. The incidence of spectral curvature effects caused by soil contamination that can be conveniently corrected with pretreatments such as derivatives was confirmed.