Designing a Proximal Sensing Camera Acquisition System for Vineyard Applications: Results and Feedback on 8 Years of Experiments.

The potential of image proximal sensing for agricultural applications has been a prolific scientific subject in the recent literature. Its main appeal lies in the sensing of precise information about plant status, which is either harder or impossible to extract from lower-resolution downward-looking image sensors such as satellite or drone imagery. Yet, many theoretical and practical problems arise when dealing with proximal sensing, especially on perennial crops such as vineyards. Indeed, vineyards exhibit challenging physical obstacles and many degrees of variability in their layout. In this paper, we present the design of a mobile camera suited to vineyards and harsh experimental conditions, as well as the results and assessments of 8 years' worth of studies using that camera. These projects ranged from in-field yield estimation (berry counting) to disease detection, providing new insights on typical viticulture problems that could also be generalized to orchard crops. Different recommendations are then provided using small case studies, such as the difficulties related to framing plots with different structures or the mounting of the sensor on a moving vehicle. While results stress the obvious importance and strong benefits of a thorough experimental design, they also indicate some inescapable pitfalls, illustrating the need for more robust image analysis algorithms and better databases. We believe sharing that experience with the scientific community can only benefit the future development of these innovative approaches.

An annotated image dataset of downy mildew symptoms on Merlot grape variety.

This article introduces a dataset of high-resolution colour images of grapevines. It contains 99 images acquired in the vineyard from a cruising tractor. Each image includes the full foliage of a grapevine plant. These images display a diverse range of symptoms caused by the grapevine downy mildew (Plasmopara viticola), a major fungal disease. The dataset also includes various confounding factors, i.e. anomalies that are not related to the disease.  These anomalies are the natural and common phenomena affecting vineyards such as results of mechanical wounds, necroses, chemical burns or yellowing and discolorations due to nutritional or hydric deficiencies. Images were acquired in-situ on "Le Domaine de la Grande Ferrade" a public experimental facility of INRAE, in the area of Bordeaux. Acquisitions took place at early fruiting stages (BBCH 75-79) corresponding to the main sanitary pressure during growth. The acquisition device, embedded on a vine tractor, is composed of an industrial colour camera synchronised with powerful flashes. The purpose of this device is to produce a "day for night" effect that mitigates the variation of sunlight. It enables to homogenise images acquired during different weathers and time of the day and to ensure that the foreground (containing foliage) displays appropriate brightness, with minimum shadows while the background is darker. The images of the dataset were annotated manually by photo-interpretation with a careful review of expertise regarding phytopathology and physiological disorders. The annotation process consists in associating pixels with a class that defines its membership to a type of organ and its physiological state. Pixels from healthy, symptomatic or abnormal grapevine tissues were labelled into seven classes: "Limbus", "Leaf edges", "Berries", "Stems", "Foliar mildew", "Berries mildew" and "Anomalies". The annotation is achieved with the GIMP2 software as mask images where the value attributed to each pixel corresponds to one of the seven considered classes.

In Field Detection of Downy Mildew Symptoms with Proximal Colour Imaging.

This paper proposes to study the potentialities of on-board colour imaging for the in-field detection of a textbook case disease: the grapevine downy mildew. It introduces an algorithmic strategy for the detection of various forms of foliar symptoms on proximal high-resolution images. The proposed strategy is based on structure-colour representations and probabilistic models of grapevine tissues. It operates in three steps: (i) Formulating descriptors to extract the characteristic and discriminating properties of each class. They combine the Local Structure Tensors (LST) with colorimetric statistics calculated in pixel's neighbourhood. (ii) Modelling the statistical distributions of these descriptors in each class. To account for the specific nature of LSTs, the descriptors are mapped in the Log-Euclidean space. In this space, the classes of interest can be modelled with mixtures of multivariate Gaussian distributions. (iii) Assigning each pixel to one of the classes according to its suitability to their models. The decision method is based on a "seed growth segmentation" process. This step exploits statistical criteria derived from the probabilistic model. The resulting processing chain reliably detects downy mildew symptoms and estimates the area of the affected tissues. A leave-one-out cross-validation is conducted on a dataset constituted of a hundred independent images of grapevines affected only by downy mildew and/or abiotic stresses. The proposed method achieves an extensive and accurate recovery of foliar symptoms, with on average, a 83% pixel-wise precision and a 76% pixel-wise recall.

Near infrared hyperspectral dataset of healthy and infected apple tree leaves images for the early detection of apple scab disease.

This dataset presents two series of hyperspectral images of healthy and infected apple tree leaves acquired daily, from two days after inoculation until an advanced stage of infection (11 days after inoculation). The hyperspectral images were calibrated by reflection correction and registered to match the geometry of one reference image. On the last experiment day, scab positions are provided.

Texture Synthesis Using the Structure Tensor.

This paper proposes a two-stage texture synthesis algorithm. At the first stage, a structure tensor map carrying information about the local orientation is synthesized from the exemplar's data and used at the second stage to constrain the synthesis of the texture. Keeping in mind that the algorithm should be able to reproduce as faithfully as possible the visual aspect, statistics, and morphology of the input sample, the method is tested on various textures and compared objectively with existing methods, highlighting its strength in successfully synthesizing the output texture in many situations where traditional algorithms fail to reproduce the exemplar's patterns. The promising results pave the way towards the synthesis of accurately large and multi-scale patterns as it is the case for carbon material samples showing laminar structures, for example.

Maximum-likelihood based synthesis of volumetric textures from a 2D sample.

We propose a genuine 3D texture synthesis algorithm based on a probabilistic 2D Markov random field conceptualization, capable of capturing the visual characteristics of a texture into a unique statistical texture model. We intend to reproduce, in the volumetric texture, the interactions between pixels learned in an input 2D image. The learning is done by nonparametric Parzen-windowing. Optimization is handled voxel by a relaxation algorithm, aiming at maximizing the likelihood of each voxel in terms of its local conditional probability function. Variants are proposed regarding the relaxation algorithm and the heuristic strategies used for the simultaneous handling of the orthogonal slices containing the voxel. The procedures are materialized on various textures through a comparative study and a sensitivity analysis, highlighting the variants strengths and weaknesses. Finally, the probabilistic model is compared objectively with a nonparametric neighborhood-search-based algorithm.

Stereological estimation of orientation distribution of generalized cylinders from a unique 2D slice.

Though three-dimensional (3D) imaging gives deep insight into the inner structure of complex materials, the stereological analysis of 2D snapshots of material sections is still necessary for large-scale industrial applications for reasons related to time and cost constraints. In this paper, we propose an original framework to estimate the orientation distribution of generalized cylindrical structures from a single 2D section. Contrary to existing approaches, knowledge of the cylinder cross-section shape is not necessary. The only requirement is to know the area distribution of the cross-sections. The approach relies on minimization of a least squares criterion under linear equality and inequality constraints that can be solved with standard optimization solvers. It is evaluated on synthetic data, including simulated images, and is applied to experimental microscopy images of fibrous composite structures. The results show the relevance and capabilities of the approach though some limitations have been identified regarding sensitivity to deviations from the assumed model.

Confidence bounds for the estimation of the volume phase fraction from a single image in a nickel base superalloy.

We propose an image-based framework to evaluate the uncertainty in the estimation of the volume fraction of specific microstructures based on the observation of a single section. These microstructures consist of cubes organized on a cubic mesh, such as monocrystalline nickel base superalloys. The framework is twofold: a model-based stereological analysis allows relating two-dimensional image observations to three-dimensional microstructure features, and a spatial statistical analysis allows computing approximate confidence bounds while assessing the representativeness of the image. The reliability of the method is assessed on synthetic models. Volume fraction estimation variances and approximate confidence intervals are computed on real superalloy images in the context of material characterization.

Assessment of texture stationarity using the asymptotic behavior of the empirical mean and variance.

Given textured images considered as realizations of 2-D stochastic processes, a framework is proposed to evaluate the stationarity of their mean and variance. Existing strategies focus on the asymptotic behavior of the empirical mean and variance (respectively EM and EV), known for some types of nondeterministic processes. In this paper, the theoretical asymptotic behaviors of the EM and EV are studied for large classes of second-order stationary ergodic processes, in the sense of the Wold decomposition scheme, including harmonic and evanescent processes. Minimal rates of convergence for the EM and the EV are derived for these processes; they are used as criteria for assessing the stationarity of textures. The experimental estimation of the rate of convergence is achieved using a nonparametric block sub-sampling method. Our framework is evaluated on synthetic processes with stationary or nonstationary mean and variance and on real textures. It is shown that anomalies in the asymptotic behavior of the empirical estimators allow detecting nonstationarities of the mean and variance of the processes in an objective way.