Quantitative and qualitative phenotyping of disease resistance of crops by hyperspectral sensors: seamless interlocking of phytopathology, sensors, and machine learning is needed!

Determination and characterization of resistance reactions of crops against fungal pathogens are essential to select resistant genotypes. In plant breeding, phenotyping of genotypes is realized by time consuming and expensive visual plant ratings. During resistance reactions and during pathogenesis plants initiate different structural and biochemical defence mechanisms, which partly affect the optical properties of plant organs. Recently, intensive research has been conducted to develop innovative optical methods for an assessment of compatible and incompatible plant pathogen interaction. These approaches, combining classical phytopathology or microbiology with technology driven methods - such as sensors, robotics, machine learning, and artificial intelligence - are summarized by the term digital phenotyping. In contrast to common visual rating, detection and assessment methods, optical sensors in combination with advanced data analysis methods are able to retrieve pathogen induced changes in the physiology of susceptible or resistant plants non-invasively and objectively. Phenotyping disease resistance aims different tasks. In an early breeding step, a qualitative assessment and characterization of specific resistance action is aimed to link it, for example, to a genetic marker. Later, during greenhouse and field screening, the assessment of the level of susceptibility of different genotypes is relevant. Within this review, recent advances of digital phenotyping technologies for the detection of subtle resistance reactions and resistance breeding are highlighted and methodological requirements are critically discussed.

Monitoring wound healing in a 3D wound model by hyperspectral imaging and efficient clustering.

Wound healing is a complex and dynamic process with different distinct and overlapping phases from homeostasis, inflammation and proliferation to remodelling. Monitoring the healing response of injured tissue is of high importance for basic research and clinical practice. In traditional application, biological markers characterize normal and abnormal wound healing. Understanding functional relationships of these biological processes is essential for developing new treatment strategies. However, most of the present techniques (in vitro or in vivo) include invasive microscopic or analytical tissue sampling. In the present study, a non-invasive alternative for monitoring processes during wound healing is introduced. Within this context, hyperspectral imaging (HSI) is an emerging and innovative non-invasive imaging technique with different opportunities in medical applications. HSI acquires the spectral reflectance of an object, depending on its biochemical and structural characteristics. An in-vitro 3-dimensional (3-D) wound model was established and incubated without and with acute and chronic wound fluid (AWF, CWF), respectively. Hyperspectral images of each individual specimen of this 3-D wound model were assessed at day 0/5/10 in vitro, and reflectance spectra were evaluated. For analysing the complex hyperspectral data, an efficient unsupervised approach for clustering massive hyperspectral data was designed, based on efficient hierarchical decomposition of spectral information according to archetypal data points. It represents, to the best of our knowledge, the first application of an advanced Data Mining approach in context of non-invasive analysis of wounds using hyperspectral imagery. By this, temporal and spatial pattern of hyperspectral clusters were determined within the tissue discs and among the different treatments. Results from non-invasive imaging were compared to the number of cells in the various clusters, assessed by Hematoxylin/Eosin (H/E) staining. It was possible to correlate cell quantity and spectral reflectance during wound closure in a 3-D wound model in vitro.

Spectral Patterns Reveal Early Resistance Reactions of Barley Against Blumeria graminis f. sp. hordei.

Differences in early plant-pathogen interactions are mainly characterized by using destructive methods. Optical sensors are advanced techniques for phenotyping host-pathogen interactions on different scales and for detecting subtle plant resistance responses against pathogens. A microscope with a hyperspectral camera was used to study interactions between Blumeria graminis f. sp. hordei and barley (Hordeum vulgare) genotypes with high susceptibility or resistance due to hypersensitive response (HR) and papilla formation. Qualitative and quantitative assessment of pathogen development was used to explain changes in hyperspectral signatures. Within 48 h after inoculation, genotype-specific changes in the green and red range (500 to 690 nm) and a blue shift of the red-edge inflection point were observed. Manual analysis indicated resistance-specific reflectance patterns from 1 to 3 days after inoculation. These changes could be linked to host plant modifications depending on individual host-pathogen interactions. Retrospective analysis of hyperspectral images revealed spectral characteristics of HR against B. graminis f. sp. hordei. For early HR detection, an advanced data mining approach localized HR spots before they became visible on the RGB images derived from hyperspectral imaging. The link among processes during pathogenesis and host resistance to changes in hyperspectral signatures provide evidence that sensor-based phenotyping is suitable to advance time-consuming and cost-expensive visual rating of plant disease resistances.

Plant Phenotyping using Probabilistic Topic Models: Uncovering the Hyperspectral Language of Plants.

Modern phenotyping and plant disease detection methods, based on optical sensors and information technology, provide promising approaches to plant research and precision farming. In particular, hyperspectral imaging have been found to reveal physiological and structural characteristics in plants and to allow for tracking physiological dynamics due to environmental effects. In this work, we present an approach to plant phenotyping that integrates non-invasive sensors, computer vision, as well as data mining techniques and allows for monitoring how plants respond to stress. To uncover latent hyperspectral characteristics of diseased plants reliably and in an easy-to-understand way, we "wordify" the hyperspectral images, i.e., we turn the images into a corpus of text documents. Then, we apply probabilistic topic models, a well-established natural language processing technique that identifies content and topics of documents. Based on recent regularized topic models, we demonstrate that one can track automatically the development of three foliar diseases of barley. We also present a visualization of the topics that provides plant scientists an intuitive tool for hyperspectral imaging. In short, our analysis and visualization of characteristic topics found during symptom development and disease progress reveal the hyperspectral language of plant diseases.

Non-negative matrix factorization for the near real-time interpretation of absorption effects in elemental distribution images acquired by X-ray fluorescence imaging.

Elemental distribution images acquired by imaging X-ray fluorescence analysis can contain high degrees of redundancy and weakly discernible correlations. In this article near real-time non-negative matrix factorization (NMF) is described for the analysis of a number of data sets acquired from samples of a bi-modal α+ß Ti-6Al-6V-2Sn alloy. NMF was used for the first time to reveal absorption artefacts in the elemental distribution images of the samples, where two phases of the alloy, namely α and ß, were in superposition. The findings and interpretation of the NMF results were confirmed by Monte Carlo simulation of the layered alloy system. Furthermore, it is shown how the simultaneous factorization of several stacks of elemental distribution images provides uniform basis vectors and consequently simplifies the interpretation of the representation.

Hyperspectral imaging reveals the effect of sugar beet quantitative trait loci on Cercospora leaf spot resistance.

The quantitative resistance of sugar beet (Beta vulgaris L.) against Cercospora leaf spot (CLS) caused by Cercospora beticola (Sacc.) was characterised by hyperspectral imaging. Two closely related inbred lines, differing in two quantitative trait loci (QTL), which made a difference in disease severity of 1.1-1.7 on the standard scoring scale (1-9), were investigated under controlled conditions. The temporal and spatial development of CLS lesions on the two genotypes were monitored using a hyperspectral microscope. The lesion development on the QTL-carrying, resistant genotype was characterised by a fast and abrupt change in spectral reflectance, whereas it was slower and ultimately more severe on the genotype lacking the QTL. An efficient approach for clustering of hyperspectral signatures was adapted in order to reveal resistance characteristics automatically. The presented method allowed a fast and reliable differentiation of CLS dynamics and lesion composition providing a promising tool to improve resistance breeding by objective and precise plant phenotyping.

Observation of plant-pathogen interaction by simultaneous hyperspectral imaging reflection and transmission measurements.

Hyperspectral imaging sensors are valuable tools for plant disease detection and plant phenotyping. Reflectance properties are influenced by plant pathogens and resistance responses, but changes of transmission characteristics of plants are less described. In this study we used simultaneously recorded reflectance and transmittance imaging data of resistant and susceptible barley genotypes that were inoculated with Blumeria graminis f. sp. hordei to evaluate the added value of imaging transmission, reflection and absorption for characterisation of disease development. These datasets were statistically analysed using principal component analysis, and compared with visual and molecular disease estimation. Reflection measurement performed significantly better for early detection of powdery mildew infection, colonies could be detected 2 days before symptoms became visible in RGB images. Transmission data could be used to detect powdery mildew 2 days after symptoms becoming visible in reflection based RGB images. Additionally distinct transmission changes occurred at 580-650nm for pixels containing disease symptoms. It could be shown that the additional information of the transmission data allows for a clearer spatial differentiation and localisation between powdery mildew symptoms and necrotic tissue on the leaf then purely reflectance based data. Thus the information of both measurement approaches are complementary: reflectance based measurements facilitate an early detection, and transmission measurements provide additional information to better understand and quantify the complex spatio-temporal dynamics of plant-pathogen interactions.

Automated interpretation of 3D laserscanned point clouds for plant organ segmentation.

BACKGROUND: Plant organ segmentation from 3D point clouds is a relevant task for plant phenotyping and plant growth observation. Automated solutions are required to increase the efficiency of recent high-throughput plant phenotyping pipelines. However, plant geometrical properties vary with time, among observation scales and different plant types. The main objective of the present research is to develop a fully automated, fast and reliable data driven approach for plant organ segmentation. RESULTS: The automated segmentation of plant organs using unsupervised, clustering methods is crucial in cases where the goal is to get fast insights into the data or no labeled data is available or costly to achieve. For this we propose and compare data driven approaches that are easy-to-realize and make the use of standard algorithms possible. Since normalized histograms, acquired from 3D point clouds, can be seen as samples from a probability simplex, we propose to map the data from the simplex space into Euclidean space using Aitchisons log ratio transformation, or into the positive quadrant of the unit sphere using square root transformation. This, in turn, paves the way to a wide range of commonly used analysis techniques that are based on measuring the similarities between data points using Euclidean distance. We investigate the performance of the resulting approaches in the practical context of grouping 3D point clouds and demonstrate empirically that they lead to clustering results with high accuracy for monocotyledonous and dicotyledonous plant species with diverse shoot architecture. CONCLUSION: An automated segmentation of 3D point clouds is demonstrated in the present work. Within seconds first insights into plant data can be deviated - even from non-labelled data. This approach is applicable to different plant species with high accuracy. The analysis cascade can be implemented in future high-throughput phenotyping scenarios and will support the evaluation of the performance of different plant genotypes exposed to stress or in different environmental scenarios.

Hyperspectral phenotyping on the microscopic scale: towards automated characterization of plant-pathogen interactions.

BACKGROUND: The detection and characterization of resistance reactions of crop plants against fungal pathogens are essential to select resistant genotypes. In breeding practice phenotyping of plant genotypes is realized by time consuming and expensive visual rating. In this context hyperspectral imaging (HSI) is a promising non-invasive sensor technique in order to accelerate and to automate classical phenotyping methods. A hyperspectral microscope was established to determine spectral changes on the leaf and cellular level of barley (Hordeum vulgare) during resistance reactions against powdery mildew (Blumeria graminis f.sp. hordei, isolate K1). Experiments were conducted with near isogenic barley lines of cv. Ingrid, including the susceptible wild type (WT), mildew locus a 12 (Mla12 based resistance) and the resistant mildew locus o 3 (mlo3 based resistance), respectively. The reflection of inoculated and non-inoculated leaves was recorded daily with a hyperspectral linescanner in the visual (400 - 700 nm) and near infrared (700 - 1000 nm) range 3 to 14 days after inoculation. RESULTS: Data analysis showed no significant differences in spectral signatures between non-inoculated genotypes. Barley leaves of the near-isogenic genotypes, inoculated with B. graminis f.sp. hordei differed in the spectral reflectance over time, respectively. The susceptible genotypes (WT, Mla12) showed an increase in reflectance in the visible range according to symptom development. However, the spectral signature of the resistant mlo-genotype did not show significant changes over the experimental period. In addition, a recent data driven approach for automated discovery of disease specific signatures, which is based on a new representation of the data using Simplex Volume Maximization (SiVM) was applied. The automated approach - evaluated in only a fraction of time revealed results similar to the time and labor intensive manually assessed hyperspectral signatures. The new representation determined by SiVM was also used to generate intuitive and easy to interpretable summaries, e.g. fingerprints or traces of hyperspectral dynamics of the different genotypes. CONCLUSION: With this HSI based and data driven phenotyping approach an evaluation of host-pathogen interactions over time and a discrimination of barley genotypes differing in susceptibility to powdery mildew is possible.

Metro maps of plant disease dynamics--automated mining of differences using hyperspectral images.

Understanding the response dynamics of plants to biotic stress is essential to improve management practices and breeding strategies of crops and thus to proceed towards a more sustainable agriculture in the coming decades. In this context, hyperspectral imaging offers a particularly promising approach since it provides non-destructive measurements of plants correlated with internal structure and biochemical compounds. In this paper, we present a cascade of data mining techniques for fast and reliable data-driven sketching of complex hyperspectral dynamics in plant science and plant phenotyping. To achieve this, we build on top of a recent linear time matrix factorization technique, called Simplex Volume Maximization, in order to automatically discover archetypal hyperspectral signatures that are characteristic for particular diseases. The methods were applied on a data set of barley leaves (Hordeum vulgare) diseased with foliar plant pathogens Pyrenophora teres, Puccinia hordei and Blumeria graminis hordei. Towards more intuitive visualizations of plant disease dynamics, we use the archetypal signatures to create structured summaries that are inspired by metro maps, i.e. schematic diagrams of public transport networks. Metro maps of plant disease dynamics produced on several real-world data sets conform to plant physiological knowledge and explicitly illustrate the interaction between diseases and plants. Most importantly, they provide an abstract and interpretable view on plant disease progression.

Early drought stress detection in cereals: simplex volume maximisation for hyperspectral image analysis.

Early water stress recognition is of great relevance in precision plant breeding and production. Hyperspectral imaging sensors can be a valuable tool for early stress detection with high spatio-temporal resolution. They gather large, high dimensional data cubes posing a significant challenge to data analysis. Classical supervised learning algorithms often fail in applied plant sciences due to their need of labelled datasets, which are difficult to obtain. Therefore, new approaches for unsupervised learning of relevant patterns are needed. We apply for the first time a recent matrix factorisation technique, simplex volume maximisation (SiVM), to hyperspectral data. It is an unsupervised classification approach, optimised for fast computation of massive datasets. It allows calculation of how similar each spectrum is to observed typical spectra. This provides the means to express how likely it is that one plant is suffering from stress. The method was tested for drought stress, applied to potted barley plants in a controlled rain-out shelter experiment and to agricultural corn plots subjected to a two factorial field setup altering water and nutrient availability. Both experiments were conducted on the canopy level. SiVM was significantly better than using a combination of established vegetation indices. In the corn plots, SiVM clearly separated the different treatments, even though the effects on leaf and canopy traits were subtle.