Bridging the gap: how to adopt opportunistic plant observations for phenology monitoring.

Plant phenology plays a vital role in assessing climate change. To monitor this, individual plants are traditionally visited and observed by trained volunteers organized in national or international networks - in Germany, for example, by the German Weather Service, DWD. However, their number of observers is continuously decreasing. In this study, we explore the feasibility of using opportunistically captured plant observations, collected via the plant identification app Flora Incognita to determine the onset of flowering and, based on that, create interpolation maps comparable to those of the DWD. Therefore, the opportunistic observations of 17 species collected in 2020 and 2021 were assigned to "Flora Incognita stations" based on location and altitude in order to mimic the network of stations forming the data basis for the interpolation conducted by the DWD. From the distribution of observations, the percentile representing onset of flowering date was calculated using a parametric bootstrapping approach and then interpolated following the same process as applied by the DWD. Our results show that for frequently observed, herbaceous and conspicuous species, the patterns of onset of flowering were similar and comparable between both data sources. We argue that a prominent flowering stage is crucial for accurately determining the onset of flowering from opportunistic plant observations, and we discuss additional factors, such as species distribution, location bias and societal events contributing to the differences among species and phenology data. In conclusion, our study demonstrates that the phenological monitoring of certain species can benefit from incorporating opportunistic plant observations. Furthermore, we highlight the potential to expand the taxonomic range of monitored species for phenological stage assessment through opportunistic plant observation data.

Deep Learning in Plant Phenological Research: A Systematic Literature Review.

Climate change represents one of the most critical threats to biodiversity with far-reaching consequences for species interactions, the functioning of ecosystems, or the assembly of biotic communities. Plant phenology research has gained increasing attention as the timing of periodic events in plants is strongly affected by seasonal and interannual climate variation. Recent technological development allowed us to gather invaluable data at a variety of spatial and ecological scales. The feasibility of phenological monitoring today and in the future depends heavily on developing tools capable of efficiently analyzing these enormous amounts of data. Deep Neural Networks learn representations from data with impressive accuracy and lead to significant breakthroughs in, e.g., image processing. This article is the first systematic literature review aiming to thoroughly analyze all primary studies on deep learning approaches in plant phenology research. In a multi-stage process, we selected 24 peer-reviewed studies published in the last five years (2016-2021). After carefully analyzing these studies, we describe the applied methods categorized according to the studied phenological stages, vegetation type, spatial scale, data acquisition- and deep learning methods. Furthermore, we identify and discuss research trends and highlight promising future directions. We present a systematic overview of previously applied methods on different tasks that can guide this emerging complex research field.

Image-Based Automated Recognition of 31 Poaceae Species: The Most Relevant Perspectives.

Poaceae represent one of the largest plant families in the world. Many species are of great economic importance as food and forage plants while others represent important weeds in agriculture. Although a large number of studies currently address the question of how plants can be best recognized on images, there is a lack of studies evaluating specific approaches for uniform species groups considered difficult to identify because they lack obvious visual characteristics. Poaceae represent an example of such a species group, especially when they are non-flowering. Here we present the results from an experiment to automatically identify Poaceae species based on images depicting six well-defined perspectives. One perspective shows the inflorescence while the others show vegetative parts of the plant such as the collar region with the ligule, adaxial and abaxial side of the leaf and culm nodes. For each species we collected 80 observations, each representing a series of six images taken with a smartphone camera. We extract feature representations from the images using five different convolutional neural networks (CNN) trained on objects from different domains and classify them using four state-of-the art classification algorithms. We combine these perspectives via score level fusion. In order to evaluate the potential of identifying non-flowering Poaceae we separately compared perspective combinations either comprising inflorescences or not. We find that for a fusion of all six perspectives, using the best combination of feature extraction CNN and classifier, an accuracy of 96.1% can be achieved. Without the inflorescence, the overall accuracy is still as high as 90.3%. In all but one case the perspective conveying the most information about the species (excluding inflorescence) is the ligule in frontal view. Our results show that even species considered very difficult to identify can achieve high accuracies in automatic identification as long as images depicting suitable perspectives are available. We suggest that our approach could be transferred to other difficult-to-distinguish species groups in order to identify the most relevant perspectives.

Flora Capture: a citizen science application for collecting structured plant observations.

BACKGROUND: Digital plant images are becoming increasingly important. First, given a large number of images deep learning algorithms can be trained to automatically identify plants. Second, structured image-based observations provide information about plant morphological characteristics. Finally in the course of digitalization, digital plant collections receive more and more interest in schools and universities. RESULTS: We developed a freely available mobile application called Flora Capture allowing users to collect series of plant images from predefined perspectives. These images, together with accompanying metadata, are transferred to a central project server where each observation is reviewed and validated by a team of botanical experts. Currently, more than 4800 plant species, naturally occurring in the Central European region, are covered by the application. More than 200,000 images, depicting more than 1700 plant species, have been collected by thousands of users since the initial app release in 2016. CONCLUSION: Flora Capture allows experts, laymen and citizen scientists to collect a digital herbarium and share structured multi-modal observations of plants. Collected images contribute, e.g., to the training of plant identification algorithms, but also suit educational purposes. Additionally, presence records collected with each observation allow contribute to verifiable records of plant occurrences across the world.

Flowers, leaves or both? How to obtain suitable images for automated plant identification.

BACKGROUND: Deep learning algorithms for automated plant identification need large quantities of precisely labelled images in order to produce reliable classification results. Here, we explore what kind of perspectives and their combinations contain more characteristic information and therefore allow for higher identification accuracy. RESULTS: We developed an image-capturing scheme to create observations of flowering plants. Each observation comprises five in-situ images of the same individual from predefined perspectives (entire plant, flower frontal- and lateral view, leaf top- and back side view). We collected a completely balanced dataset comprising 100 observations for each of 101 species with an emphasis on groups of conspecific and visually similar species including twelve Poaceae species. We used this dataset to train convolutional neural networks and determine the prediction accuracy for each single perspective and their combinations via score level fusion. Top-1 accuracies ranged between 77% (entire plant) and 97% (fusion of all perspectives) when averaged across species. Flower frontal view achieved the highest accuracy (88%). Fusing flower frontal, flower lateral and leaf top views yields the most reasonable compromise with respect to acquisition effort and accuracy (96%). The perspective achieving the highest accuracy was species dependent. CONCLUSIONS: We argue that image databases of herbaceous plants would benefit from multi organ observations, comprising at least the front and lateral perspective of flowers and the leaf top view.

A meta food web for invertebrate species collected in a European grassland.

Patterns of feeding interactions between species are thought to influence the stability of communities and the flux of nutrients and energy through ecosystems. However, surprisingly few well-resolved food webs allow us to evaluate factors that influence the architecture of species interactions. We constructed a meta food web consisting of 714 invertebrate species collected over 9 years of suction and pitfall sampling campaigns in the Jena Experiment, a long-term grassland biodiversity experiment located in Jena, Germany. We summarize information on the 51,496 potential trophic links, which were established using information on diet specificity and species traits that typically constrain feeding interactions (trophic group, body size, and vertical stratification). The list of species identities, traits, and link-derivation rules will be useful not only for tests of plant diversity effects on food web structure within the Jena Experiment, but also for considering consistent construction of food webs from empirical data, and for comparisons of network structure across ecosystems. No copyright or proprietary restrictions are associated with the use of this data set other than citation of this Data Paper.

Plant diversity alters the representation of motifs in food webs.

Changes in the diversity of plant communities may undermine the economically and environmentally important consumer species they support. The structure of trophic interactions determines the sensitivity of food webs to perturbations, but rigorous assessments of plant diversity effects on network topology are lacking. Here, we use highly resolved networks from a grassland biodiversity experiment to test how plant diversity affects the prevalence of different food web motifs, the smaller recurrent sub-networks that form the building blocks of complex networks. We find that the representation of tri-trophic chain, apparent competition and exploitative competition motifs increases with plant species richness, while the representation of omnivory motifs decreases. Moreover, plant species richness is associated with altered patterns of local interactions among arthropod consumers in which plants are not directly involved. These findings reveal novel structuring forces that plant diversity exerts on food webs with potential implications for the persistence and functioning of multitrophic communities.

Image-based classification of plant genus and family for trained and untrained plant species.

BACKGROUND: Modern plant taxonomy reflects phylogenetic relationships among taxa based on proposed morphological and genetic similarities. However, taxonomical relation is not necessarily reflected by close overall resemblance, but rather by commonality of very specific morphological characters or similarity on the molecular level. It is an open research question to which extent phylogenetic relations within higher taxonomic levels such as genera and families are reflected by shared visual characters of the constituting species. As a consequence, it is even more questionable whether the taxonomy of plants at these levels can be identified from images using machine learning techniques. RESULTS: Whereas previous studies on automated plant identification from images focused on the species level, we investigated classification at higher taxonomic levels such as genera and families. We used images of 1000 plant species that are representative for the flora of Western Europe. We tested how accurate a visual representation of genera and families can be learned from images of their species in order to identify the taxonomy of species included in and excluded from learning. Using natural images with random content, roughly 500 images per species are required for accurate classification. The classification accuracy for 1000 species amounts to 82.2% and increases to 85.9% and 88.4% on genus and family level. Classifying species excluded from training, the accuracy significantly reduces to 38.3% and 38.7% on genus and family level. Excluded species of well represented genera and families can be classified with 67.8% and 52.8% accuracy. CONCLUSION: Our results show that shared visual characters are indeed present at higher taxonomic levels. Most dominantly they are preserved in flowers and leaves, and enable state-of-the-art classification algorithms to learn accurate visual representations of plant genera and families. Given a sufficient amount and composition of training data, we show that this allows for high classification accuracy increasing with the taxonomic level and even facilitating the taxonomic identification of species excluded from the training process.

Recommending plant taxa for supporting on-site species identification.

BACKGROUND: Predicting a list of plant taxa most likely to be observed at a given geographical location and time is useful for many scenarios in biodiversity informatics. Since efficient plant species identification is impeded mainly by the large number of possible candidate species, providing a shortlist of likely candidates can help significantly expedite the task. Whereas species distribution models heavily rely on geo-referenced occurrence data, such information still remains largely unused for plant taxa identification tools. RESULTS: In this paper, we conduct a study on the feasibility of computing a ranked shortlist of plant taxa likely to be encountered by an observer in the field. We use the territory of Germany as case study with a total of 7.62M records of freely available plant presence-absence data and occurrence records for 2.7k plant taxa. We systematically study achievable recommendation quality based on two types of source data: binary presence-absence data and individual occurrence records. Furthermore, we study strategies for aggregating records into a taxa recommendation based on location and date of an observation. CONCLUSION: We evaluate recommendations using 28k geo-referenced and taxa-labeled plant images hosted on the Flickr website as an independent test dataset. Relying on location information from presence-absence data alone results in an average recall of 82%. However, we find that occurrence records are complementary to presence-absence data and using both in combination yields considerably higher recall of 96% along with improved ranking metrics. Ultimately, by reducing the list of candidate taxa by an average of 62%, a spatio-temporal prior can substantially expedite the overall identification problem.

Automated plant species identification-Trends and future directions.

Current rates of species loss triggered numerous attempts to protect and conserve biodiversity. Species conservation, however, requires species identification skills, a competence obtained through intensive training and experience. Field researchers, land managers, educators, civil servants, and the interested public would greatly benefit from accessible, up-to-date tools automating the process of species identification. Currently, relevant technologies, such as digital cameras, mobile devices, and remote access to databases, are ubiquitously available, accompanied by significant advances in image processing and pattern recognition. The idea of automated species identification is approaching reality. We review the technical status quo on computer vision approaches for plant species identification, highlight the main research challenges to overcome in providing applicable tools, and conclude with a discussion of open and future research thrusts.

Acquiring and preprocessing leaf images for automated plant identification: understanding the tradeoff between effort and information gain.

BACKGROUND: Automated species identification is a long term research subject. Contrary to flowers and fruits, leaves are available throughout most of the year. Offering margin and texture to characterize a species, they are the most studied organ for automated identification. Substantially matured machine learning techniques generate the need for more training data (aka leaf images). Researchers as well as enthusiasts miss guidance on how to acquire suitable training images in an efficient way. METHODS: In this paper, we systematically study nine image types and three preprocessing strategies. Image types vary in terms of in-situ image recording conditions: perspective, illumination, and background, while the preprocessing strategies compare non-preprocessed, cropped, and segmented images to each other. Per image type-preprocessing combination, we also quantify the manual effort required for their implementation. We extract image features using a convolutional neural network, classify species using the resulting feature vectors and discuss classification accuracy in relation to the required effort per combination. RESULTS: The most effective, non-destructive way to record herbaceous leaves is to take an image of the leaf's top side. We yield the highest classification accuracy using destructive back light images, i.e., holding the plucked leaf against the sky for image acquisition. Cropping the image to the leaf's boundary substantially improves accuracy, while precise segmentation yields similar accuracy at a substantially higher effort. The permanent use or disuse of a flash light has negligible effects. Imaging the typically stronger textured backside of a leaf does not result in higher accuracy, but notably increases the acquisition cost. CONCLUSIONS: In conclusion, the way in which leaf images are acquired and preprocessed does have a substantial effect on the accuracy of the classifier trained on them. For the first time, this study provides a systematic guideline allowing researchers to spend available acquisition resources wisely while yielding the optimal classification accuracy.

Plant species classification using flower images-A comparative study of local feature representations.

Steady improvements of image description methods induced a growing interest in image-based plant species classification, a task vital to the study of biodiversity and ecological sensitivity. Various techniques have been proposed for general object classification over the past years and several of them have already been studied for plant species classification. However, results of these studies are selective in the evaluated steps of a classification pipeline, in the utilized datasets for evaluation, and in the compared baseline methods. No study is available that evaluates the main competing methods for building an image representation on the same datasets allowing for generalized findings regarding flower-based plant species classification. The aim of this paper is to comparatively evaluate methods, method combinations, and their parameters towards classification accuracy. The investigated methods span from detection, extraction, fusion, pooling, to encoding of local features for quantifying shape and color information of flower images. We selected the flower image datasets Oxford Flower 17 and Oxford Flower 102 as well as our own Jena Flower 30 dataset for our experiments. Findings show large differences among the various studied techniques and that their wisely chosen orchestration allows for high accuracies in species classification. We further found that true local feature detectors in combination with advanced encoding methods yield higher classification results at lower computational costs compared to commonly used dense sampling and spatial pooling methods. Color was found to be an indispensable feature for high classification results, especially while preserving spatial correspondence to gray-level features. In result, our study provides a comprehensive overview of competing techniques and the implications of their main parameters for flower-based plant species classification.

Aquifer community structure in dependence of lithostratigraphy in groundwater reservoirs.

Groundwater microbiology with respect to different host rocks offers new possibilities to describe and map the habitat harboring approximately half of Earths' biomass. The Thuringian Basin (Germany) contains formations of the Permian (Zechstein) and Triassic (Muschelkalk and Buntsandstein) with outcrops and deeper regions at the border and central part. Hydro(geo)chemistry and bacterial community structure of 11 natural springs and 20 groundwater wells were analyzed to define typical patterns for each formation. Widespread were Gammaproteobacteria, while Bacilli were present in all wells. Halotolerant and halophilic taxa were present in Zechstein. The occurrence of specific taxa allowed a clear separation of communities from all three lithostratigraphic groups. These specific taxa could be used to follow fluid movement, e.g., from the underlying Zechstein or from nearby saline reservoirs into Buntsandstein aquifers. Thus, we developed a new tool to identify the lithostratigraphic origin of sources in mixed waters. This was verified with entry of surface water, as species not present in the underground Zechstein environments were isolated from the water samples. Thus, our tool shows a higher resolution as compared to hydrochemistry, which is prone to undergo fast dilution if water mixes with other aquifers. Furthermore, the bacteria well adapted to their respective environment showed geographic clustering allowing to differentiate regional aquifers.

Complexity of multitrophic interactions in a grassland ecosystem depends on plant species diversity.

1. We studied the theoretical prediction that a loss of plant species richness has a strong impact on community interactions among all trophic levels and tested whether decreased plant species diversity results in a less complex structure and reduced interactions in ecological networks. 2. Using plant species-specific biomass and arthropod abundance data from experimental grassland plots (Jena Experiment), we constructed multitrophic functional group interaction webs to compare communities based on 4 and 16 plant species. 427 insect and spider species were classified into 13 functional groups. These functional groups represent the nodes of ecological networks. Direct and indirect interactions among them were assessed using partial Mantel tests. Interaction web complexity was quantified using three measures of network structure: connectance, interaction diversity and interaction strength. 3. Compared with high plant diversity plots, interaction webs based on low plant diversity plots showed reduced complexity in terms of total connectance, interaction diversity and mean interaction strength. Plant diversity effects obviously cascade up the food web and modify interactions across all trophic levels. The strongest effects occurred in interactions between adjacent trophic levels (i.e. predominantly trophic interactions), while significant interactions among plant and carnivore functional groups, as well as horizontal interactions (i.e. interactions between functional groups of the same trophic level), showed rather inconsistent responses and were generally rarer. 4. Reduced interaction diversity has the potential to decrease and destabilize ecosystem processes. Therefore, we conclude that the loss of basal producer species leads to more simple structured, less and more loosely connected species assemblages, which in turn are very likely to decrease ecosystem functioning, community robustness and tolerance to disturbance. Our results suggest that the functioning of the entire ecological community is critically linked to the diversity of its component plants species.

GC content-independent amino acid patterns in bacteria and archaea.

Every organism can be characterized by the amino acid composition of its proteome. So far it was assumed that these compositions are determined by the GC content of the DNA or, in some cases, by extreme lifestyles, like thermophily or halophily. Here, we focussed our analysis on eight amino acids, each of which is encoded by both, GC and AT rich codons, to identify finer amino acid patterns beyond the GC dominance. We investigated the conceptually translated proteomes of 1029 bacterial and archaeal strains with sequenced genomes for amino acid composition. Using correspondence analysis, we found that phylogenetic groups within bacteria and archaea generally can be discriminated from other groups due to their amino acid composition. In some cases, single organisms, e.g. Treponema pallidum strains or Mycoplasma penetrans, are characterized by extreme amino acid compositions. We assume that our data could provide a basis for a new approach to analyze evolution of bacterial and archaeal groups. Furthermore, for single organisms, the detailed knowledge of the amino acid composition of the entire proteome encoded in the genome could lead to a better understanding, important for pharmaceutical or biotechnological applications. We recommend that information about amino acid compositions should be provided in databases, comparable to the GC content of genomes.