PiGLET: Pixel-Level Grounding of Language Expressions With Transformers.

This paper proposes Panoptic Narrative Grounding, a spatially fine and general formulation of the natural language visual grounding problem. We establish an experimental framework for the study of this new task, including new ground truth and metrics. We propose PiGLET, a novel multi-modal Transformer architecture to tackle the Panoptic Narrative Grounding task, and to serve as a stepping stone for future work. We exploit the intrinsic semantic richness in an image by including panoptic categories, and we approach visual grounding at a fine-grained level using segmentations. In terms of ground truth, we propose an algorithm to automatically transfer Localized Narratives annotations to specific regions in the panoptic segmentations of the MS COCO dataset. PiGLET achieves a performance of 63.2 absolute Average Recall points. By leveraging the rich language information on the Panoptic Narrative Grounding benchmark on MS COCO, PiGLET obtains an improvement of 0.4 Panoptic Quality points over its base method on the panoptic segmentation task. Finally, we demonstrate the generalizability of our method to other natural language visual grounding problems such as Referring Expression Segmentation. PiGLET is competitive with previous state-of-the-art in RefCOCO, RefCOCO+ and RefCOCOg.

The extreme floods in the Ebro River basin since 1600 CE.

Reliable and complete knowledge of the historical floods is necessary for understanding the extreme hydrological dynamics of the rivers, their natural variability and anthropic changes. In this work we reconstruct the most important floods of the Ebro basin during the last 400 years in different areas of the basin. The analysis is based on four different areas: the Ebro River at Zaragoza, the Cinca River at Fraga, the Segre River at Lleida, and the Ebro River near its mouth at Tortosa. Based on a documentary research, we have first obtained relevant information about the initial conditions (rainfall duration and distribution, snow cover influence) and the maximum flood heights that allow to reconstruct the maximum peak flows by using hydraulic models and to calculate the subbasins contributions. The results show four main types of extreme floods: a) those affecting simultaneously all the subbasins with the highest peak discharges (Ebro at Tortosa in 1787: 0.15 m3 s-1 km-2); b) those originated at the western basin, upstream from Zaragoza, with an Atlantic origin, presenting moderate maximum peak flows, caused by persistent winter rainfall and where snowmelt significantly contributes to the flood; c) those originating at the central Pyrenean subbasins, with Mediterranean origin, occurring, with high peak discharges. These mainly occur during autumn as a consequence of rainfalls of different duration (between 3 days and 1 month), and without significant snow thawing and d) finally, less frequent but very intense flash floods events centered in the Lower Ebro area with low peak flows. In terms of frequency, two different periods can be distinguished: from 1600 until 1850, the frequency of events is low; since 1850 the frequency of events is clearly higher, due to an increase of the climatic variability during last stages of the Little Ice Age. From the 1960's reservoirs construction modifies discharges regime.

Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks.

We present Convolutional Oriented Boundaries (COB), which produces multiscale oriented contours and region hierarchies starting from generic image classification Convolutional Neural Networks (CNNs). COB is computationally efficient, because it requires a single CNN forward pass for multi-scale contour detection and it uses a novel sparse boundary representation for hierarchical segmentation; it gives a significant leap in performance over the state-of-the-art, and it generalizes very well to unseen categories and datasets. Particularly, we show that learning to estimate not only contour strength but also orientation provides more accurate results. We perform extensive experiments for low-level applications on BSDS, PASCAL Context, PASCAL Segmentation, and NYUD to evaluate boundary detection performance, showing that COB provides state-of-the-art contours and region hierarchies in all datasets. We also evaluate COB on high-level tasks when coupled with multiple pipelines for object proposals, semantic contours, semantic segmentation, and object detection on MS-COCO, SBD, and PASCAL; showing that COB also improves the results for all tasks.

Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation.

We propose a unified approach for bottom-up hierarchical image segmentation and object proposal generation for recognition, called Multiscale Combinatorial Grouping (MCG). For this purpose, we first develop a fast normalized cuts algorithm. We then propose a high-performance hierarchical segmenter that makes effective use of multiscale information. Finally, we propose a grouping strategy that combines our multiscale regions into highly-accurate object proposals by exploring efficiently their combinatorial space. We also present Single-scale Combinatorial Grouping (SCG), a faster version of MCG that produces competitive proposals in under five seconds per image. We conduct an extensive and comprehensive empirical validation on the BSDS500, SegVOC12, SBD, and COCO datasets, showing that MCG produces state-of-the-art contours, hierarchical regions, and object proposals.

Supervised Evaluation of Image Segmentation and Object Proposal Techniques.

This paper tackles the supervised evaluation of image segmentation and object proposal algorithms. It surveys, structures, and deduplicates the measures used to compare both segmentation results and object proposals with a ground truth database; and proposes a new measure: the precision-recall for objects and parts. To compare the quality of these measures, eight state-of-the-art object proposal techniques are analyzed and two quantitative meta-measures involving nine state of the art segmentation methods are presented. The meta-measures consist in assuming some plausible hypotheses about the results and assessing how well each measure reflects these hypotheses. As a conclusion of the performed experiments, this paper proposes the tandem of precision-recall curves for boundaries and for objects-and-parts as the tool of choice for the supervised evaluation of image segmentation. We make the datasets and code of all the measures publicly available.