Enhancing Single-Frame Supervision for Better Temporal Action Localization.

Temporal action localization aims to identify the boundaries and categories of actions in videos, such as scoring a goal in a football match. Single-frame supervision has emerged as a labor-efficient way to train action localizers as it requires only one annotated frame per action. However, it often suffers from poor performance due to the lack of precise boundary annotations. To address this issue, we propose a visual analysis method that aligns similar actions and then propagates a few user-provided annotations (e.g., boundaries, category labels) to similar actions via the generated alignments. Our method models the alignment between actions as a heaviest path problem and the annotation propagation as a quadratic optimization problem. As the automatically generated alignments may not accurately match the associated actions and could produce inaccurate localization results, we develop a storyline visualization to explain the localization results of actions and their alignments. This visualization facilitates users in correcting wrong localization results and misalignments. The corrections are then used to improve the localization results of other actions. The effectiveness of our method in improving localization performance is demonstrated through quantitative evaluation and a case study.

Visual Analysis of Displacement Processes in Porous Media using Spatio-Temporal Flow Graphs.

We developed a new approach comprised of different visualizations for the comparative spatio-temporal analysis of displacement processes in porous media. We aim to analyze and compare ensemble datasets from experiments to gain insight into the influence of different parameters on fluid flow. To capture the displacement of a defending fluid by an invading fluid, we first condense an input image series to a single time map. From this map, we generate a spatio-temporal flow graph covering the whole process. This graph is further simplified to only reflect topological changes in the movement of the invading fluid. Our interactive tools allow the visual analysis of these processes by visualizing the graph structure and the context of the experimental setup, as well as by providing charts for multiple metrics. We apply our approach to analyze and compare ensemble datasets jointly with domain experts, where we vary either fluid properties or the solid structure of the porous medium. We finally report the generated insights from the domain experts and discuss our contribution's advantages, generality, and limitations.

A Hybrid in Situ Approach for Cost Efficient Image Database Generation.

The visualization of results while the simulation is running is increasingly common in extreme scale computing environments. We present a novel approach for in situ generation of image databases to achieve cost savings on supercomputers. Our approach, a hybrid between traditional inline and in transit techniques, dynamically distributes visualization tasks between simulation nodes and visualization nodes, using probing as a basis to estimate rendering cost. Our hybrid design differs from previous works in that it creates opportunities to minimize idle time from four fundamental types of inefficiency: variability, limited scalability, overhead, and rightsizing. We demonstrate our results by comparing our method against both inline and in transit methods for a variety of configurations, including two simulation codes and a scaling study that goes above 19 K cores. Our findings show that our approach is superior in many configurations. As in situ visualization becomes increasingly ubiquitous, we believe our technique could lead to significant amounts of reclaimed cycles on supercomputers.

Situated Visual Analysis and Live Monitoring for Manufacturing.

Modern machines continuously log status reports over long periods of time, which are valuable data to optimize working routines. Data visualization is a commonly used tool to gain insights into these data, mostly in retrospective (e.g., to determine causal dependencies between the faults of different machines). We present an approach to bring such visual analyses to the shop floor to support reacting to faults in real time. This approach combines spatio-temporal analyses of time series using a handheld touch device with augmented reality for live monitoring. Important information augments machines directly in their real-world context, and detailed logs of current and historical events are displayed on the handheld device. In collaboration with an industry partner, we designed and tested our approach on a live production line to obtain feedback from operators. We compare our approach for monitoring and analysis with existing solutions that are currently deployed.

Real-Time Visual Analysis of High-Volume Social Media Posts.

Breaking news and first-hand reports often trend on social media platforms before traditional news outlets cover them. The real-time analysis of posts on such platforms can reveal valuable and timely insights for journalists, politicians, business analysts, and first responders, but the high number and diversity of new posts pose a challenge. In this work, we present an interactive system that enables the visual analysis of streaming social media data on a large scale in real-time. We propose an efficient and explainable dynamic clustering algorithm that powers a continuously updated visualization of the current thematic landscape as well as detailed visual summaries of specific topics of interest. Our parallel clustering strategy provides an adaptive stream with a digestible but diverse selection of recent posts related to relevant topics. We also integrate familiar visual metaphors that are highly interlinked for enabling both explorative and more focused monitoring tasks. Analysts can gradually increase the resolution to dive deeper into particular topics. In contrast to previous work, our system also works with non-geolocated posts and avoids extensive preprocessing such as detecting events. We evaluated our dynamic clustering algorithm and discuss several use cases that show the utility of our system.

S4: Self-Supervised Learning of Spatiotemporal Similarity.

We introduce an ML-driven approach that enables interactive example-based queries for similar behavior in ensembles of spatiotemporal scientific data. This addresses an important use case in the visual exploration of simulation and experimental data, where data is often large, unlabeled and has no meaningful similarity measures available. We exploit the fact that nearby locations often exhibit similar behavior and train a Siamese Neural Network in a self-supervised fashion, learning an expressive latent space for spatiotemporal behavior. This space can be used to find similar behavior with just a few user-provided examples. We evaluate this approach on several ensemble datasets and compare with multiple existing methods, showing both qualitative and quantitative results.

The coupled socio-ecohydrological evolution of river systems: Towards an integrative perspective of river systems in the 21st century.

River systems have undergone a massive transformation since the Anthropocene. The natural properties of river systems have been drastically altered and reshaped, limiting the use of management frameworks, their scientific knowledge base and their ability to provide adequate solutions for current problems and those of the future, such as climate change, biodiversity crisis and increased demands for water resources. To address these challenges, a socioecologically driven research agenda for river systems that complements current approaches is needed and proposed. The implementation of the concepts of social metabolism and the colonisation of natural systems into existing concepts can provide a new basis to analyse the coevolutionary coupling of social systems with ecological and hydrological (i.e., 'socio-ecohydrological') systems within rivers. To operationalize this research agenda, we highlight four initial core topics defined as research clusters (RCs) to address specific system properties in an integrative manner. The colonisation of natural systems by social systems is seen as a significant driver of the transformation processes in river systems. These transformation processes are influenced by connectivity (RC 1), which primarily addresses biophysical aspects and governance (RC 2), which focuses on the changes in social systems. The metabolism (RC 3) and vulnerability (RC 4) of the social and natural systems are significant aspects of the coupling of social systems and ecohydrological systems with investments, energy, resources, services and associated risks and impacts. This socio-ecohydrological research agenda complements other recent approaches, such as 'socio-ecological', 'socio-hydrological' or 'socio-geomorphological' systems, by focusing on the coupling of social systems with natural systems in rivers and thus, by viewing the socioeconomic features of river systems as being just as important as their natural characteristics. The proposed research agenda builds on interdisciplinarity and transdisciplinarity and requires the implementation of such programmes into the education of a new generation of river system scientists, managers and engineers who are aware of the transformation processes and the coupling between systems.

Local Prediction Models for Spatiotemporal Volume Visualization.

We present a machine learning-based approach for detecting and visualizing complex behavior in spatiotemporal volumes. For this, we train models to predict future data values at a given position based on the past values in its neighborhood, capturing common temporal behavior in the data. We then evaluate the model's prediction on the same data. High prediction error means that the local behavior was too complex, unique or uncertain to be accurately captured during training, indicating spatiotemporal regions with interesting behavior. By training several models of varying capacity, we are able to detect spatiotemporal regions of various complexities. We aggregate the obtained prediction errors into a time series or spatial volumes and visualize them together to highlight regions of unpredictable behavior and how they differ between the models. We demonstrate two further volumetric applications: adaptive timestep selection and analysis of ensemble dissimilarity. We apply our technique to datasets from multiple application domains and demonstrate that we are able to produce meaningful results while making minimal assumptions about the underlying data.

Visual Neural Decomposition to Explain Multivariate Data Sets.

Investigating relationships between variables in multi-dimensional data sets is a common task for data analysts and engineers. More specifically, it is often valuable to understand which ranges of which input variables lead to particular values of a given target variable. Unfortunately, with an increasing number of independent variables, this process may become cumbersome and time-consuming due to the many possible combinations that have to be explored. In this paper, we propose a novel approach to visualize correlations between input variables and a target output variable that scales to hundreds of variables. We developed a visual model based on neural networks that can be explored in a guided way to help analysts find and understand such correlations. First, we train a neural network to predict the target from the input variables. Then, we visualize the inner workings of the resulting model to help understand relations within the data set. We further introduce a new regularization term for the backpropagation algorithm that encourages the neural network to learn representations that are easier to interpret visually. We apply our method to artificial and real-world data sets to show its utility.

PlotThread: Creating Expressive Storyline Visualizations using Reinforcement Learning.

Storyline visualizations are an effective means to present the evolution of plots and reveal the scenic interactions among characters. However, the design of storyline visualizations is a difficult task as users need to balance between aesthetic goals and narrative constraints. Despite that the optimization-based methods have been improved significantly in terms of producing aesthetic and legible layouts, the existing (semi-) automatic methods are still limited regarding 1) efficient exploration of the storyline design space and 2) flexible customization of storyline layouts. In this work, we propose a reinforcement learning framework to train an AI agent that assists users in exploring the design space efficiently and generating well-optimized storylines. Based on the framework, we introduce PlotThread, an authoring tool that integrates a set of flexible interactions to support easy customization of storyline visualizations. To seamlessly integrate the AI agent into the authoring process, we employ a mixed-initiative approach where both the agent and designers work on the same canvas to boost the collaborative design of storylines. We evaluate the reinforcement learning model through qualitative and quantitative experiments and demonstrate the usage of PlotThread using a collection of use cases.

Foveated Encoding for Large High-Resolution Displays.

Collaborative exploration of scientific data sets across large high-resolution displays requires both high visual detail as well as low-latency transfer of image data (oftentimes inducing the need to trade one for the other). In this work, we present a system that dynamically adapts the encoding quality in such systems in a way that reduces the required bandwidth without impacting the details perceived by one or more observers. Humans perceive sharp, colourful details, in the small foveal region around the centre of the field of view, while information in the periphery is perceived blurred and colourless. We account for this by tracking the gaze of observers, and respectively adapting the quality parameter of each macroblock used by the H.264 encoder, considering the so-called visual acuity fall-off. This allows to substantially reduce the required bandwidth with barely noticeable changes in visual quality, which is crucial for collaborative analysis across display walls at different locations. We demonstrate the reduced overall required bandwidth and the high quality inside the foveated regions using particle rendering and parallel coordinates.

2019 IEEE Scientific Visualization Contest Winner: Visual Analysis of Structure Formation in Cosmic Evolution.

Simulations of cosmic evolution are a means to explain the formation of the universe as we see it today. The resulting data of such simulations comprise numerous physical quantities, which turns their analysis into a complex task. Here, we analyze such high-dimensional and time-varying particle data using various visualization techniques from the fields of particle visualization, flow visualization, volume visualization, and information visualization. Our approach employs specialized filters to extract and highlight the development of so-called active galactic nuclei and filament structures formed by the particles. Additionally, we calculate X-ray emission of the evolving structures in a preprocessing step to complement visual analysis. Our approach is integrated into a single visual analytics framework to allow for analysis of star formation at interactive frame rates. Finally, we lay out the methodological aspects of our work that led to success at the 2019 IEEE SciVis Contest.

PyramidTags: Context-, Time- and Word Order-Aware Tag Maps to Explore Large Document Collections.

It is difficult to explore large text collections if no or little information is available on the contained documents. Hence, starting analytic tasks on such corpora is challenging for many stakeholders from various domains. As a remedy, recent visualization research suggests to use visual spatializations of representative text documents or tags to explore text collections. With PyramidTags, we introduce a novel approach for summarizing large text collections visually. In contrast to previous work, PyramidTags in particular aims at creating an improved representation that incorporates both temporal evolution and semantic relationship of visualized tags within the summarized document collection. As a result, it equips analysts with a visual starting point for interactive exploration to not only get an overview of the main terms and phrases of the corpus, but also to grasp important ideas and stories. Analysts can hover and select multiple tags to explore relationships and retrieve the most relevant documents. In this work, we apply PyramidTags to hundreds of thousands of web-crawled news reports. Our benchmarks suggest that PyramidTags creates time- and context-aware layouts, while preserving the inherent word order of important pairs.

Cyclic RGD and isoDGR Integrin Ligands Containing cis-2-amino-1-cyclopentanecarboxylic (cis-ß-ACPC) Scaffolds.

Integrin ligands containing the tripeptide sequences Arg-Gly-Asp (RGD) and iso-Asp-Gly- Arg (isoDGR) were actively investigated as inhibitors of tumor angiogenesis and directing unit in tumor-targeting drug conjugates. Reported herein is the synthesis, of two RGD and one isoDGR cyclic peptidomimetics containing (1S,2R) and (1R,2S) cis-2-amino-1-cyclopentanecarboxylic acid (cis-ß-ACPC), using a mixed solid phase/solution phase synthetic protocol. The three ligands were examined in vitro in competitive binding assays to the purified αvß3 and α5ß1 receptors using biotinylated vitronectin (αvß3) and fibronectin (α5ß1) as natural displaced ligands. The IC50 values of the ligands ranged from nanomolar (the two RGD ligands) to micromolar (the isoDGR ligand) with a pronounced selectivity for αvß3 over α5ß1. In vitro cell adhesion assays were also performed using the human skin melanoma cell line WM115 (rich in integrin αvß3). The two RGD ligands showed IC50 values in the same micromolar range as the reference compound (cyclo[RGDfV]), while for the isoDGR derivative an IC50 value could not be measured for the cell adhesion assay. A conformational analysis of the free RGD and isoDGR ligands by NMR (VT-NMR and NOESY experiments) and computational studies (MC/EM and MD), followed by docking simulations performed in the αVß3 integrin active site, provided a rationale for the behavior of these ligands toward the receptor.

Stereoselective Synthesis of Tropanes via a 6π-Electrocyclic Ring-Opening/ Huisgen [3+2]-Cycloaddition Cascade of Monocyclopropanated Heterocycles.

The synthesis of tropanes via a microwave-assisted, stereoselective 6π-electrocyclic ring-opening/ Huisgen [3+2]-cycloaddition cascade of cyclopropanated pyrrole and furan derivatives with electron-deficient dipolarophiles is demonstrated. Starting from furans or pyrroles, 8-aza- and 8-oxabicyclo[3.2.1]octanes are accessible in two steps in dia- and enantioselective pure form, being versatile building blocks for the synthesis of pharmaceutically relevant targets, especially for new cocaine analogues bearing various substituents at the C-6/C-7 positions of the tropane ring system. Moreover, the 2-azabicyclo[2.2.2]octane core (isoquinuclidines), being prominently represented in many natural and pharmaceutical products, is accessible via this approach.

Visual Quality Guidance for Document Exploration with Focus+Context Techniques.

Magic lens based focus+context techniques are powerful means for exploring document spatializations. Typically, they only offer additional summarized or abstracted views on focused documents. As a consequence, users might miss important information that is either not shown in aggregated form or that never happens to get focused. In this work, we present the design process and user study results for improving a magic lens based document exploration approach with exemplary visual quality cues to guide users in steering the exploration and support them in interpreting the summarization results. We contribute a thorough analysis of potential sources of information loss involved in these techniques, which include the visual spatialization of text documents, user-steered exploration, and the visual summarization. With lessons learned from previous research, we highlight the various ways those information losses could hamper the exploration. Furthermore, we formally define measures for the aforementioned different types of information losses and bias. Finally, we present the visual cues to depict these quality measures that are seamlessly integrated into the exploration approach. These visual cues guide users during the exploration and reduce the risk of misinterpretation and accelerate insight generation. We conclude with the results of a controlled user study and discuss the benefits and challenges of integrating quality guidance in exploration techniques.

On Evaluating Runtime Performance of Interactive Visualizations.

As our field matures, evaluation of visualization techniques has extended from reporting runtime performance to studying user behavior. Consequently, many methodologies and best practices for user studies have evolved. While maintaining interactivity continues to be crucial for the exploration of large data sets, no similar methodological foundation for evaluating runtime performance has been developed. Our analysis of 50 recent visualization papers on new or improved techniques for rendering volumes or particles indicates that only a very limited set of parameters like different data sets, camera paths, viewport sizes, and GPUs are investigated, which make comparison with other techniques or generalization to other parameter ranges at least questionable. To derive a deeper understanding of qualitative runtime behavior and quantitative parameter dependencies, we developed a framework for the most exhaustive performance evaluation of volume and particle visualization techniques that we are aware of, including millions of measurements on ten different GPUs. This paper reports on our insights from statistical analysis of this data, discussing independent and linear parameter behavior and non-obvious effects. We give recommendations for best practices when evaluating runtime performance of scientific visualization applications, which can serve as a starting point for more elaborate models of performance quantification.

VeCHArt: Visually Enhanced Comparison of Historic Art Using an Automated Line-Based Synchronization Technique.

The analysis of subtle deviations between different versions of historical prints has been a long-standing challenge in art history research. So far, this challenge has required extensive domain knowledge, fine-tuned expert perception, and time-consuming manual labor. In this paper we introduce an explorative visual approach to facilitate fast and accurate support for the task of comparing differences between prints such as engravings and woodcuts. To this end, we have developed a customized algorithm that detects similar stroke-patterns in prints and matches them in order to allow visual alignment and automated deviation highlighting. Our visual analytics system enables art history researchers to quickly detect, document, and categorize qualitative and quantitative discrepancies, and to analyze these discrepancies using comprehensive interactions. To evaluate our approach, we conducted a user study involving both experts on historical prints and laypeople. Using our new interactive technique, our subjects found about 20 percent more differences compared to regular image viewing software as well as "paper-based" comparison. Moreover, the laypeople found the same differences as the experts when they used our system, which was not the case for conventional methods. Informal feedback showed that both laypeople and experts strongly preferred employing our system to working with conventional methods.

A methodical framework for analysing the cause of urban pluvial flooding in a hillside settlement.

Pluvial flood risk is increasing in urban and rural areas due to changes in precipitation patterns and urbanization. Pluvial flooding is often associated with insufficient capacities of the sewer system or low surface drainage efficiency of urban areas. In hilly areas, hillside runoff additionally affects the risk of pluvial flooding. This article introduces a methodical approach and related evaluation criteria for a systematic analysis of potential causes of urban pluvial flooding. In the presented case study, the cause of pluvial flooding at two selected sites in a hillside settlement is investigated based on a coupled 1D/2D model of the whole hydrological catchment. The results show that even though bottlenecks in the sewer system are important, the effect of low surface drainage efficiency and hillside runoff greatly influence pluvial flooding. The knowledge of different causes of flooding can be further used for selecting and positioning appropriate adaption measures. The presented approach proved its practicability and can thus serve as a guidance and template for other applications to gain better understanding and knowledge of local specific pluvial flooding events.

Regio- and Stereoselective Synthesis of Functionalized Dihydropyridines, Pyridines, and 2H-Pyrans: Heck Coupling of Monocyclopropanated Heterocycles.

A palladium-catalyzed coupling between aryl halides and monocyclopropanated pyrroles or furans has been developed, leading to valuable six-membered N- and O-heterocycles. As the key step, a selective cleavage of the non-activated endocyclic C-C bond of the 2-heterobicyclo-[3.1.0]hexane framework is achieved. The developed method offers access to highly functionalized piperidines, pyridines, and pyrans that are challenging to access by traditional methods.