From Information to Choice: A Critical Inquiry Into Visualization Tools for Decision Making.

In the face of complex decisions, people often engage in a three-stage process that spans from (1) exploring and analyzing pertinent information (intelligence); (2) generating and exploring alternative options (design); and ultimately culminating in (3) selecting the optimal decision by evaluating discerning criteria (choice). We can fairly assume that all good visualizations aid in the "intelligence" stage by enabling data exploration and analysis. Yet, to what degree and how do visualization systems currently support the other decision making stages, namely "design" and "choice"? To further explore this question, we conducted a comprehensive review of decision-focused visualization tools by examining publications in major visualization journals and conferences, including VIS, EuroVis, and CHI, spanning all available years. We employed a deductive coding method and in-depth analysis to assess whether and how visualization tools support design and choice. Specifically, we examined each visualization tool by (i) its degree of visibility for displaying decision alternatives, criteria, and preferences, and (ii) its degree of flexibility for offering means to manipulate the decision alternatives, criteria, and preferences with interactions such as adding, modifying, changing mapping, and filtering. Our review highlights the opportunities and challenges that decision-focused visualization tools face in realizing their full potential to support all stages of the decision making process. It reveals a surprising scarcity of tools that support all stages, and while most tools excel in offering visibility for decision criteria and alternatives, the degree of flexibility to manipulate these elements is often limited, and the lack of tools that accommodate decision preferences and their elicitation is notable. Based on our findings, to better support the choice stage, future research could explore enhancing flexibility levels and variety, exploring novel visualization paradigms, increasing algorithmic support, and ensuring that this automation is user-controlled via the enhanced flexibility I evels. Our curated list of the 88 surveyed visualization tools is available in the OSF link (https://osf.io/nrasz/?view_only=b92a90a34ae241449b5f2cd33383bfcb).

Scientometric Analysis of Interdisciplinary Collaboration and Gender Trends in 30 Years of IEEE VIS Publications.

We present the results of a scientometric analysis of 30 years of IEEE VIS publications between 1990-2020, in which we conducted a multifaceted analysis of interdisciplinary collaboration and gender composition among authors. To this end, we curated BiblioVIS, a bibliometric dataset that contains rich metadata about IEEE VIS publications, including 3032 articles and 6113 authors. One of the main factors differentiating BiblioVIS from similar datasets is the authors' gender and discipline data, which we inferred through iterative rounds of computational and manual processes. Our analysis shows that, by and large, inter-institutional and interdisciplinary collaboration has been steadily growing over the past 30 years. However, interdisciplinary research was mainly between a few fields, including Computer Science, Engineering and Technology, and Medicine and Health disciplines. Our analysis of gender shows steady growth in women's authorship. Despite this growth, the gender distribution is still highly skewed, with men dominating ( ≈ 75%) of this space. Our predictive analysis of gender balance shows that if the current trends continue, gender parity in the visualization field will not be reached before the third quarter of the century ( ≈ 2070). Our primary goal in this work is to call the visualization community's attention to the critical topics of collaboration, diversity, and gender. Our research offers critical insights through the lens of diversity and gender to help accelerate progress towards a more diverse and representative research community.

From Invisible to Visible: Impacts of Metadata in Communicative Data Visualization.

Leaving the context of visualizations invisible can have negative impacts on understanding and transparency. While common wisdom suggests that recontextualizing visualizations with metadata (e.g., disclosing the data source or instructions for decoding the visualizations' encoding) may counter these effects, the impact remains largely unknown. To fill this gap, we conducted two experiments. In Experiment 1, we explored how chart type, topic, and user goal impacted which categories of metadata participants deemed most relevant. We presented 64 participants with four real-world visualizations. For each visualization, participants were given four goals and selected the type of metadata they most wanted from a set of 18 types. Our results indicated that participants were most interested in metadata which explained the visualization's encoding for goals related to understanding and metadata about the source of the data for assessing trustworthiness. In Experiment 2, we explored how these two types of metadata impact transparency, trustworthiness and persuasiveness, information relevance, and understanding. We asked 144 participants to explain the main message of two pairs of visualizations (one with metadata and one without); rate them on scales of transparency and relevance; and then predict the likelihood that they were selected for a presentation to policymakers. Our results suggested that visualizations with metadata were perceived as more thorough than those without metadata, but similarly relevant, accurate, clear, and complete. Additionally, we found that metadata did not impact the accuracy of the information extracted from visualizations, but may have influenced which information participants remembered as important or interesting.

Designing With Pictographs: Envision Topics Without Sacrificing Understanding.

Past studies have shown that when a visualization uses pictographs to encode data, they have a positive effect on memory, engagement, and assessment of risk. However, little is known about how pictographs affect one's ability to understand a visualization, beyond memory for values and trends. We conducted two crowdsourced experiments to compare the effectiveness of using pictographs when showing part-to-whole relationships. In Experiment 1, we compared pictograph arrays to more traditional bar and pie charts. We tested participants' ability to generate high-level insights following Bloom's taxonomy of educational objectives via 6 free-response questions. We found that accuracy for extracting information and generating insights did not differ overall between the two versions. To explore the motivating differences between the designs, we conducted a second experiment where participants compared charts containing pictograph arrays to more traditional charts on 5 metrics and explained their reasoning. We found that some participants preferred the way that pictographs allowed them to envision the topic more easily, while others preferred traditional bar and pie charts because they seem less cluttered and faster to read. These results suggest that, at least in simple visualizations depicting part-to-whole relationships, the choice of using pictographs has little influence on sensemaking and insight extraction. When deciding whether to use pictograph arrays, designers should consider visual appeal, perceived comprehension time, ease of envisioning the topic, and clutteredness.

Designing Technology for Sociotechnical Problems: Challenges and Considerations.

Designing technology for sociotechnical problems is challenging due to the heterogeneity of stakeholders' needs, the diversity among their values and perspectives, and the disparity in their technical skills. Careful considerations are needed to ensure that data collection is inclusive and representative of the target populations. It is also important to employ data analysis methods that are compatible with users' technical skills and are capable of drawing a representative picture of people's values, priorities, and needs. However, current technical solutions often fail to meet these critical requirements. In this article, we present a set of empirically-driven design considerations for building technological interventions to address sociotechnical issues. We then discuss open challenges and tradeoffs around privacy, ethics, bias, uncertainty, and trust. We conclude with a call to action for researchers to advance the domain knowledge and improve our technological arsenal for addressing sociotechnical problems.

Visualizing Dimension Coverage to Support Exploratory Analysis.

Data analysis involves constantly formulating and testing new hypotheses and questions about data. When dealing with a new dataset, especially one with many dimensions, it can be cumbersome for the analyst to clearly remember which aspects of the data have been investigated (i.e., visually examined for patterns, trends, outliers etc.) and which combinations have not. Yet this information is critical to help the analyst formulate new questions that they have not already answered. We observe that for tabular data, questions are typically comprised of varying combinations of data dimensions (e.g., what are the trends of Sales and Profit for different Regions?). We propose representing analysis history from the angle of dimension coverage (i.e., which data dimensions have been investigated and in which combinations). We use scented widgets [30] to incorporate dimension coverage of the analysts' past work into interaction widgets of a visualization tool. We demonstrate how this approach can assist analysts with the question formation process. Our approach extends the concept of scented widgets to reveal aspects of one's own analysis history, and offers a different perspective on one's past work than typical visualization history tools. Results of our empirical study showed that participants with access to embedded dimension coverage information relied on this information when formulating questions, asked more questions about the data, generated more top-level findings, and showed greater breadth of their analysis without sacrificing depth.

Supporting Communication and Coordination in Collaborative Sensemaking.

When people work together to analyze a data set, they need to organize their findings, hypotheses, and evidence, share that information with their collaborators, and coordinate activities amongst team members. Sharing externalizations (recorded information such as notes) could increase awareness and assist with team communication and coordination. However, we currently know little about how to provide tool support for this sort of sharing. We explore how linked common work (LCW) can be employed within a `collaborative thinking space', to facilitate synchronous collaborative sensemaking activities in Visual Analytics (VA). Collaborative thinking spaces provide an environment for analysts to record, organize, share and connect externalizations. Our tool, CLIP, extends earlier thinking spaces by integrating LCW features that reveal relationships between collaborators' findings. We conducted a user study comparing CLIP to a baseline version without LCW. Results demonstrated that LCW significantly improved analytic outcomes at a collaborative intelligence task. Groups using CLIP were also able to more effectively coordinate their work, and held more discussion of their findings and hypotheses. LCW enabled them to maintain awareness of each other's activities and findings and link those findings to their own work, preventing disruptive oral awareness notifications.