Lessons Learned From Quantitatively Exploring Visualization Rubric Utilization for Peer Feedback.

We present our experience of adapting a rubric for peer feedback in our data visualization course and exploring the utilization of that rubric by students across two semesters. We first discuss the results of an automatable quantitative analysis of the rubric responses, and then compare those results to a qualitative analysis of summative survey responses from students regarding the rubric and peer-feedback process. We conclude with lessons learned about the visualization rubric we used, as well as what we learned more broadly about using quantitative analysis to explore this type of data. These lessons may be useful for other educators wanting to utilize the same data visualization rubric, or wanting to explore the utilization of rubrics already deployed for peer feedback.

Unboxing cluster heatmaps.

BACKGROUND: Cluster heatmaps are commonly used in biology and related fields to reveal hierarchical clusters in data matrices. This visualization technique has high data density and reveal clusters better than unordered heatmaps alone. However, cluster heatmaps have known issues making them both time consuming to use and prone to error. We hypothesize that visualization techniques without the rigid grid constraint of cluster heatmaps will perform better at clustering-related tasks. RESULTS: We developed an approach to "unbox" the heatmap values and embed them directly in the hierarchical clustering results, allowing us to use standard hierarchical visualization techniques as alternatives to cluster heatmaps. We then tested our hypothesis by conducting a survey of 45 practitioners to determine how cluster heatmaps are used, prototyping alternatives to cluster heatmaps using pair analytics with a computational biologist, and evaluating those alternatives with hour-long interviews of 5 practitioners and an Amazon Mechanical Turk user study with approximately 200 participants. We found statistically significant performance differences for most clustering-related tasks, and in the number of perceived visual clusters. Visit git.io/vw0t3 for our results. CONCLUSIONS: The optimal technique varied by task. However, gapmaps were preferred by the interviewed practitioners and outperformed or performed as well as cluster heatmaps for clustering-related tasks. Gapmaps are similar to cluster heatmaps, but relax the heatmap grid constraints by introducing gaps between rows and/or columns that are not closely clustered. Based on these results, we recommend users adopt gapmaps as an alternative to cluster heatmaps.