Seeing confusion through a new lens: on the impact of atoms of confusion on novices' code comprehension.

Code comprehension is crucial for software maintenance and evolution, but it can be hindered by tiny code snippets that can confuse the developers, called atoms of confusion. Previous studies investigated how atoms impact code comprehension through the perspectives of time, accuracy, and opinions of developers. However, we need more studies evaluating other perspectives and the combination of these perspectives on a common ground through experiments. In our study, we evaluate how the eye tracking method can be used to gain new insights when we compare programs obfuscated by the atoms with functionally equivalent clarified versions. We conduct a controlled experiment with 32 novices in Python and measure their time, number of attempts, and visual effort with eye tracking through fixation duration, fixations count, and regressions count. We also conduct interviews and investigate the subjects' difficulties with the programs. In our results, the clarified version of the code with Operator Precedence reduced the time spent in the region that contains the atom to the extent of 38.6%, and the number of answer attempts by 28%. Most subjects found the obfuscated version more difficult to solve than the clarified one, and they reported the order of precedence to be difficult to validate. By analyzing their visual effort, in the obfuscated version, we observed an increase of 47.3% in the horizontal regressions count in the atom region, making its reading more difficult. The additional atoms evaluated revealed other interesting nuances. Based on our findings, we encourage researchers to consider eye tracking combined with other perspectives to evaluate atoms of confusion and educators to favor patterns that do not impact the understanding and visual effort of undergraduates.

Evaluating refactorings for disciplining #ifdef annotations: An eye tracking study with novices.

The C preprocessor is widely used in practice. Conditional compilation with #ifdef annotations allows developers to flexibly introduce variability in their programs. Developers can use disciplined annotations, entirely enclosing full statements with preprocessor directives, or undisciplined ones, enclosing only parts of the statements. Despite some debate, there is no consensus on whether a developer should use exclusively disciplined annotations. While one prior study found undisciplined annotations more time-consuming and error-prone, another study found no difference between disciplined and undisciplined annotations regarding task completion time and accuracy. In this article, we evaluate whether three fine-grained refactorings to discipline #ifdef annotations correlate with improvements in code comprehension and visual effort with an eye tracker. We conduct a controlled experiment with 64 human subjects who were majoritarily novices in the C programming language. We observed statistically significant differences for two refactorings to discipline annotations with respect to the analyzed metrics (time, fixation duration, fixation count, and regressions count) in the code regions changed by each refactoring.