Any problem in code may have multiple solutions that differ in details large and small. Because modern software development is characterized by an abundance of searchable and reusable code, effective developers must be able to judge not only the meaning of new algorithms but also the differences between alternatives. Therefore, we use a multi-method study to explore how developers perform comparative comprehension— the cognitive activity of understanding how algorithms behave relative to each other.

To explore how developers compare code, we performed a controlled experiment with 16 developers in a mixed think-aloud and interview format and another 95 developers in a survey format. In this experiment, participants investigated whether a pair of code snippets would demonstrate equivalent behavior when run, controlling for differences in behavior, programming languages, algorithmic structures, and meaningful names. Overall, our results describe how comparison fits into learning, reviewing, and reusing code. Our task observations shed light on how developers move between code similarities at different levels—textual, structural, and schematic—when simultaneously inspecting multiple snippets. In our experiment, developers made more accurate conclusions about behavior given similar languages and structures, with names acting as additional evidence in interaction with other cues, but they also overestimated whether behavior is equivalent in many cases. From this, we identify challenges developers face in comprehending alternatives and we highlight opportunities to better support developers in comparison activities.