We are submitting the tasks and data we collected for our study investigating the effect of information presentation on developer search behaviour. We are seeking the available and reusable badges for these artifacts as we believe they provide value to the community as we outline below.

Available data: Having access to existing data can help researchers evaluate comparative approaches and explore alternative perspectives that facilitate the development of interesting hypotheses. Researchers are actively investigating approaches that help developers find information for their development tasks. Our multi-faceted dataset, which captures the entire web search process of 40 developers, offers insights into query formulation and navigation behaviour within the context of focused API search tasks. The developers’ solutions, combined with their qualitative feedback, provide insights that future approaches may wish to build upon. For example, with the recent advances in generative AI, researchers could compare information seeking strategies between developers using traditional web search and LLMs. In particular, by comparing search query and prompt refinements with the navigational effort required for developers to locate the appropriate information.

Reusable tasks: Creating experimental tasks which are representative of the tasks developers perform in industry can be challenging. The tasks need to be aligned with the goals of the study, respectful of developers time, and presented with clear descriptions and definitions of done. For our study, we created seven short, but realistic, API search tasks (including a tutorial task) with detailed instructions and test cases. These tasks were refined through multiple rounds within our research group and pilots with 20 developers to ensure sufficient clarity, level of difficulty, and realism. We believe these tasks could be reused by other members of the community either directly or by acting as concrete examples.

All of the files in our artifact are viewable directly within the browser. We conducted our analysis in an RMarkdown notebook using the Tidy Verse library. Our tasks are written in JavaScript but only use basic language features.