Programmers ask complex questions in their search for solutions during software development. Along with traditional tools such as debuggers and profilers, state-of-the-art approaches like Babylonian Programming can help programmers answer those questions through interactive and visual feedback. Large language models (LLMs) and programming agents are part of programmers’ toolboxes and are well-integrated into their development workflows. However, they are not yet helpful in considering questions involving run-time behavior.

In this paper, we first review the literature to identify concerns programmers face during development and highlight how humans usually address them. We then focus on questions about program behavior and propose integrating Babylonian-style programming techniques with LLMs to help answer related questions. Finally, we suggest four key properties that future LLM-based development tools should support: (1) LLM tool usage traceability for explainability, (2) resumability of development progress for handovers between human programmers and LLM-based programming agents, (3) context efficiency through selective data querying, and (4) multi-source synthesis for tool integration.