Villanelle is an interactive fiction authoring tool designed to support autonomous non-player characters, or “character AI.” Character AI is notoriously challenging for interactive fiction authors to develop, especially for authors approaching interactive fiction from a writing rather than programming background. This paper describes a participatory design process in which we assess the author experience with Villanelle and build a new tool iteration to support their needs. The results of our first user study demonstrate the strong potential of Villanelle’s incorporation of behavior trees as an easy-to-learn computational model for character AI, but they also indicate syntax challenges for inexperienced programmers. Consequently, we developed a block-based programming interface for Villanelle and recruited a new set of study participants to evaluate this iteration using the same study instruments. The results indicate improvements in Villanelle’s usability and creativity support for inexperienced programmers.