TokenScope: Token-Level Explainability and Interpretability for Code-Oriented Tasks in Large Language Models
Understanding how Large Language Models (LLMs) make token-level decisions during code generation remains a major challenge for both researchers and practitioners. While recent tools provide insights into model internals or generation outcomes, they often lack decoding-time signals, fine-grained uncertainty measures, and interactive mechanisms for exploring alternative generation paths. We present TokenScope, an interactive interpretability and analysis tool for decoder-based LLMs that exposes token-level metrics, attention patterns, and structural information during generation. TokenScope supports interactive token replacement, counterfactual branching, and code-aware aggregation via abstract syntax trees. By unifying decoding-time signals with structural program analysis, TokenScope enables systematic investigation of LLM behaviour during code generation.
Wed 8 JulDisplayed time zone: Eastern Time (US & Canada) change
14:00 - 15:30 | Code and LLM 1Research Papers / Ideas, Visions and Reflections / Tool Demonstrations / Industry Papers at MB 3.270 Chair(s): Zhijie Wang Concordia University | ||
14:00 20mTalk | Does In-IDE Calibration of Large Language Models work at Scale? Industry Papers Roham Koohestani JetBrains Research & Delft University of Technology, Agnia Sergeyuk JetBrains Research, David Gros University of California, Davis, Claudio Spiess University of California, Davis, Sergey Titov JetBrains Research, Prem Devanbu University of California at Davis, Mali Izadi Google & TU Delft | ||
14:20 10mTalk | Projectional Decoding: Towards Semantic-Aware LLM Generation Ideas, Visions and Reflections Boqi Chen University of Ottawa, José Antonio Hernández López Department of Computer Science and Systems, University of Murcia, Aren Babikian University of Toronto | ||
14:30 10mTalk | The Stylistic Blind Spot: Uncovering the Hidden Implicit Bias of Coding Style on LLM Code Evaluation Ideas, Visions and Reflections DOI | ||
14:40 10mTalk | TokenScope: Token-Level Explainability and Interpretability for Code-Oriented Tasks in Large Language Models Tool Demonstrations Amirreza Esmaeili University of British Columbia, Fatemeh Hendijani Fard University of British Columbia, Okanagan | ||
14:50 20mTalk | Bash-Commenter: Leveraging Syntax-Aware Preference Optimization to Reinforce Large Language Model for Bash Code Comment Generation Research Papers Lei Yu Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Jingyuan Zhang Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Xin Wang Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Li Yang Institute of Software, Chinese Academy of Sciences, Fengjun Zhang Institute of Software, Chinese Academy of Sciences, China, Peng Wang Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Jia Xu Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Jiajia Ma Institute of Software, Chinese Academy of Sciences, China | ||