ICSE 2026
Sun 12 - Sat 18 April 2026 Rio de Janeiro, Brazil
Tue 14 Apr 2026 15:05 - 15:20 at Bora Bora III - Session 3 Chair(s): Sidong Feng

Autonomous LLM-based GUI agents demonstrate the potential to revolutionize mobile task automation, but they also introduce significant privacy risks due to their comprehensive access to on-screen user data. Standard architectures often leak Personally Identifiable Information (PII) by sending raw UI data to external LLM services. We address this vulnerability by designing, implementing, and evaluating \textit{PrivAuto}, a mobile GUI agent framework featuring a \textit{Perceive-Scan-Plan-Execute} loop. The core privacy-preserving ability lies in the on-device \textit{Scan} stage, which uses a lightweight model to sanitize UI data before it leaves the device, creating a practical privacy firewall. Empirical evaluation on the \textit{AndroidWorld} benchmark demonstrates the feasibility of this approach but reveals critical challenges. While effective at redacting Critical PII (>90% accuracy), the module struggles with Contextual PII (<30%) and is prone to failures in some typical scenarios. These findings offer crucial insights into the complex trade-offs between privacy and usability, highlighting the need for domain-specific PII identification models and robust engineering to ensure trustworthy LLM-based GUI agents.

Tue 14 Apr

Displayed time zone: Brasilia, Distrito Federal, Brazil change

14:00 - 15:30
Session 3UISE at Bora Bora III
Chair(s): Sidong Feng Monash University
14:00
50m
Keynote
Engineering Adaptive Systems that Learn from Human Interaction
UISE
Silvia Abrahão Universitat Politècnica de València
14:50
15m
Talk
Towards Human-in-the-loop User Interface Adaptation with Reinforcement Learning
UISE
Daniel Gaspar Figueiredo Universitat Politècnica de València, Spain, Silvia Abrahão Universitat Politècnica de València, Emilio Insfran Universitat Politècnica de València, Spain
15:05
15m
Talk
PrivAuto: Practical Insights of On-Device Privacy-Preserving Agentic Systems for Mobile GUI Automation
UISE
Shen Hu Technical University of Munich, Evtim Kostadinov Technical University of Munich, Han Wang , Ludwig Felder Technical University of Munich