CAIN 2025
Sun 27 - Mon 28 April 2025 Ottawa, Ontario, Canada
co-located with ICSE 2025

This program is tentative and subject to change.

Sun 27 Apr 2025 11:45 - 12:00 at 208 - session 2

Context. While on-device LLMs offer higher privacy over their remotely-hosted counterparts and do not require Internet connectivity, their energy consumption on the client device still remains insufficiently investigated.

Goal. This study empirically evaluates the energy usage of client devices when fetching LLM-generated content on-device versus from a remote server. Our goal is to help software developers make informed decisions on the most energy-efficient method for fetching content in different scenarios, so as to optimize the client device’s energy consumption.

Method. We conduct a controlled experiment with seven LLMs with varying parameter sizes running on a MacBook Pro M2 and on a remote server. The experiment involves fetching content of different lengths from the LLMs deployed either on-device or remotely, while measuring the client device’s energy usage and performance metrics such as execution time, CPU, GPU, and memory usage.

Results. Fetching LLM-generated content from a remote server uses 4 to 9 times less energy compared to the on-device method, with a large effect size. We observe a consistent strong positive correlation between energy usage and execution time across all content lengths and fetch methods. For the on-device method, GPU and memory usage are positively correlated with energy usage.

Conclusions. We recommend offloading LLM-generated content to a remote server rather than generating it on-device to optimize energy efficiency on the client side. Developers should optimize on-device LLMs to decrease execution time, GPU usage, and memory usage.

This program is tentative and subject to change.

Sun 27 Apr

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
11:00
15m
Talk
How Do Model Export Formats Impact the Development of ML-Enabled Systems? A Case Study on Model Integration
Research and Experience Papers
Shreyas Kumar Parida ETH Zurich, Ilias Gerostathopoulos Vrije Universiteit Amsterdam, Justus Bogner Vrije Universiteit Amsterdam
11:15
15m
Talk
MLScent: A tool for Anti-pattern detection in ML projects
Research and Experience Papers
Karthik Shivashankar University of Oslo, Antonio Martini University of Oslo
11:30
15m
Talk
RAGProbe: Breaking RAG Pipelines with Evaluation Scenarios
Research and Experience Papers
Shangeetha Sivasothy Applied Artificial Intelligence Institute, Deakin University, Scott Barnett Deakin University, Australia, Stefanus Kurniawan Deakin University, Zafaryab Rasool Applied Artificial Intelligence Institute, Deakin University, Rajesh Vasa Deakin University, Australia
11:45
15m
Talk
On-Device or Remote? On the Energy Efficiency of Fetching LLM-Generated Content
Research and Experience Papers
Vince Nguyen Vrije Universiteit Amsterdam, Hieu Huynh Vrije Universiteit Amsterdam, Vidya Dhopate Vrije Universiteit Amsterdam, Anusha Annengala Vrije Universiteit Amsterdam, Hiba Bouhlal Vrije Universiteit Amsterdam, Gian Luca Scoccia Gran Sasso Science Institute, Matias Martinez Universitat Politècnica de Catalunya (UPC), Vincenzo Stoico Vrije Universiteit Amsterdam, Ivano Malavolta Vrije Universiteit Amsterdam
12:00
15m
Talk
Rule-Based Assessment of Reinforcement Learning Practices Using Large Language Models
Research and Experience Papers
Evangelos Ntentos University of Vienna, Stephen John Warnett University of Vienna, Uwe Zdun University of Vienna
12:15
15m
Talk
Investigating Issues that Lead to Code Technical Debt in Machine Learning Systems
Research and Experience Papers
Rodrigo Ximenes Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Antonio Pedro Santos Alves Pontifical Catholic University of Rio de Janeiro, Tatiana Escovedo Pontifical Catholic University of Rio de Janeiro, Rodrigo Spinola Virginia Commonwealth University, Marcos Kalinowski Pontifical Catholic University of Rio de Janeiro (PUC-Rio)
:
:
:
: