CAIN 2025
Sun 27 - Mon 28 April 2025 Ottawa, Ontario, Canada
co-located with ICSE 2025
Jeremy Barnes

Jeremy Barnes

VP AI Product at ServiceNow

Is Prompt Engineering?

Abstract

Have you ever pleaded with a large language model to follow its prompt more precisely by asking it to be more helpful, repeating yourself, SHOUTING AT IT, manipulating or threatening it? Can we call this "software engineering" and what can we guarantee about the resulting systems? Large language models enable software to be built in powerful new ways, but their use turns many notions of software engineering on their head. We explore how to maintain an engineering mindset whilst taking advantage of new opportunities with language models, based on a decade of experience researching and developing LLMs and incorporating them into shipping products.

Bio

Jeremy has spent 25 years building real-world applications using machine learning and AI. For at least 20 of those years, he has maintained that it would be the next big thing in "a year or two". He has founded and served as CTO, CEO and led product management of many companies in the ML / AI universe, doing natural language processing, recommendation engines, real-time advertising and enterprise workflows where he has served as CTO and CEO. Notably, he was CTO of Element AI and was the VP responsible for AI at ServiceNow for the two years following the release of ChatGPT. He's based in Montréal, Canada.

Marcos Kalinowski

Prof. Marcos Kalinowski

Professor of Software Engineering at the Pontifical Catholic University of Rio de Janeiro

Beyond Machine Learning and Foundation Models: A Research Roadmap for Multi-Paradigm AI Engineering

Abstract

Rapid advancements in AI paradigms—Machine Learning, Reinforcement Learning, Symbolic AI, Foundation Models, and Multi-Agent Systems—have expanded AI’s capabilities while introducing new software engineering challenges. Each paradigm has inherent limitations: Machine Learning excels at pattern recognition but lacks explicit reasoning; Reinforcement Learning optimizes rewards but is unpredictable; Symbolic AI provides structure but struggles with flexibility; Foundation Models generalize knowledge but raise interpretability concerns; and Multi-Agent Systems enable distributed intelligence but introduce emergent behaviors. Just as the human brain integrates perception, learning, reasoning, and decision-making, AI systems can combine multiple paradigms to achieve adaptable intelligence. While AI Engineering has made progress in building robust Machine Learning systems, our understanding of Intelligence Engineering—the integration of multiple AI paradigms—remains limited. Furthermore, software engineering aspects such as requirements and architecture are still poorly understood across different AI paradigms and their combinations. Without deeper insights and systematic methodologies, we still lack the necessary means to build trustworthy, scalable, and maintainable intelligent systems that effectively integrate these paradigms. This talk presents a research roadmap for AI Engineering, highlighting the urgent need for evidence-based intelligence engineering and the development of systematic specification approaches, architectural patterns, and tactics for multi-paradigm AI integration. It also underscores the critical role of empirical software engineering in navigating the hype responsibly and reinforcing AI Engineering as an evidence-driven discipline.

Bio

Marcos Kalinowski is a Professor of Software Engineering at PUC-Rio, Brazil. His research focuses on AI Engineering, Experimental Software Engineering, and Human Aspects of Software Engineering. Before transitioning to academia, he accumulated over a decade of industry experience. He co-leads the ExACTa PUC-Rio lab, where he supervises PhD research and leads R&D projects with industry partners to develop AI engineering solutions and real-world AI systems. His lab's work has resulted in award-winning AI deployments and multiple US patents. He has authored several books and over 200 research publications, earning 19 distinguished paper awards, including ACM Distinguished Paper Awards at ICSE and CAIN. He holds a research productivity distinction from the Brazilian Research Council (CNPq) and an honorary 'Scientist of the State' distinction from Rio de Janeiro's state research agency (FAPERJ). He serves the community as associate editor of the Journal of Systems and Software and as part of the organizing and program committees of several international conferences. He is an active member of ACM, IEEE, ISERN, and the Brazilian Computer Society.