CD4AI: Regression Testing for AI-Enabled Systems
Download and read the papers before the MiniPLoP
This paper describes the Continuous Delivery for Artificial Intelligence (CD4AI) pattern, a strategy to balance deployment velocity and rigorous validation in AI-enabled systems. The pattern establishes a three-stage lifecycle for these systems: (1) testing: a Continuous Integra- tion / Delivery (CI/CD) step combining automated tests and probabilis- tic evaluations, allowing new versions into production if they succeed at a minimum rate (e.g., 95% success rate); (2) monitoring: a monitoring step that collects real-world interactions for later analysis, flagging po- tential anomalies via automated heuristics; and (3) curation: a feedback loop step to gather candidate interactions that can be converted into new tests, incrementing the regression suite. By leveraging production observability as a source of truth for test generation, this pattern enables teams to maintain high deployment velocity while automatically evolv- ing their test suites to match the complexity of real-world usage. This ensures both the testability and robustness of the system over time.
| PDF of the paper (RegressionTestingForAIEnabledSystems.pdf) | 257KiB |
Wed 8 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
15:40 - 17:40 | Writer's WorkshopWS: Patterns for GenAI at Room 513 Chair(s): Kyle Brown IBM, USA, Douglas C. Schmidt William & Mary, Joseph (Joe) Yoder The Refactory, Inc.
| ||
15:40 20mPaper | Enterprise Tool and Agent Patterns WS: Patterns for GenAI File Attached | ||
16:00 20mPaper | Toward A Pattern Language for Agentic Retrieval Pipelines: From Intent to Adaptive Knowledge Acquisition WS: Patterns for GenAI File Attached | ||
16:20 20mPaper | CD4AI: Regression Testing for AI-Enabled Systems WS: Patterns for GenAI João Paulo Pereira da Silva , Renato Cordeiro Ferreira IME-USP | JADS-TiU/TUe, Alfredo Goldman University of São Paulo File Attached | ||
16:40 20mPaper | A Pattern Hierarchy for Using Generative AI for Task Allocation in Agile Project Management WS: Patterns for GenAI Maan Patel , David Gershony University of California, Irvine, Dustin Lee University of California, Irvine, Joe Hoshina University of California, Irvine, Alberto Krone-Martins UC Irvine, Hadar Ziv File Attached | ||
17:00 20mPaper | Extract Prompt WS: Patterns for GenAI Graham Lee Chiron Codex File Attached | ||
17:20 20mPaper | senpAI: A Pattern for Contextual Onboarding and Knowledge Transfer WS: Patterns for GenAI Sócrates Veridiano Faria Lopes Instituto de Pesquisas Tecnológicas do Estado de São Paulo File Attached | ||