Artificial intelligence (AI) is becoming an increasingly common feature in software engineering. As AI systems grow in autonomy, they can address progressively complex, multi-step tasks without human interaction. This evolution, however, introduces significant challenges with respect to ensuring quality attributes, such as correctness, completeness, reliability, traceability, and trustworthiness. Given that many AI techniques are commonly considered as black box systems, assurance for AI tends to focus on the inputs and outputs, as well as the execution paths within AI-based workflows and agentic systems. We review the key assurance modes for AI usage and demonstrate them using an example from software modelling. Then we situate our vision within the existing body of related work. Finally, we outline how graph transformation techniques can be used to support and enhance AI assurance.