This session will outline the various performance engineering challenges as one optimizes the performance of an AIware for production. Topics covered include:

• Transformer decoding 101
• Single model serving challenges and the state-of-the-art (e.g., vllm)
• End-to-end performance engineering challenges in the AIware development lifecycle
• Latency considerations for different cognitive architectures
• Multi-model pipeline serving challenges and vision
• Curated SPE challenges for AIware
• Semantic caching and FM routers