With over 760 stars on GitHub and being part of the official ONNX repository, the ONNX Optimizer is the default tool for applying graph-based optimizations to ONNX models. Despite its widespread use, its ability to maintain model accuracy during optimization has not been thoroughly investigated. In this work, we present DiTOX, a utility designed to automatically and comprehensively evaluate the correctness of the ONNX Optimizer. DiTOX adopts a straightforward yet powerful differential testing, \nick{fault localization,} and evaluation methodology, which can be readily adapted for use with other compiler optimizers. Specifically, DiTOX takes a collection of ONNX models, applies optimizations, and executes both the original and optimized versions across a user-defined input set, automatically capturing any issues encountered during optimization. When discrepancies in accuracy arise, DiTOX iteratively isolates the responsible optimization pass by repeating the process at a finer granularity. We applied DiTOX to 130 well-known models from the official ONNX Model Hub, spanning diverse tasks including classification, object detection, semantic segmentation, text summarization, question answering, and sentiment analysis. Our evaluation revealed that $9.2$% of the model instances either caused the optimizer to crash or led to the generation of invalid models using default optimization strategies. Additionally, $30$% of classification models and $16.6$% of object detection and segmentation models exhibited differing outputs across original and optimized versions, whereas models focused on text-related tasks were generally robust to optimization. In total, DiTOX uncovered 15 issues, 14 of which were previously unknown—related to optimizer crashes and accuracy regressions, impacting 9 of the 47 available optimization passes as well as the optimizer more broadly. All findings were reported to the ONNX Optimizer development team. DiTOX demonstrates a simple but effective framework for validating AI model optimizers and with minimal work, is applicable beyond the ONNX ecosystem.