Most A/B testing frameworks in the industry are based on randomized allocation of users to control and test groups. Such systems are not suitable to test infrastructure changes, and engineers in charge of such releases have developed practical solutions to manage releases. At Facebook, a common approach is to roll out the new “feature” to a subset of traffic on multiple staging environments, and make a decision based on observed metrics. This approach is subjective and manual. To improve the efficiency of the process for Facebook Video Infrastructure team (“Video Infra”), we developed Shadow Experiment Analysis Tool (SEAT), a framework containing hypothesis testing library and visualization tool. SEAT was adopted and proven to be successful in a variety of scenarios, ranging from deployment A/B testing analysis to service healthcheck at scale. Most recently, it was also integrated into end-to-end Media Validation platform where SEAT helps verify system, network, playback and computer vision metrics. With SEAT, teams are able to catch issues earlier in the process and can prevent them from impacting end users.