Effective performance testing is essential for ensuring the reliability of large-scale software systems under varying load conditions. However, conventional workload generation techniques often rely on static thresholds and rule-based heuristics, which lack adaptability and generalizability across diverse systems. This paper presents PerfScout, a reinforcement learning–based framework for adaptive workload generation that automates the performance testing process. PerfScout integrates three components: dynamic breaking point identification using SPOT, local stationarity assessment via ADF and KPSS tests, and a policy optimization module based on Proximal Policy Optimization (PPO). By continuously observing key performance indicators and adapting the testing workload in real time, PerfScout enhances the automation of performance testing while improving testing efficiency. PerfScout has been deployed in Huawei Cloud, and experimental results on two datasets collected from its industrial environment demonstrate that PerfScout achieves over 82% accuracy in breaking point identification and improves testing efficiency by up to 90%, underscoring its practical applicability and effectiveness.