Runtime verification (RV) uses monitors, which are dynamically created based on formal specifications (specs), to check running programs against specs. RV of passing tests in many open-source projects found hundreds of new bugs. But, high overheads make it hard to use RV for testing in practice. We propose Valg, the first on-the-fly selective RV technique for testing, and the first to use reinforcement learning (RL) to speed up RV. Valg leverages a recent finding: 99.87% of monitors are redundant for testing; they wastefully re-check unique traces—sequences of events, e.g., method calls—that the other necessary 0.13% already checked. Valg uses feedback about redundancy of prior monitors and events to selectively monitor only necessary ones subsequently. A key idea in Valg is our novel formulation of selective monitor creation as a two-armed bandit RL problem that rewards necessary monitors, and penalizes redundant ones. We implement Valg for Java and compare it with state-of-the-art RV tools on one revision each of 64 open-source projects. With default RL hyperparameters, Valg is up to 20.2x and 555.6x faster than JavaMOP and TraceMOP, respectively. For example, Valg takes only 54.8 minutes in total to monitor three projects where TraceMOP takes 3.02 days in total. Valg finds 99.6% of JavaMOP and TraceMOP’s spec violations, but it only checks 76.7% of their unique traces on average. After tuning hyperparameters, Valg checks 95.1% of unique traces on average with little loss in speed. Using tuned hyperparameters from one revision “into the future” as code evolves preserves Valg’s high rate of checked unique traces and speedups, without needing frequent re-tuning.