Testing of autonomous UAV systems poses significant challenges due to its complex nature. The complexity lies in creating realistic and, diverse test scenarios. This report documents a method that applies Q-learning combined with Upper Confidence Bound (UCB) to generate obstacle configurations in a simulated environment. The goal is to evaluate the PX4-Avoidance system by inducing unsafe UAV behaviors through generated obstacle placements. This approach enhances fault detection by balancing exploration and exploitation in the test case generation, ultimately increasing scenario diversity and system robustness.