Competition Overview

Unmanned Aerial Vehicles (UAVs) equipped with onboard cameras and various sensors have already demonstrated the possibility of autonomous flights in real environments, leading to great interest in various application scenarios: crop monitoring, surveillance, medical and food delivery. Over the years, support for UAV developers has increased with open-access projects for software and hardware, such as the autopilot support provided by PX4 and Ardupilot. However, despite the necessity of systematically testing such complex and automated systems to ensure their safe operation in real-world environments, there has been relatively limited investment in this direction so far.

The UAV Testing Competition organized jointly by the International Conference on Software Testing, Verification and Validation (ICST) and Search-Based and Fuzz Testing (SBFT) workshop is an initiative designed to inspire and encourage the Software Testing Community to direct their attention toward UAVs as a rapidly emerging and crucial domain. The joint call is meant to help interested authors/participants reduce travel costs by selecting the most convenient and close venue.

The Competition is documented in detail on Github: https://github.com/skhatiri/UAV-Testing-Competition

Competition Goal

In the UAV Testing Competition, we aim to provide software testing researchers with a simple platform to facilitate their onboarding in the UAV domain. Using the provided platform and case study, the goal is to use search-based techniques to generate challenging test cases for autonomous vision-based UAV navigation systems.

• The Software Under Test is PX4-Avoidance, a vision-based autonomous obstacle avoidance system developed on top of PX4-Autopilot.
• We create challenging scenarios for PX4-Avoidance by placing static obstacles on the UAV’s path.
• The ultimate goal is to find some specific obstacle configurations (size, position) that could lead to a crash or unsafe flight by the autopilot.

Competition Guideline

Please find the guidelines for the competition on Github: https://github.com/skhatiri/UAV-Testing-Competition

Please read the report of the previous edition in detail to gain a better understanding of the competition guidelines, evaluation criteria, and process. Some details may vary in the new edition (e.g., the evaluation metrics).

Tool Submission

Follow the Submission Guideline, prepare your code as explained, and send it to the organization committee.

You can freely decide on the venue you want to compete in: ICST 2025 or SBFT@ICSE 2025. Participants in each venue will be evaluated and ranked independently.

Tool Paper Submission

The camera-ready deadline is January 10, 2024 (AoE). Please share your draft with us before 3rd January for a review.

You will be contacted by the publisher with the instructions for preparing and submitting the camera-ready version of your paper. The given page limits (i.e., 2 pages for tool papers) are strict. The list of authors (names, emails, affiliations, order) and the paper title must not change. If a correction is needed, please email us promptly.

References: Since you will be using the tool built on top of our previous work, please include the following references in your paper:

@inproceedings{khatiri2023simulation,
  title={Simulation-based test case generation for unmanned aerial vehicles in the neighborhood of real flights},
  author={Khatiri, Sajad and Panichella, Sebastiano and Tonella, Paolo},
  booktitle={2023 16th IEEE International Conference on Software Testing, Verification and Validation (ICST)},
  year={2023},
}

@inproceedings{icse2024Aerialist,
  title={Simulation-based Testing of Unmanned Aerial Vehicles with Aerialist},
  author={Khatiri, Sajad and Panichella, Sebastiano and Tonella, Paolo},
  booktitle={ International Conference on Software Engineering (ICSE)},
  year={2024},
}

If submitting to SBFT 2025:

@inproceedings{SBFT-UAV2025,
  author       = {Sajad Khatiri and Tahereh Zohdinasab and Prasun Saurabh and Dmytro Humeniuk and Sebastiano Panichella},
  title        = {{SBFT} Tool Competition 2025 - CPS-UAV Test Case Generation Track},
  booktitle    = {{IEEE/ACM} International Workshop on Search-Based and Fuzz Testing, SBFT@ICSE 2025},
  year         = {2025}
}
If submitting to ICST 2025:

@inproceedings{ICST-UAV2025,
  author       = {Sajad Khatiri and Tahereh Zohdinasab and Prasun Saurabh and Dmytro Humeniuk and Sebastiano Panichella},
  title        = {{ICST} Tool Competition 2025 - CPS-UAV Test Case Generation Track},
  booktitle    = {{IEEE/ACM} International Conference on Software Testing, Verification and Validation, ICST 2025},
  year         = {2025}
}

We look forward to receiving your submissions. Thank you for your interest in the competition!

Contacts

• Sajad Khatiri: mazr@zhaw.ch
• Tahereh Zohdinasab: tahereh.zohdinasab@usi.ch
• Prasun Saurabh: saurabhprasun20@gmail.com
• Dmytro Humeniuk: dmytro.humeniuk@polymtl.ca
• Sebastiano Panichella: spanichella@gmail.com