In the era of AI-driven autonomous vehicles (AVs), the integration of machine learning, control systems, and embedded technologies introduces critical software engineering challenges—particularly in cyber-physical system security. Intrusion Detection Systems (IDS) play a pivotal role in safeguarding AVs by detecting anomalies and cyberattacks in real time. This vision paper proposes the design and implementation of deep learning-based IDSs tailored for real-time anomaly detection on autonomous drones, cars, and robots. Our methodology begins with the systematic creation of platform-specific taxonomies of AV software vulnerabilities and a formal threat model. We then plan to conduct controlled experiments to generate labeled datasets of AV anomalies using physical aerial and ground vehicles and industrial-grade simulators operating in both network-connected and isolated environments. These datasets will support the design and training of IDSs customized for each AV platform. Finally, we will deploy these IDSs on physical devices to evaluate their performance and reliability under realistic conditions. This paper highlights key empirical software engineering challenges, including the sim-to-real transfer gap in machine learning, the risk of overfitting in data-driven IDS models, and the complexities of hardware-software integration in AV systems. Anticipated contributions include: (i) detailed, platform-specific taxonomies of AV software vulnerabilities; (ii) multiple labeled datasets capturing both normal and compromised AV behaviors; and (iii) a family of validated, deployable IDS frameworks for securing AVs in dynamic environments.