The integration of AI into Cyber-Physical Systems (CPS) has enhanced their functionality but introduced challenges for traditional verification methods. Temporal logic falsification techniques, which are designed for deterministic models, struggle with the complexity of AI-driven systems. To address these challenges, this Ph.D. dissertation focuses on two primary directions: (i) conduct an empirical analysis to categorize CPS models, identify verification challenges specific to AI systems, and (ii) propose a novel falsification method that combines stochastic optimization and reinforcement learning to improve fault detection in AI-enabled CPS. This research is expected to contribute significantly to the CPS verification domain by improving the accuracy and efficiency of the verification process and providing a public data set to support further research.