Automated test generation tools often produce assertions that reflect implemented behavior, limiting their usage to regression testing. In this paper, we propose LLMProphet, a black-box approach that trains LLMs on automatically generated regression tests using Few-Shot Learning to identify non-regression faults without relying on source code. By employing iterative cross-validation and a leave-one-out strategy, LLMProphet identifies regression assertions that are misaligned with expected behaviors. We outline LLMProphet’s workflow, feasibility, and preliminary findings, demonstrating its potential for LLM-driven fault detection.