AIOps has become increasingly crucial in managing modern IT infrastructures, leveraging AI techniques to enhance operational efficiency and reliability in log anomaly detection. However, existing approaches, such as Deeplog, face two significant challenges in log anomaly detection: frequent changes in data patterns due to software and hardware upgrades, and the demand for high efficiency in online scenarios. To address these issues, we propose LogX, a novel method based on Large Language Models and optimized prompting strategies, particularly interactive modes, to promptly correct previously unseen errors. By integrating input-label pairs directly into the prompt, LogX eliminates the need for iterative training processes and additional resource costs, ensuring high adaptability in online scenarios. Furthermore, to maintain control over sensitive data while ensuring privacy and security, we utilize open-source tools and on-premise infrastructure for AIOps system. It seamlessly integrates with LogX’s online log diagnostic capabilities, providing a robust solution for companies aiming to manage their software maintenance processes internally.