Third-party libraries (TPLs) are an integral part of Android app development, offering app developers essential tools for enhancing app functionality, design, and integration capabilities. However, the fast-paced evolution of Android APIs introduces compatibility issues not only in Android apps but also in TPLs as they rely heavily on these Android APIs too. These challenges necessitate continuous updates and compatibility checks to maintain apps as well as TPL’s compatibility across Android’s diverse ecosystem. Prior research primarily focused on detecting compatibility issues induced by native Android APIs in Android apps falling short in detecting the incompatible APIs associated with TPLs due to the additional layer of abstraction they introduce as well as the obfuscation used by the TPL developers. In this paper, we propose LibCT that leverages a pretrained Large Language Model (LLM), GPT-4, for detecting incompatible APIs in Android TPLs. It leverages GPT-4 to generate tests on TPL API usages and executes them across a wide range of Android devices available in the Amazon Device Farm. In our experimental evaluation, we tested 312 libraries with 12,831 APIs on 86 devices in the Amazon device farm, which revealed 274 incompatible APIs, highlighting the LibCT’s capability in identifying both evolution-induced and device-specific compatibility issues.