One of the trends in the evolution of Android malware is the increasing diversity of malicious behaviors, such as SMS-related and Internet-related actions. Traditional binary or family-based classification methods are inadequate for fine-grained detection of these behaviors. Thus, multi-label classification is required to identify various malicious behaviors within a single malware sample. This paper employs an active learning strategy to add multi-behavior labels to large-scale datasets based on expert-annotated small-scale datasets. To address the issue of noisy labels (simulating real-world mislabeling), we propose CalmDroid, an active learning framework utilizing the core-set strategy, instead of the confuse-set strategy for updating the model with out-of-distribution (OOD) points. We evaluate CalmDroid’s performance using the Drebin and VirusShare datasets. Experimental results demonstrate that CalmDroid achieves superior detection performance under varying noise conditions, with an accuracy improvement of up to 0.704 compared to the confuse-set strategy. In high-noise environments (15%), it reaches detection accuracy as high as 0.944. Additionally, we validate CalmDroid’s capability to detect evolving malware. Despite behavioral evolution in Drebin malware across different time steps, CalmDroid consistently achieves detection rates above 70% in the newest time step.