Alerts, which record details about system failures, are crucial data for monitoring a online service system. Due to the complex correlation between system components, a system failure usually triggers a large number of alerts, making the traditional manual handling of alerts insufficient. Thus, automatically summarizing alerts is a problem demanding prompt solution. This paper tackles this challenge through a novel approach based on supervised learning. The proposed approach, OAS (Online Alert Summarizing), first learns two types of information from alerts, semantic information and behavior information, respectively. Then, OAS adopts a specific deep learning model to aggregate semantic and behavior representations of alerts and thus determines the correlation between alerts. OAS is able to summarize the newly reported alert online. Extensive experiments, which are conducted on real alert datasets from two large commercial banks, demonstrate the efficiency and the effectiveness of OAS.