Building structured and reusable security knowledge from unstructured threat reports is a challenging and predominantly manual task in the AI security domain. Although existing knowledgebases systematically organize adversarial threats, their reliance on manual curation by security experts limits scalability and responsiveness to new threats. Automating the structuring and maintenance of such knowledgebases not only improves scalability but also bridges the gap between threat intelligence and software engineering practices, facilitating the secure-by-design development of AI systems. We propose a pipeline based on a large language model that automatically extracts, refines, and classifies threat actions from incident reports into structured security knowledge. Our pipeline encompasses four critical phases: (1) threat-action extraction, (2) quality-guided refinement, (3) mapping to an established threat classification framework, and (4) structured knowledge construction with metadata generation. Evaluation shows 0.85 precision in extraction, with refinement further enhancing coverage. The classification phase effectively maps actions to known threat patterns while also identifying potentially novel threats that have yet to be documented, supporting the continuous evolution of the knowledgebase. These results lay the groundwork for scalable, automated construction and maintenance of AI security knowledge.