Machine learning (ML) components are increasingly integrated into software products, yet their complexity and inherent uncertainty often lead to unintended and potentially hazardous consequences, both for individuals and society at large. Despite these risks, practitioners rarely adopt proactive approaches to anticipate and mitigate potential hazards before they occur. Traditional safety engineering approaches, such as Failure Mode and Effects Analysis (FMEA) and System Theoretic Process Analysis (STPA), offer promising frameworks for systematic early risk identification but are rarely adopted. In this position paper, we argue that hazard analysis should be an integral part of developing any ML-powered software product and that greater support is needed to make this process manageable for developers. By using large language models (LLMs) to partially automate a modified STPA process with human oversight at critical steps, we expect to address two key challenges: the heavy dependency on highly experienced safety engineering experts, and the time-consuming, labor-intensive nature of traditional hazard analysis, which often impedes its integration into real-world development workflows. We illustrate our approach with a running example, demonstrating that many seemingly unanticipated issues can, in fact, be anticipated.