Building production systems with machine learning components is challenging and many projects fail when moving into production even when showing initial success with training machine-learned models. Unfortunately data science education focuses narrowly on data analysis, machine-learning algorithms, and model building but rarely engages with how the model may be used as part of a system. Engineering aspects beyond deploying models are often ignored or underappreciated, including requirements engineering, user experience design, planning and testing integration with non-ML components, and planning for evolution, leading to poor outcomes in many real-world projects. Software engineers and data scientists often clash in teams due to different goals, processes, and expectations, finding it hard to effectively coordinate and integrate work. In this talk, I argue for the important roles that software engineers have in machine learning projects that want to move beyond a prototype model. I argue that truly a system-wide perspective is needed if we want to have any hope at making meaningful progress on safety, usability, fairness, or security. I explore the common collaboration problems and discuss strategies to overcome them. This talk is a call for more and better education in this space at the intersection of software engineering and machine learning, as well as for more system-wide research on building software systems with machine-learning components.