Autonomous robots are more and more deployed in human environments, logistics, collaborative manufacturing applications, agriculture, or last-mile delivery tasks. These environments are characterized by dynamic changes, uncertainty from sensor readings and actuator movements, and disturbances by humans and other agents. That is, there are plenty of situations in which a robot needs to autonomously decide what to do in order to successfully accomplish its tasks. Traditional methods for programming robots often cannot provide this autonomy because they are lacking perception and decision-making capabilities. We believe that new, behavior-based programming methods are required to enable robots to robustly adapt and properly react to such unforeseen situations. However, when developing, deploying and operating products that exhibit highly autonomous behavior, new challenges arise: For guaranteeing a certain level of behavioural quality, novel testing and QA structures need to consider the complexity and non-determinism in the system. When operating a fleet of autonomous robots, the role of a robot manufacturer shifts from shipping products towards operating a distributed system of embodied computers connected to the Internet – which creates a number of opportunities, but also challenges to address.