In this paper we draw attention to the challenges of managing software projects for vulnerable populations, i.e., people potentially exposed to harm or not capable of protecting their own interests. The focus on human aspects, and particularly, the inclusion of human-centered approaches, has been a popular topic in the software engineering community. We argue, however, that current literature provides little understanding and guidance on how to approach these type of scenarios. Here, we shed some light on the topic by reporting on our experiences in developing innovative solutions for the residential care scenario, outlining potential issues and recommendations.