Onboarding new software developers presents persistent challenges for teams, with newcomers facing steep learning curves and senior staff burdened by providing training and mentoring. This research explores leveraging Large Language Models (LLMs) to streamline onboarding, mitigating productivity losses from disrupted workflows. We collaborated with LKS Next, an IT consulting firm that has experienced first hand the challenges onboarding new developers brings. This paper presents an ongoing case study within LKS Next, conducted using Action Design Research methodology. Two cycles have been completed, gathering feedback from newcomers, team leaders, and managers on issues like the need to prioritize self-directed learning resources over burdening mentors, and considerations around using third-party LLMs. The third cycle focuses on evaluating open source LLMs to maintain control within the company. Our envisioned goal is to develop an LLM-powered conversational agent delivering tailored onboarding support while avoiding privacy risks.