In the rapidly evolving landscape of software development, Large Language Models (LLMs) for code have emerged as groundbreaking tools for code completion, synthesis, and analysis. BigCode is an open scientific collaboration for the responsible development of code LLMs. In this talk, we will cover some of the foundational elements of BigCode, including open large-scale code datasets such as The Stack, data governance, and transparency standards, as well as our approach for training the competitive StarCoder and StarCoder2 models.

Loubna Ben Allal is a Machine Learning Engineer in the Science team at Hugging Face working on Large Language Models for code & Synthetic data generation. She is part of the core team behind the BigCode Project and has co-authored The Stack dataset and StarCoder models for code generation. Loubna holds Mathematics & Deep Learning Master’s Degrees from Ecole des Mines de Nancy and ENS Paris Saclay.