In this talk, we describe the motivations behind the creation of AndroZoo, a collection of 5+ millions of Android Application, and provide concrete examples of the problems implied by the absence of a widely available and widely used benchmark dataset. Next, we come back on some of the engineering challenges linked with the creation of a large scale dataset (70+TB), and highlight the organisational challenges of maintaining a dataset in the long run in an academic setting. We also quickly present our approach to try to stay as clear as possible of the legal difficulties of distributing non-Free software. Finally, we discuss the adoption of AndroZoo and the fulfillment of our original goals, and we address what we believe are the main limiting factors with regards to AndroZoo’s adoption.