Modern C++ server workloads rely on 2 MB huge pages to improve memory system performance via higher TLB hit rates with larger address space coverage. Huge pages have traditionally been supported at the kernel level, but recent work shows user-level, huge page-aware memory allocators can achieve higher huge page coverage and thus performance. These memory allocators deal with a trade-off: 1) allocate memory from the operating system (OS) at the granularity of a huge page, achieve high performance, but potentially waste memory due to fragmentation, or 2) limit fragmentation by breaking up huge pages into smaller 4 KB pages and returning them to the OS, but reduce performance due to lower huge page coverage.

For example, TCMalloc’s huge page-aware memory allocation handles this trade-off by releasing memory to the operating system at a configurable release rate, breaking up huge pages as necessary. This approach balances performance and limiting fragmentation well for machines running one workload. For multiple applications on the same machine however, the reduction in memory usage is only useful to overall performance if another application will use it. In warehouse-scale computers, when an application releases and then requires the same amount or more memory quickly, but no other application uses the memory in the meantime, the release causes poorer huge page coverage without any system-wide benefit.

We introduce an adaptive release policy that dynamically determines whether or not to break up huge pages and return them to the OS to optimize system-wide performance. We built this policy into TCMalloc, a state-of-the-art memory allocator that uses huge pages. We deploy this strategy fleet-wide in warehouse-scale datacenters, delivering significant performance improvements at negligible real memory overhead, leading to a 1% fleet-wide throughput improvement.