BatchIt: Optimizing Message-Passing Allocators for Producer-Consumer Workloads: An Intellectual Abstract
Modern, high-performance memory allocators must scale to a wide array of uses, including producer-consumer workloads. In such workloads, objects are allocated by one thread and deallocated by another, which we call remote deallocations. These remote deallocations lead to contention on the allocator’s synchronization mechanisms. Message-passing allocators, such as mimalloc and snmalloc, use message queues to communicate remote deallocations between threads. These queues work well for producer-consumer workloads, but there is room for optimization. We propose and characterize BatchIt, a conceptually simple optimization for such allocators: a per-slab cache of remote deallocations that enables batching of objects destined for the same slab. This optimization aims to exploit naturally-arising locality of allocations, and it generalizes across particular implementations; we have implementations for both mimalloc and snmalloc. Multi-threaded, producer-consumer benchmarks show improved performance from reduced rates of atomic operations and cache misses in the underlying allocator. Experimental results using the mimalloc-bench suite and a custom message-passing workload show that some producer-consumer workloads see over 20% performance improvement even atop the high performance these allocators already provide.
Tue 25 JunDisplayed time zone: Windhoek change
14:40 - 15:40 | ISMM: Session 3 - Intellectual AbstractsISMM 2024 at Iceland Chair(s): Michael D. Bond Ohio State University | ||
14:40 20mTalk | The One Pass (OP) Compactor: An Intellectual Abstract ISMM 2024 DOI | ||
15:00 20mTalk | BatchIt: Optimizing Message-Passing Allocators for Producer-Consumer Workloads: An Intellectual Abstract ISMM 2024 DOI Pre-print | ||
15:20 20mTalk | Reference Counting Deeply Immutable Data Structures with Cycles: An Intellectual Abstract ISMM 2024 Matthew J. Parkinson Microsoft Azure Research, Sylvan Clebsch Microsoft Azure Research, Tobias Wrigstad Uppsala University DOI Pre-print |