OMRGx: Programmable and Transparent Out-of-Core Graph Partitioning and Processing
Partitioning and processing of large graphs on a single machine with limited memory is a challenge. While many custom solutions for out-of-core \emph{processing} have been developed, limited work has been done on out-of-core \emph{partitioning} that can be far more memory intensive than processing. In this paper we present the OMRGx system whose programming interface allows the programmer to rapidly prototype existing as well as new partitioning and processing strategies with minimal programming effort and oblivious of the graph size. The OMRGx engine transparently implements these strategies in an out-of-core manner while hiding the complexities of managing limited memory, parallel computation, and parallel IO from the programmer. The execution model allows multiple partitions to be simultaneously constructed and simultaneously processed by dividing the machine memory among the partitions. In contrast, existing systems process partitions one at a time. Using OMRGx we developed the first out-of-core implementation of the popular MtMetis partitioner. OMRGx implementations of existing GridGraph and GraphChi out-of-core processing frameworks deliver performance better than their standalone optimized implementations. The runtimes of implementations produced by OMRGx decrease with the number of partitions requested and increase \emph{linearly} with the graph size. Finally OMRGx default implementation performs the best of all.
Sun 18 JunDisplayed time zone: Eastern Time (US & Canada) change
16:00 - 17:20 | |||
16:00 20mTalk | Blast from the Past: Least Expected Use (LEU) Cache Replacement with Statistical History ISMM 2023 Sayak Chakraborti University of Rochester, Zhizhou (Chris) Zhang Uber Technologies, Noah Bertram Cornell University, Sandhya Dwarkadas University of Rochester, Chen Ding University of Rochester DOI | ||
16:20 20mTalk | OMRGx: Programmable and Transparent Out-of-Core Graph Partitioning and Processing ISMM 2023 DOI | ||
16:40 20mTalk | ZipKV: In-Memory Key-Value Store with Built-In Data Compression ISMM 2023 Linsen Ma Rensselaer Polytechnic Institute, Rui Xie Rensselaer Polytechnic Institute, Tong Zhang Rensselaer Polytechnic Institute DOI | ||
17:00 20mTalk | Flexible and Effective Object Tiering for Heterogeneous Memory Systems ISMM 2023 Brandon Kammerdiener University of Tennessee, Jeffrey Zachariah McMichael University of Tennessee, Michael Jantz University of Tennessee, Kshitij Doshi Intel Corporation, Terry Jones Oak Ridge National Laboratory DOI |