ISMM 2021
Tue 22 Jun 2021 PLDI
co-located with PLDI 2021

In our information-driven societies, full-text search is ubiquitous. Search is memory-intensive. Quickly searching massive corpora requires building indices, which consumes big volatile heaps. Search is storage I/O-intensive. Limited main memory necessitates writing large partial indices on non-volatile storage, where they finally live in merged form. These indices reside in memory, in full or in part, during query evaluation. Memory and I/O intensity make it hard to index and search content rapidly and efficiently. On the hardware side, the recently introduced Intel Optane DC persistent memory (PM) offers byte-addressability, high capacity, and non-volatility. This paper evaluates and exploits Optane PM for text indexing and search on multicore platforms.

We identify essential structures in inverted indices (hash table, merge tree, and key-value store), where they reside (memory or storage), and key operations over them (sort, flush, and merge). We allocate these structures in DRAM, Optane PM, and block storage by modifying an existing search engine. We then evaluate a myriad of hybrid memory and storage configurations. Our findings include: (1) careful placement of index structures across DRAM, Optane PM, and SSD, speeds up indexing with a single core compared to a high-performance baseline but does not scale to many cores, (2) crash-consistent indexing with Optane PM is feasible without incurring a high overhead, and (3) the tail latency of the longest multi-term conjunctive queries is lower with a PM-backed index than an SSD-backed one. This paper opens up persistent memory to a practical role in full-text search.

presentation slides (ismm-2021-search.pdf)6.39MiB

Shoaib Akram is an assistant professor at the Australian National University. He holds a Ph.D. in Computer Science and Engineering from Ghent University in Belgium and an M.S. in Electrical and Computer Engineering from the University of Illinois at Urbana-Champaign. His current research focuses on the evaluation, optimization, and applications of emerging memory and storage hardware.

Tue 22 Jun

Displayed time zone: Eastern Time (US & Canada) change

18:00 - 21:00
Session 4: Compacting/Indexing/Transactioning & ClosingISMM 2021 at ISMM
Chair(s): Timothy M. Jones University of Cambridge, UK
Exploiting Intel Optane Persistent Memory for Full Text Search
ISMM 2021
Shoaib Akram Australian National University
Pre-print File Attached
Understanding and Utilizing Hardware Transactional Memory Capacity
ISMM 2021
Zixian Cai Australian National University, Steve Blackburn Australian National University, Michael D. Bond Ohio State University, USA
Link to publication DOI Media Attached
Fusuma: Double-ended Threaded Compaction
ISMM 2021
Hiro Onozawa The University of Electro-Communications, Tomoharu Ugawa University of Tokyo, Hideya Iwasaki University of Electro-Communications, Japan
Day closing
Closing Remarks
ISMM 2021
Tobias Wrigstad Uppsala University, Sweden, Zhenlin Wang Michigan Technological University