DeFS: A Decentralized and High-Performance File System for Consortium Systems
Consortium decentralized systems, also known as consortium systems, enable consensus and availability among limited untrusted participants. Given the growing necessity for inter-organizational data sharing, consortium systems have gained significant prominence in cross-enterprise collaboration. File systems, which play a fundamental role in data sharing, face new challenges within consortium systems. The consortium system involves characteristics of both decentralized and centralized systems. The first requirement is decentralization. The system’s functionality, availability, and security must not depend on any individual node. The second requirement is characteristic-awareness, which necessitates optimal data placement across nodes based on policy constraints, performance requirements, and security considerations. The third requirement is high performance and flexible access control. Neither centralized nor existing decentralized file systems can satisfy all three requirements.
This paper presents DeFS, a novel decentralized file system designed for consortium systems. DeFS implements a two-layer architecture that incorporates public nodes into the consortium system, thereby enhancing decentralization. Additionally, we propose a Multi-Ring Distributed Hash Table (MR-DHT) protocol to facilitate characteristic-aware data block distribution. To optimize data routing efficiency, we introduce the Location Cache (L-Cache) mechanism. We have implemented a DeFS prototype and conducted comprehensive performance evaluations across three distinct network configurations, with the largest one having over 1,500 nodes. Results show that DeFS successfully achieves characteristic-aware data placement while delivering \emph{\textbf{10.32X}} lower latency compared to IPFS on average.
Sat 21 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
11:00 - 13:00 | Session4: Code Optimization and Software ArchitectureResearch Track at Cosmos 3A Chair(s): Changhai Nie Nanjing University | ||
11:00 15mTalk | Take Kernel Stack Overhead Out: eBPF-Enhanced Network Acceleration for Distributed Training within Ethernet Research Track Zhenyu Zhang School of Computer Science and Engineering, Sun Yat-sen University, Pengfei Chen Sun Yat-sen University, Guangba Yu School of Computer Science and Engineering, Sun Yat-sen University, Zilong He Sun Yat-sen University, Xiaoyun Li Sun Yat-sen University Pre-print | ||
11:15 15mTalk | Exploiting Booster Pass Chain for Compiler Phase Ordering Research Track yihan chen , Huanhuan Chen Nanjing University, Yuan Yao Nanjing University, Ping Yu Nanjing University, Feng Xu Nanjing University, Xiaoxing Ma Nanjing University File Attached | ||
11:30 15mTalk | DeFS: A Decentralized and High-Performance File System for Consortium Systems Research Track Yitong Cheng Shanghai JiaoTong University, Shenglong Zhao Shanghai JiaoTong University, Yang Yu Shanghai Jiao Tong University, China, Zhichao Hua Shanghai Jiao Tong University | ||
11:45 15mTalk | Proteus: An Automatical High-Efficiency Framework for Generating Compact and Printable Shellcode on ARMv8 Research Track Jian Lin Information Engineering University, Guoan Liu Information Engineering University, Rui Chang Zhejiang University, Ruimin Wang Information Engineering University | ||
12:00 15mTalk | Modeling Go Concurrency: A Static Analysis Approach to Data Race Detection Research Track Fengjuan Gao Nanjing University of Science and Technology, Mumu Zhang Nanjing University, Zixiao Zhao Nanjing University, Yu Wang Nanjing University, Xuandong Li Nanjing University | ||
12:15 15mTalk | RABBIT: Managing Hierarchical Memory with Intelligent Tiering Aware Deduplication Research Track | ||
12:30 15mTalk | DPCapsule: A Decentralized Private Computing System With Self-Controlled Data Research Track Yitong Cheng Shanghai JiaoTong University, Yang Yu Shanghai Jiao Tong University, China, Zhichao Hua Shanghai Jiao Tong University | ||
12:45 15mTalk | MicroGuard:Non-Intrusive Dynamic Analysis for Inter-Service Access Control of Microservices Research Track Haoming Luo School of Computer Science and Engineering, Sun Yat-sen University, Wanqi Yang Sun Yat-Sen University, Pengfei Chen Sun Yat-sen University | ||
Cosmos 3A is the first room in the Cosmos 3 wing.
When facing the main Cosmos Hall, access to the Cosmos 3 wing is on the left, close to the stairs. The area is accessed through a large door with the number “3”, which will stay open during the event.