PPMDS: A Distributed Metadata Server Based on Nonblocking Transactions

Kohei Hiraga, Osamu Tatebe, Hideyuki Kawashima

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

There is a critical problem of metadata performance scalability in high-performance computing when accessing many small files from millions of clients. This paper proposes a design of PPMDS, a scalable distributed metadata server, to improve metadata performance and scalability. To efficiently manage hierarchical namespace of file system across multiple servers, it manages inode entries using a parent inode number and an entry name as a key. To update multiple inode entries atomically, it uses a nonblocking distributed transaction based on an algorithm of dynamic software transactional memory. It improves the metadata performance by introducing server-side transaction processing, multi-reader, and a shared lock mode to reduce the number of remote procedure calls and to prevent unnecessary blocking. A prototype implementation achieves 62,000 operations per second when creating files in a single directory using 8 metadata servers, which is 2.58x performance improvement compared to a single metadata performance.

Original languageEnglish
Title of host publication2018 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages202-208
Number of pages7
ISBN (Electronic)9781538695883
DOIs
Publication statusPublished - 2018 Nov 30
Event5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018 - Valencia, Spain
Duration: 2018 Oct 152018 Oct 18

Other

Other5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018
CountrySpain
CityValencia
Period18/10/1518/10/18

    Fingerprint

Keywords

  • distributed file system
  • Distributed metadata server
  • nonblocking distributed transaction

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality
  • Communication

Cite this

Hiraga, K., Tatebe, O., & Kawashima, H. (2018). PPMDS: A Distributed Metadata Server Based on Nonblocking Transactions. In 2018 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018 (pp. 202-208). [8554478] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SNAMS.2018.8554478