TY - GEN
T1 - PPMDS
T2 - 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018
AU - Hiraga, Kohei
AU - Tatebe, Osamu
AU - Kawashima, Hideyuki
N1 - Funding Information:
This work is partially supported by the JST CREST Grant Numbers JPMJCR1303 and JPMJCR1414, and JSPS KAK-ENHI Grant Number JP17H01748.
Publisher Copyright:
© 2018 IEEE.
PY - 2018/11/30
Y1 - 2018/11/30
N2 - There is a critical problem of metadata performance scalability in high-performance computing when accessing many small files from millions of clients. This paper proposes a design of PPMDS, a scalable distributed metadata server, to improve metadata performance and scalability. To efficiently manage hierarchical namespace of file system across multiple servers, it manages inode entries using a parent inode number and an entry name as a key. To update multiple inode entries atomically, it uses a nonblocking distributed transaction based on an algorithm of dynamic software transactional memory. It improves the metadata performance by introducing server-side transaction processing, multi-reader, and a shared lock mode to reduce the number of remote procedure calls and to prevent unnecessary blocking. A prototype implementation achieves 62,000 operations per second when creating files in a single directory using 8 metadata servers, which is 2.58x performance improvement compared to a single metadata performance.
AB - There is a critical problem of metadata performance scalability in high-performance computing when accessing many small files from millions of clients. This paper proposes a design of PPMDS, a scalable distributed metadata server, to improve metadata performance and scalability. To efficiently manage hierarchical namespace of file system across multiple servers, it manages inode entries using a parent inode number and an entry name as a key. To update multiple inode entries atomically, it uses a nonblocking distributed transaction based on an algorithm of dynamic software transactional memory. It improves the metadata performance by introducing server-side transaction processing, multi-reader, and a shared lock mode to reduce the number of remote procedure calls and to prevent unnecessary blocking. A prototype implementation achieves 62,000 operations per second when creating files in a single directory using 8 metadata servers, which is 2.58x performance improvement compared to a single metadata performance.
KW - Distributed metadata server
KW - distributed file system
KW - nonblocking distributed transaction
UR - http://www.scopus.com/inward/record.url?scp=85060015896&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85060015896&partnerID=8YFLogxK
U2 - 10.1109/SNAMS.2018.8554478
DO - 10.1109/SNAMS.2018.8554478
M3 - Conference contribution
AN - SCOPUS:85060015896
T3 - 2018 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018
SP - 202
EP - 208
BT - 2018 5th International Conference on Social Networks Analysis, Management and Security, SNAMS 2018
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 15 October 2018 through 18 October 2018
ER -