Minimizing WAN communications in inter-datacenter key-value stores

Hikaru Horie, Masato Asahara, Hiroshi Yamada, Kenji Kono

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Cloud-federations have emerged as popular platforms for Internet-scale services. Cloud-federations are running over multiple datacenters, because a cloud-federation is an aggregate of cloud services each of which runs in a single datacenter. In such inter-datacenter environments, distributed key-value stores (DKVSs) are attractive databases in terms of scalability. However, inter-datacenter communications degrade the performance of these DKVSs because of their large latency and narrow bandwidth. In this paper, we demonstrate how to reduce and hide the weak points of inter-datacenter communications for DKVSs. To solve the problems we introduce two techniques called multi-layered DHT (ML-DHT) and local-first data rebuilding (LDR). ML-DHT provides a global and consistent index of key-value pairs with the efficient expandability of the storage capacity. It employs a routing protocol which reduces routing hops that pass through interdatacenter connections. LDR reduces data transfer on interdatacenter connections by using erasure coding techniques. It enables KVS administrators to flexibly make trade-offs between expandability of storage capacity and the performance of data transfer. Experimental results demonstrate that our techniques improve the latency up to 74 % compared with a Chord-based system and enable us to balance the amount of storage usage and remote data transfer.

Original languageEnglish
Title of host publicationIEEE International Conference on Cloud Computing, CLOUD
PublisherIEEE Computer Society
Pages490-497
Number of pages8
ISBN (Print)9781479950638
DOIs
Publication statusPublished - 2014 Dec 3
Event7th IEEE International Conference on Cloud Computing, CLOUD 2014 - Anchorage, United States
Duration: 2014 Jun 272014 Jul 2

Other

Other7th IEEE International Conference on Cloud Computing, CLOUD 2014
CountryUnited States
CityAnchorage
Period14/6/2714/7/2

Fingerprint

Wide area networks
Data transfer
Communication
Routing protocols
Scalability
Internet
Bandwidth

Keywords

  • cloud federations
  • datacenter
  • distributed hash table
  • key-value store

ASJC Scopus subject areas

  • Artificial Intelligence
  • Information Systems
  • Software

Cite this

Horie, H., Asahara, M., Yamada, H., & Kono, K. (2014). Minimizing WAN communications in inter-datacenter key-value stores. In IEEE International Conference on Cloud Computing, CLOUD (pp. 490-497). [6973778] IEEE Computer Society. https://doi.org/10.1109/CLOUD.2014.72

Minimizing WAN communications in inter-datacenter key-value stores. / Horie, Hikaru; Asahara, Masato; Yamada, Hiroshi; Kono, Kenji.

IEEE International Conference on Cloud Computing, CLOUD. IEEE Computer Society, 2014. p. 490-497 6973778.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Horie, H, Asahara, M, Yamada, H & Kono, K 2014, Minimizing WAN communications in inter-datacenter key-value stores. in IEEE International Conference on Cloud Computing, CLOUD., 6973778, IEEE Computer Society, pp. 490-497, 7th IEEE International Conference on Cloud Computing, CLOUD 2014, Anchorage, United States, 14/6/27. https://doi.org/10.1109/CLOUD.2014.72
Horie H, Asahara M, Yamada H, Kono K. Minimizing WAN communications in inter-datacenter key-value stores. In IEEE International Conference on Cloud Computing, CLOUD. IEEE Computer Society. 2014. p. 490-497. 6973778 https://doi.org/10.1109/CLOUD.2014.72
Horie, Hikaru ; Asahara, Masato ; Yamada, Hiroshi ; Kono, Kenji. / Minimizing WAN communications in inter-datacenter key-value stores. IEEE International Conference on Cloud Computing, CLOUD. IEEE Computer Society, 2014. pp. 490-497
@inproceedings{8a60e406597e4cdcb5c20b5865687893,
title = "Minimizing WAN communications in inter-datacenter key-value stores",
abstract = "Cloud-federations have emerged as popular platforms for Internet-scale services. Cloud-federations are running over multiple datacenters, because a cloud-federation is an aggregate of cloud services each of which runs in a single datacenter. In such inter-datacenter environments, distributed key-value stores (DKVSs) are attractive databases in terms of scalability. However, inter-datacenter communications degrade the performance of these DKVSs because of their large latency and narrow bandwidth. In this paper, we demonstrate how to reduce and hide the weak points of inter-datacenter communications for DKVSs. To solve the problems we introduce two techniques called multi-layered DHT (ML-DHT) and local-first data rebuilding (LDR). ML-DHT provides a global and consistent index of key-value pairs with the efficient expandability of the storage capacity. It employs a routing protocol which reduces routing hops that pass through interdatacenter connections. LDR reduces data transfer on interdatacenter connections by using erasure coding techniques. It enables KVS administrators to flexibly make trade-offs between expandability of storage capacity and the performance of data transfer. Experimental results demonstrate that our techniques improve the latency up to 74 {\%} compared with a Chord-based system and enable us to balance the amount of storage usage and remote data transfer.",
keywords = "cloud federations, datacenter, distributed hash table, key-value store",
author = "Hikaru Horie and Masato Asahara and Hiroshi Yamada and Kenji Kono",
year = "2014",
month = "12",
day = "3",
doi = "10.1109/CLOUD.2014.72",
language = "English",
isbn = "9781479950638",
pages = "490--497",
booktitle = "IEEE International Conference on Cloud Computing, CLOUD",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Minimizing WAN communications in inter-datacenter key-value stores

AU - Horie, Hikaru

AU - Asahara, Masato

AU - Yamada, Hiroshi

AU - Kono, Kenji

PY - 2014/12/3

Y1 - 2014/12/3

N2 - Cloud-federations have emerged as popular platforms for Internet-scale services. Cloud-federations are running over multiple datacenters, because a cloud-federation is an aggregate of cloud services each of which runs in a single datacenter. In such inter-datacenter environments, distributed key-value stores (DKVSs) are attractive databases in terms of scalability. However, inter-datacenter communications degrade the performance of these DKVSs because of their large latency and narrow bandwidth. In this paper, we demonstrate how to reduce and hide the weak points of inter-datacenter communications for DKVSs. To solve the problems we introduce two techniques called multi-layered DHT (ML-DHT) and local-first data rebuilding (LDR). ML-DHT provides a global and consistent index of key-value pairs with the efficient expandability of the storage capacity. It employs a routing protocol which reduces routing hops that pass through interdatacenter connections. LDR reduces data transfer on interdatacenter connections by using erasure coding techniques. It enables KVS administrators to flexibly make trade-offs between expandability of storage capacity and the performance of data transfer. Experimental results demonstrate that our techniques improve the latency up to 74 % compared with a Chord-based system and enable us to balance the amount of storage usage and remote data transfer.

AB - Cloud-federations have emerged as popular platforms for Internet-scale services. Cloud-federations are running over multiple datacenters, because a cloud-federation is an aggregate of cloud services each of which runs in a single datacenter. In such inter-datacenter environments, distributed key-value stores (DKVSs) are attractive databases in terms of scalability. However, inter-datacenter communications degrade the performance of these DKVSs because of their large latency and narrow bandwidth. In this paper, we demonstrate how to reduce and hide the weak points of inter-datacenter communications for DKVSs. To solve the problems we introduce two techniques called multi-layered DHT (ML-DHT) and local-first data rebuilding (LDR). ML-DHT provides a global and consistent index of key-value pairs with the efficient expandability of the storage capacity. It employs a routing protocol which reduces routing hops that pass through interdatacenter connections. LDR reduces data transfer on interdatacenter connections by using erasure coding techniques. It enables KVS administrators to flexibly make trade-offs between expandability of storage capacity and the performance of data transfer. Experimental results demonstrate that our techniques improve the latency up to 74 % compared with a Chord-based system and enable us to balance the amount of storage usage and remote data transfer.

KW - cloud federations

KW - datacenter

KW - distributed hash table

KW - key-value store

UR - http://www.scopus.com/inward/record.url?scp=84919807900&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84919807900&partnerID=8YFLogxK

U2 - 10.1109/CLOUD.2014.72

DO - 10.1109/CLOUD.2014.72

M3 - Conference contribution

AN - SCOPUS:84919807900

SN - 9781479950638

SP - 490

EP - 497

BT - IEEE International Conference on Cloud Computing, CLOUD

PB - IEEE Computer Society

ER -