A Case for Uni-directional Network Topologies in Large-Scale Clusters

Michihiro Koibuchi, Tomohiro Totoki, Hiroki Matsutani, Hideharu Amano, Fabien Chaix, Ikki Fujiwara, Henri Casanova

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Designing low-latency network topologies of switches is a key objective for next-generation large-scale clusters. Low latency is preconditioned on low hop counts, but existing network topologies have hop counts much larger than theoretical lower bounds. To alleviate this problem, we propose building network topologies based on uni-directional graphs that are known to have hop counts close to theoretical lower bounds. A practical difficulty with uni-directional topologies is switch-by-switch flow control, which we resolve by using hot-potato routing. Cycle-Accurate network simulation experiments for various traffic patterns on uni-directional topologies show that hot-potato routing achieves performance comparable to that of conventional deadlock-free routing. Similar experiments are used to compare several uni-directional topologies to bi-directional topologies, showing that the former achieve significantly lower latency and higher throughput. We quantify end-To-end application performance for parallel application benchmarks via discrete-even simulation, showing that uni-directional topologies can lead to large application performance improvements over their bi-directional counterparts. Finally, we discuss practical issues for uni-directional topologies such as cabling complexity and cost, power consumption, and soft-error tolerance. Our results make a compelling case for considering uni-directional topologies for upcoming large-scale clusters.

Original languageEnglish
Title of host publicationProceedings - 2017 IEEE International Conference on Cluster Computing, CLUSTER 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages178-187
Number of pages10
Volume2017-September
ISBN (Electronic)9781538623268
DOIs
Publication statusPublished - 2017 Sep 22
Event2017 IEEE International Conference on Cluster Computing, CLUSTER 2017 - Honolulu, United States
Duration: 2017 Sep 52017 Sep 8

Other

Other2017 IEEE International Conference on Cluster Computing, CLUSTER 2017
CountryUnited States
CityHonolulu
Period17/9/517/9/8

Fingerprint

Topology
Switches
Flow control
Electric power utilization
Experiments
Throughput
Costs

Keywords

  • Hot-potato routing
  • HPC clusters
  • Interconnection networks
  • Uni-directional network topologies

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Signal Processing

Cite this

Koibuchi, M., Totoki, T., Matsutani, H., Amano, H., Chaix, F., Fujiwara, I., & Casanova, H. (2017). A Case for Uni-directional Network Topologies in Large-Scale Clusters. In Proceedings - 2017 IEEE International Conference on Cluster Computing, CLUSTER 2017 (Vol. 2017-September, pp. 178-187). [8048929] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CLUSTER.2017.33

A Case for Uni-directional Network Topologies in Large-Scale Clusters. / Koibuchi, Michihiro; Totoki, Tomohiro; Matsutani, Hiroki; Amano, Hideharu; Chaix, Fabien; Fujiwara, Ikki; Casanova, Henri.

Proceedings - 2017 IEEE International Conference on Cluster Computing, CLUSTER 2017. Vol. 2017-September Institute of Electrical and Electronics Engineers Inc., 2017. p. 178-187 8048929.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Koibuchi, M, Totoki, T, Matsutani, H, Amano, H, Chaix, F, Fujiwara, I & Casanova, H 2017, A Case for Uni-directional Network Topologies in Large-Scale Clusters. in Proceedings - 2017 IEEE International Conference on Cluster Computing, CLUSTER 2017. vol. 2017-September, 8048929, Institute of Electrical and Electronics Engineers Inc., pp. 178-187, 2017 IEEE International Conference on Cluster Computing, CLUSTER 2017, Honolulu, United States, 17/9/5. https://doi.org/10.1109/CLUSTER.2017.33
Koibuchi M, Totoki T, Matsutani H, Amano H, Chaix F, Fujiwara I et al. A Case for Uni-directional Network Topologies in Large-Scale Clusters. In Proceedings - 2017 IEEE International Conference on Cluster Computing, CLUSTER 2017. Vol. 2017-September. Institute of Electrical and Electronics Engineers Inc. 2017. p. 178-187. 8048929 https://doi.org/10.1109/CLUSTER.2017.33
Koibuchi, Michihiro ; Totoki, Tomohiro ; Matsutani, Hiroki ; Amano, Hideharu ; Chaix, Fabien ; Fujiwara, Ikki ; Casanova, Henri. / A Case for Uni-directional Network Topologies in Large-Scale Clusters. Proceedings - 2017 IEEE International Conference on Cluster Computing, CLUSTER 2017. Vol. 2017-September Institute of Electrical and Electronics Engineers Inc., 2017. pp. 178-187
@inproceedings{f00201d0460749569cfbe2cfde35811c,
title = "A Case for Uni-directional Network Topologies in Large-Scale Clusters",
abstract = "Designing low-latency network topologies of switches is a key objective for next-generation large-scale clusters. Low latency is preconditioned on low hop counts, but existing network topologies have hop counts much larger than theoretical lower bounds. To alleviate this problem, we propose building network topologies based on uni-directional graphs that are known to have hop counts close to theoretical lower bounds. A practical difficulty with uni-directional topologies is switch-by-switch flow control, which we resolve by using hot-potato routing. Cycle-Accurate network simulation experiments for various traffic patterns on uni-directional topologies show that hot-potato routing achieves performance comparable to that of conventional deadlock-free routing. Similar experiments are used to compare several uni-directional topologies to bi-directional topologies, showing that the former achieve significantly lower latency and higher throughput. We quantify end-To-end application performance for parallel application benchmarks via discrete-even simulation, showing that uni-directional topologies can lead to large application performance improvements over their bi-directional counterparts. Finally, we discuss practical issues for uni-directional topologies such as cabling complexity and cost, power consumption, and soft-error tolerance. Our results make a compelling case for considering uni-directional topologies for upcoming large-scale clusters.",
keywords = "Hot-potato routing, HPC clusters, Interconnection networks, Uni-directional network topologies",
author = "Michihiro Koibuchi and Tomohiro Totoki and Hiroki Matsutani and Hideharu Amano and Fabien Chaix and Ikki Fujiwara and Henri Casanova",
year = "2017",
month = "9",
day = "22",
doi = "10.1109/CLUSTER.2017.33",
language = "English",
volume = "2017-September",
pages = "178--187",
booktitle = "Proceedings - 2017 IEEE International Conference on Cluster Computing, CLUSTER 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - A Case for Uni-directional Network Topologies in Large-Scale Clusters

AU - Koibuchi, Michihiro

AU - Totoki, Tomohiro

AU - Matsutani, Hiroki

AU - Amano, Hideharu

AU - Chaix, Fabien

AU - Fujiwara, Ikki

AU - Casanova, Henri

PY - 2017/9/22

Y1 - 2017/9/22

N2 - Designing low-latency network topologies of switches is a key objective for next-generation large-scale clusters. Low latency is preconditioned on low hop counts, but existing network topologies have hop counts much larger than theoretical lower bounds. To alleviate this problem, we propose building network topologies based on uni-directional graphs that are known to have hop counts close to theoretical lower bounds. A practical difficulty with uni-directional topologies is switch-by-switch flow control, which we resolve by using hot-potato routing. Cycle-Accurate network simulation experiments for various traffic patterns on uni-directional topologies show that hot-potato routing achieves performance comparable to that of conventional deadlock-free routing. Similar experiments are used to compare several uni-directional topologies to bi-directional topologies, showing that the former achieve significantly lower latency and higher throughput. We quantify end-To-end application performance for parallel application benchmarks via discrete-even simulation, showing that uni-directional topologies can lead to large application performance improvements over their bi-directional counterparts. Finally, we discuss practical issues for uni-directional topologies such as cabling complexity and cost, power consumption, and soft-error tolerance. Our results make a compelling case for considering uni-directional topologies for upcoming large-scale clusters.

AB - Designing low-latency network topologies of switches is a key objective for next-generation large-scale clusters. Low latency is preconditioned on low hop counts, but existing network topologies have hop counts much larger than theoretical lower bounds. To alleviate this problem, we propose building network topologies based on uni-directional graphs that are known to have hop counts close to theoretical lower bounds. A practical difficulty with uni-directional topologies is switch-by-switch flow control, which we resolve by using hot-potato routing. Cycle-Accurate network simulation experiments for various traffic patterns on uni-directional topologies show that hot-potato routing achieves performance comparable to that of conventional deadlock-free routing. Similar experiments are used to compare several uni-directional topologies to bi-directional topologies, showing that the former achieve significantly lower latency and higher throughput. We quantify end-To-end application performance for parallel application benchmarks via discrete-even simulation, showing that uni-directional topologies can lead to large application performance improvements over their bi-directional counterparts. Finally, we discuss practical issues for uni-directional topologies such as cabling complexity and cost, power consumption, and soft-error tolerance. Our results make a compelling case for considering uni-directional topologies for upcoming large-scale clusters.

KW - Hot-potato routing

KW - HPC clusters

KW - Interconnection networks

KW - Uni-directional network topologies

UR - http://www.scopus.com/inward/record.url?scp=85032626894&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85032626894&partnerID=8YFLogxK

U2 - 10.1109/CLUSTER.2017.33

DO - 10.1109/CLUSTER.2017.33

M3 - Conference contribution

AN - SCOPUS:85032626894

VL - 2017-September

SP - 178

EP - 187

BT - Proceedings - 2017 IEEE International Conference on Cluster Computing, CLUSTER 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -