Performance evaluation of power-aware multi-tree ethernet for HPC interconnects

Michihiro Koibuchi, Takafumi Watanabe, Atsushi Minamihata, Masahiro Nakao, Tomoyuki Hiroyasu, Hiroki Matsutani, Hideharu Amano

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Ethernet has been used for interconnection networks of high-performance computing (HPC) systems that include PC clusters. Although a layer-2 Ethernet topology is limited to a tree structure in order to avoid broadcast storms and deadlocks of frames, various topologies with deadlock-free routing, that include loops, suitable for parallel processing can be used by the application of IEEE 802.1Q VLAN technology. However, their performance and power evaluations on real PC clusters with Ethernet have been rarely done. Thus, their evaluation on real PC clusters with Ethernet is important to analyze their impact on the total systems and validate their simulation results. In this paper, firstly we measure tree, and fully connected topologies with link aggregation on a 66-node/528-core PC cluster with a number of Ethernet switches. Secondly, we measure the optimization of the power consumption of Ethernet by a power-aware link regulation in order to reduce the power consumption of Ethernet switches on a real PC cluster. Measurement results show that the fully connected topology achieves 3.21 TFlops in High-Performance Linpack Benchmark (HPL), and its Rmax/Rpeak value is 67%. Up to 23% of the power consumption of networks can be reduced by the power-aware link regulation, while the performance is degraded by less than 1%.

Original languageEnglish
Title of host publicationProceedings - 2011 2nd International Conference on Networking and Computing, ICNC 2011
Pages50-57
Number of pages8
DOIs
Publication statusPublished - 2011
Event2nd International Conference on Networking and Computing, ICNC 2011 - Osaka, Japan
Duration: 2011 Nov 302011 Dec 2

Other

Other2nd International Conference on Networking and Computing, ICNC 2011
CountryJapan
CityOsaka
Period11/11/3011/12/2

Fingerprint

Ethernet
Topology
Electric power utilization
Switches
Agglomeration
Processing

Keywords

  • cluster interconnects
  • Ethernet
  • interconnection networks
  • performance measurement
  • topology

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications

Cite this

Koibuchi, M., Watanabe, T., Minamihata, A., Nakao, M., Hiroyasu, T., Matsutani, H., & Amano, H. (2011). Performance evaluation of power-aware multi-tree ethernet for HPC interconnects. In Proceedings - 2011 2nd International Conference on Networking and Computing, ICNC 2011 (pp. 50-57). [6131793] https://doi.org/10.1109/ICNC.2011.17

Performance evaluation of power-aware multi-tree ethernet for HPC interconnects. / Koibuchi, Michihiro; Watanabe, Takafumi; Minamihata, Atsushi; Nakao, Masahiro; Hiroyasu, Tomoyuki; Matsutani, Hiroki; Amano, Hideharu.

Proceedings - 2011 2nd International Conference on Networking and Computing, ICNC 2011. 2011. p. 50-57 6131793.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Koibuchi, M, Watanabe, T, Minamihata, A, Nakao, M, Hiroyasu, T, Matsutani, H & Amano, H 2011, Performance evaluation of power-aware multi-tree ethernet for HPC interconnects. in Proceedings - 2011 2nd International Conference on Networking and Computing, ICNC 2011., 6131793, pp. 50-57, 2nd International Conference on Networking and Computing, ICNC 2011, Osaka, Japan, 11/11/30. https://doi.org/10.1109/ICNC.2011.17
Koibuchi M, Watanabe T, Minamihata A, Nakao M, Hiroyasu T, Matsutani H et al. Performance evaluation of power-aware multi-tree ethernet for HPC interconnects. In Proceedings - 2011 2nd International Conference on Networking and Computing, ICNC 2011. 2011. p. 50-57. 6131793 https://doi.org/10.1109/ICNC.2011.17
Koibuchi, Michihiro ; Watanabe, Takafumi ; Minamihata, Atsushi ; Nakao, Masahiro ; Hiroyasu, Tomoyuki ; Matsutani, Hiroki ; Amano, Hideharu. / Performance evaluation of power-aware multi-tree ethernet for HPC interconnects. Proceedings - 2011 2nd International Conference on Networking and Computing, ICNC 2011. 2011. pp. 50-57
@inproceedings{324c999a96cf4f44979542832cada5c8,
title = "Performance evaluation of power-aware multi-tree ethernet for HPC interconnects",
abstract = "Ethernet has been used for interconnection networks of high-performance computing (HPC) systems that include PC clusters. Although a layer-2 Ethernet topology is limited to a tree structure in order to avoid broadcast storms and deadlocks of frames, various topologies with deadlock-free routing, that include loops, suitable for parallel processing can be used by the application of IEEE 802.1Q VLAN technology. However, their performance and power evaluations on real PC clusters with Ethernet have been rarely done. Thus, their evaluation on real PC clusters with Ethernet is important to analyze their impact on the total systems and validate their simulation results. In this paper, firstly we measure tree, and fully connected topologies with link aggregation on a 66-node/528-core PC cluster with a number of Ethernet switches. Secondly, we measure the optimization of the power consumption of Ethernet by a power-aware link regulation in order to reduce the power consumption of Ethernet switches on a real PC cluster. Measurement results show that the fully connected topology achieves 3.21 TFlops in High-Performance Linpack Benchmark (HPL), and its Rmax/Rpeak value is 67{\%}. Up to 23{\%} of the power consumption of networks can be reduced by the power-aware link regulation, while the performance is degraded by less than 1{\%}.",
keywords = "cluster interconnects, Ethernet, interconnection networks, performance measurement, topology",
author = "Michihiro Koibuchi and Takafumi Watanabe and Atsushi Minamihata and Masahiro Nakao and Tomoyuki Hiroyasu and Hiroki Matsutani and Hideharu Amano",
year = "2011",
doi = "10.1109/ICNC.2011.17",
language = "English",
isbn = "9780769545691",
pages = "50--57",
booktitle = "Proceedings - 2011 2nd International Conference on Networking and Computing, ICNC 2011",

}

TY - GEN

T1 - Performance evaluation of power-aware multi-tree ethernet for HPC interconnects

AU - Koibuchi, Michihiro

AU - Watanabe, Takafumi

AU - Minamihata, Atsushi

AU - Nakao, Masahiro

AU - Hiroyasu, Tomoyuki

AU - Matsutani, Hiroki

AU - Amano, Hideharu

PY - 2011

Y1 - 2011

N2 - Ethernet has been used for interconnection networks of high-performance computing (HPC) systems that include PC clusters. Although a layer-2 Ethernet topology is limited to a tree structure in order to avoid broadcast storms and deadlocks of frames, various topologies with deadlock-free routing, that include loops, suitable for parallel processing can be used by the application of IEEE 802.1Q VLAN technology. However, their performance and power evaluations on real PC clusters with Ethernet have been rarely done. Thus, their evaluation on real PC clusters with Ethernet is important to analyze their impact on the total systems and validate their simulation results. In this paper, firstly we measure tree, and fully connected topologies with link aggregation on a 66-node/528-core PC cluster with a number of Ethernet switches. Secondly, we measure the optimization of the power consumption of Ethernet by a power-aware link regulation in order to reduce the power consumption of Ethernet switches on a real PC cluster. Measurement results show that the fully connected topology achieves 3.21 TFlops in High-Performance Linpack Benchmark (HPL), and its Rmax/Rpeak value is 67%. Up to 23% of the power consumption of networks can be reduced by the power-aware link regulation, while the performance is degraded by less than 1%.

AB - Ethernet has been used for interconnection networks of high-performance computing (HPC) systems that include PC clusters. Although a layer-2 Ethernet topology is limited to a tree structure in order to avoid broadcast storms and deadlocks of frames, various topologies with deadlock-free routing, that include loops, suitable for parallel processing can be used by the application of IEEE 802.1Q VLAN technology. However, their performance and power evaluations on real PC clusters with Ethernet have been rarely done. Thus, their evaluation on real PC clusters with Ethernet is important to analyze their impact on the total systems and validate their simulation results. In this paper, firstly we measure tree, and fully connected topologies with link aggregation on a 66-node/528-core PC cluster with a number of Ethernet switches. Secondly, we measure the optimization of the power consumption of Ethernet by a power-aware link regulation in order to reduce the power consumption of Ethernet switches on a real PC cluster. Measurement results show that the fully connected topology achieves 3.21 TFlops in High-Performance Linpack Benchmark (HPL), and its Rmax/Rpeak value is 67%. Up to 23% of the power consumption of networks can be reduced by the power-aware link regulation, while the performance is degraded by less than 1%.

KW - cluster interconnects

KW - Ethernet

KW - interconnection networks

KW - performance measurement

KW - topology

UR - http://www.scopus.com/inward/record.url?scp=84863020874&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863020874&partnerID=8YFLogxK

U2 - 10.1109/ICNC.2011.17

DO - 10.1109/ICNC.2011.17

M3 - Conference contribution

SN - 9780769545691

SP - 50

EP - 57

BT - Proceedings - 2011 2nd International Conference on Networking and Computing, ICNC 2011

ER -