Performance Evaluation of PEACH3: Field-Programmable Gate Array Switch for Tightly Coupled Accelerators

Takahiro Kaneda, Ryotaro Sakai, Naoki Nishikawa, Toshihiro Hanawa, Chiharu Tsuruta, Hideharu Amano

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

An FPGA switching hub for tightly coupled accelerators (TCA) architecture called PEACH3 (PCI-Express Adaptive Communication Hub ver. 3) is evaluated and its communication speed is analyzed. PEACH3 connects a number of GPUs directly through PCI express Gen3x8 ports. The latency of inter-node GPU-GPU communication of PEACH3 was about 2.8 µ sec which is one third of that of CUDA API with MPI/Infiniband. The bandwidth was about 1.21 times of that of the previous version PEACH2, and 1.54 times of that with MPI/Infiniband for 512KB data transfer. Two application programs: BFS (breadth first search) and CG (conjugate gradient) were implemented with TCA IP and CUDA IP with MPI/Infiniband. The performance of BFS with PEACH3 was 1.16 times better than that with PEACH2, and 1.3 times better than that with MPI/Infiniband for a graph with scale = 15. In CG, for the small matrix (CLASS=S), the PEACH3 achieved 12% better performance than that with PEACH2 and 25% with MPI/Infiniband. However, since the bandwidth of PEACH3 with PCI gen3x8 is smaller than Infiniband with PCI gen3x16, the performance benefit was disappeared for CLASS=A matrix. Through the evaluation, it appears that if the data size is small, using TCA API with PEACH3 is advantageous even for intra-node communication.

Original languageEnglish
Title of host publicationProceedings of the 8th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies, HEART 2017
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450353168
DOIs
Publication statusPublished - 2017 Jun 7
Event8th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies, HEART 2017 - Bochum, Germany
Duration: 2017 Jun 72017 Jun 9

Other

Other8th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies, HEART 2017
CountryGermany
CityBochum
Period17/6/717/6/9

    Fingerprint

Keywords

  • Cluster
  • GPU
  • PEACH3
  • TCA

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Kaneda, T., Sakai, R., Nishikawa, N., Hanawa, T., Tsuruta, C., & Amano, H. (2017). Performance Evaluation of PEACH3: Field-Programmable Gate Array Switch for Tightly Coupled Accelerators. In Proceedings of the 8th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies, HEART 2017 [9] Association for Computing Machinery. https://doi.org/10.1145/3120895.3120911