In-switch approximate processing: Delayed tasks management for MapReduce applications

Koya Mitsuzuka, Ami Hayashi, Michihiro Koibuchi, Hideharu Amano, Hiroki Matsutani

研究成果: Conference contribution

2 被引用数 (Scopus)

抄録

In MapReduce, the parallel processing performance is often limited by only a few compute nodes that delay to complete given tasks. Although various techniques have been invented to handle such stragglers, these techniques mostly impose a burden on master node to monitor the progress of all the compute nodes, resulting in a new bottleneck as the number of compute nodes increases. As an alternative approach, in this paper, we propose to move such straggler management burden from master node to network switch that connects the master and compute nodes, because all the information goes through the switch. More specifically, the proposed network switch monitors output packets from Map tasks to detect stragglers. When detected, the proposed switch generates a response instead of the straggler based on the outputs of the other normal Map tasks, so that Reduce tasks can be started without delay. We introduce some approximate techniques for the proxy computation and response at the switch; thus our switch is called 'ApproxSW.' We implement ApproxSW on NetFPGA-SUME board that has four 10Gbit Ethernet (10GbE) interfaces and a Virtex-7 FPGA. An experiment shows that the ApproxSW functions do not degrade the original 10GbE switch performance. We also analyze the accuracy of the proxy computation and response for stragglers and show that the proposed approximation based on task similarity achieves the best accuracy.

本文言語English
ホスト出版物のタイトル2017 27th International Conference on Field Programmable Logic and Applications, FPL 2017
編集者Diana Gohringer, Dirk Stroobandt, Nele Mentens, Marco Santambrogio, Jari Nurmi
出版社Institute of Electrical and Electronics Engineers Inc.
ISBN(電子版)9789090304281
DOI
出版ステータスPublished - 2017 10月 2
イベント27th International Conference on Field Programmable Logic and Applications, FPL 2017 - Gent, Belgium
継続期間: 2017 9月 42017 9月 6

出版物シリーズ

名前2017 27th International Conference on Field Programmable Logic and Applications, FPL 2017

Other

Other27th International Conference on Field Programmable Logic and Applications, FPL 2017
国/地域Belgium
CityGent
Period17/9/417/9/6

ASJC Scopus subject areas

  • コンピュータ ネットワークおよび通信
  • コンピュータ サイエンスの応用
  • ハードウェアとアーキテクチャ
  • ソフトウェア

フィンガープリント

「In-switch approximate processing: Delayed tasks management for MapReduce applications」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル