Task allocation scheme based on computational and network resources for heterogeneous Hadoop clusters

Tomohiro Matsuno, Bijoy Chand Chatterjee, Eiji Oki, Malathi Veeraraghavan, Satoru Okamoto, Naoaki Yamanaka

研究成果: Conference contribution

2 被引用数 (Scopus)

抄録

This paper aims to design a Hadoop system and evaluates the performance of a task allocation scheme. The task allocation scheme splits each job into tasks using an appropriate splitting ratio, and assigns tasks to slave servers based on server processing performance and network resource availability. We experimentally evaluate the performance of the scale out of the task allocation scheme with five machines. We focus on the configuration of jobtracker and tasktracker in Hadoop. In cases with heterogeneous Hadoop clusters, we distribute task blocks to high-capability slaves with proportionally larger-sized tasks than to low-capability slaves. We create an environment in which high-capability slaves perform more work than low-capability slaves. The experimental testbed results indicate that the task allocation scheme is effective.

本文言語English
ホスト出版物のタイトル2016 IEEE 17th International Conference on High Performance Switching and Routing, HPSR 2016
出版社IEEE Computer Society
ページ200-205
ページ数6
2016-July
ISBN(電子版)9781479989508
DOI
出版ステータスPublished - 2016 7 28
イベント17th IEEE International Conference on High Performance Switching and Routing, HPSR 2016 - Yokohama, Japan
継続期間: 2016 6 142016 6 17

Other

Other17th IEEE International Conference on High Performance Switching and Routing, HPSR 2016
国/地域Japan
CityYokohama
Period16/6/1416/6/17

ASJC Scopus subject areas

  • ハードウェアとアーキテクチャ
  • 電子工学および電気工学

フィンガープリント

「Task allocation scheme based on computational and network resources for heterogeneous Hadoop clusters」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル