Designing a Hadoop system based on computational resources and network delay for wide area networks

Tomohiro Matsuno, Bijoy Chand Chatterjee, Nattapong Kitsuwan, Eiji Oki, Malathi Veeraraghavan, Satoru Okamoto, Naoaki Yamanaka

研究成果: Article査読

抄録

This paper proposes a Hadoop system that considers both slave server’s processing capacity and network delay for wide area networks to reduce the job processing time. The task allocation scheme in the proposed Hadoop system divides each individual job into multiple tasks using suitable splitting ratios and then allocates the tasks to different slaves according to the computational capability of each server and the availability of network resources. We incorporate software-defined networking to the proposed Hadoop system to manage path computation elements and network resources. The performance of proposed Hadoop system is experimentally evaluated with fourteen machines located in the different parts of the globe using a scale-out approach. A scale-out experiment using the proposed and conventional Hadoop systems is conducted by executing both single job and multiple jobs. The practical testbed and simulation results indicate that the proposed Hadoop system is effective compared to the conventional Hadoop system in terms of processing time.

本文言語English
ページ(範囲)13-25
ページ数13
ジャーナルTelecommunication Systems
70
1
DOI
出版ステータスPublished - 2019 1 15

ASJC Scopus subject areas

  • 電子工学および電気工学

フィンガープリント

「Designing a Hadoop system based on computational resources and network delay for wide area networks」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル