Hadoop triggered opt/electrical data-center orchestration architecture for reducing power consumption

Akira Yamashita, Wataru Muro, Masayuki Hirono, Takehiro Sato, Satoru Okamoto, Naoaki Yamanaka, Malathi Veeraraghavan

研究成果: Conference contribution

5 被引用数 (Scopus)

抄録

In this paper, a data-center network (DCN) system that distinguishes Hadoop job types and allocates optical/electrical circuits to data flows depending on the types automatically is proposed. The proposed system calculates the predicted shuffle value (PSV) of the Hadoop job and determines which type of flow is allocated by comparing the PSV and the threshold value. The PSV can be calculated from a part of the input data based on the shuffle ratio, which is known to have a relationship to the heaviness of the shuffle phase of the Hadoop job. The threshold of PSV can be dynamically changed according to the network load condition to exploit the optical circuit efficiently. By orchestrating the DCN and the Hadoop system, the proposed system achieves the reduction of power consumption. In this study, the orchestration part of the proposed DCN system is implemented and the feasibility of switching data flows between optical and electrical circuits is verified.

本文言語English
ホスト出版物のタイトルICTON 2017 - 19th International Conference on Transparent Optical Networks
出版社IEEE Computer Society
ISBN(電子版)9781538608586
DOI
出版ステータスPublished - 2017 9 1
イベント19th International Conference on Transparent Optical Networks, ICTON 2017 - Girona, Catalonia, Spain
継続期間: 2017 7 22017 7 6

出版物シリーズ

名前International Conference on Transparent Optical Networks
ISSN(電子版)2162-7339

Other

Other19th International Conference on Transparent Optical Networks, ICTON 2017
国/地域Spain
CityGirona, Catalonia
Period17/7/217/7/6

ASJC Scopus subject areas

  • コンピュータ ネットワークおよび通信
  • 電子工学および電気工学
  • 電子材料、光学材料、および磁性材料

フィンガープリント

「Hadoop triggered opt/electrical data-center orchestration architecture for reducing power consumption」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル