Wire-speed implementation of sliding-window aggregate operator over out-of-order data streams

Yasin Oge, Masato Yoshimi, Takefumi Miyoshi, Hideyuki Kawashima, Hidetsugu Irie, Tsutomu Yoshinaga

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

This paper shows the design and evaluation of an FPGA-based accelerator for sliding-window aggregation over data streams with out-of-order data arrival. We propose an order-agnostic hardware implementation technique for windowing operators based on a one-pass query evaluation strategy called Window-ID, which is originally proposed for software implementation. The proposed implementation succeeds to process out-of-order data items, or tuples, at wire speed due to the simultaneous evaluations of overlapping sliding-windows. In order to verify the effectiveness of the proposed approach, we have also implemented an experimental system as a case study. Our experiments demonstrate that the proposed accelerator with a network interface achieves an effective throughput around 760 Mbps or equivalently nearly 6 million tuples per second, by fully utilizing the available bandwidth of the network interface.

Original languageEnglish
Title of host publicationProceedings - IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013
PublisherIEEE Computer Society
Pages55-60
Number of pages6
ISBN (Print)9780768550862
DOIs
Publication statusPublished - 2013 Jan 1
Externally publishedYes
Event2013 IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013 - Tokyo, Japan
Duration: 2013 Sep 262013 Sep 28

Other

Other2013 IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013
CountryJapan
CityTokyo
Period13/9/2613/9/28

Fingerprint

Interfaces (computer)
Particle accelerators
Wire
Field programmable gate arrays (FPGA)
Agglomeration
Throughput
Hardware
Bandwidth
Experiments

Keywords

  • Data stream processing
  • Disordered data
  • FPGA
  • Sliding-window aggregates
  • Stream punctuation

ASJC Scopus subject areas

  • Hardware and Architecture

Cite this

Oge, Y., Yoshimi, M., Miyoshi, T., Kawashima, H., Irie, H., & Yoshinaga, T. (2013). Wire-speed implementation of sliding-window aggregate operator over out-of-order data streams. In Proceedings - IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013 (pp. 55-60). [6657904] IEEE Computer Society. https://doi.org/10.1109/MCSoC.2013.23

Wire-speed implementation of sliding-window aggregate operator over out-of-order data streams. / Oge, Yasin; Yoshimi, Masato; Miyoshi, Takefumi; Kawashima, Hideyuki; Irie, Hidetsugu; Yoshinaga, Tsutomu.

Proceedings - IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013. IEEE Computer Society, 2013. p. 55-60 6657904.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Oge, Y, Yoshimi, M, Miyoshi, T, Kawashima, H, Irie, H & Yoshinaga, T 2013, Wire-speed implementation of sliding-window aggregate operator over out-of-order data streams. in Proceedings - IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013., 6657904, IEEE Computer Society, pp. 55-60, 2013 IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013, Tokyo, Japan, 13/9/26. https://doi.org/10.1109/MCSoC.2013.23
Oge Y, Yoshimi M, Miyoshi T, Kawashima H, Irie H, Yoshinaga T. Wire-speed implementation of sliding-window aggregate operator over out-of-order data streams. In Proceedings - IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013. IEEE Computer Society. 2013. p. 55-60. 6657904 https://doi.org/10.1109/MCSoC.2013.23
Oge, Yasin ; Yoshimi, Masato ; Miyoshi, Takefumi ; Kawashima, Hideyuki ; Irie, Hidetsugu ; Yoshinaga, Tsutomu. / Wire-speed implementation of sliding-window aggregate operator over out-of-order data streams. Proceedings - IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013. IEEE Computer Society, 2013. pp. 55-60
@inproceedings{2da6356006294232a0fd2f0d91d13445,
title = "Wire-speed implementation of sliding-window aggregate operator over out-of-order data streams",
abstract = "This paper shows the design and evaluation of an FPGA-based accelerator for sliding-window aggregation over data streams with out-of-order data arrival. We propose an order-agnostic hardware implementation technique for windowing operators based on a one-pass query evaluation strategy called Window-ID, which is originally proposed for software implementation. The proposed implementation succeeds to process out-of-order data items, or tuples, at wire speed due to the simultaneous evaluations of overlapping sliding-windows. In order to verify the effectiveness of the proposed approach, we have also implemented an experimental system as a case study. Our experiments demonstrate that the proposed accelerator with a network interface achieves an effective throughput around 760 Mbps or equivalently nearly 6 million tuples per second, by fully utilizing the available bandwidth of the network interface.",
keywords = "Data stream processing, Disordered data, FPGA, Sliding-window aggregates, Stream punctuation",
author = "Yasin Oge and Masato Yoshimi and Takefumi Miyoshi and Hideyuki Kawashima and Hidetsugu Irie and Tsutomu Yoshinaga",
year = "2013",
month = "1",
day = "1",
doi = "10.1109/MCSoC.2013.23",
language = "English",
isbn = "9780768550862",
pages = "55--60",
booktitle = "Proceedings - IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Wire-speed implementation of sliding-window aggregate operator over out-of-order data streams

AU - Oge, Yasin

AU - Yoshimi, Masato

AU - Miyoshi, Takefumi

AU - Kawashima, Hideyuki

AU - Irie, Hidetsugu

AU - Yoshinaga, Tsutomu

PY - 2013/1/1

Y1 - 2013/1/1

N2 - This paper shows the design and evaluation of an FPGA-based accelerator for sliding-window aggregation over data streams with out-of-order data arrival. We propose an order-agnostic hardware implementation technique for windowing operators based on a one-pass query evaluation strategy called Window-ID, which is originally proposed for software implementation. The proposed implementation succeeds to process out-of-order data items, or tuples, at wire speed due to the simultaneous evaluations of overlapping sliding-windows. In order to verify the effectiveness of the proposed approach, we have also implemented an experimental system as a case study. Our experiments demonstrate that the proposed accelerator with a network interface achieves an effective throughput around 760 Mbps or equivalently nearly 6 million tuples per second, by fully utilizing the available bandwidth of the network interface.

AB - This paper shows the design and evaluation of an FPGA-based accelerator for sliding-window aggregation over data streams with out-of-order data arrival. We propose an order-agnostic hardware implementation technique for windowing operators based on a one-pass query evaluation strategy called Window-ID, which is originally proposed for software implementation. The proposed implementation succeeds to process out-of-order data items, or tuples, at wire speed due to the simultaneous evaluations of overlapping sliding-windows. In order to verify the effectiveness of the proposed approach, we have also implemented an experimental system as a case study. Our experiments demonstrate that the proposed accelerator with a network interface achieves an effective throughput around 760 Mbps or equivalently nearly 6 million tuples per second, by fully utilizing the available bandwidth of the network interface.

KW - Data stream processing

KW - Disordered data

KW - FPGA

KW - Sliding-window aggregates

KW - Stream punctuation

UR - http://www.scopus.com/inward/record.url?scp=84892635459&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84892635459&partnerID=8YFLogxK

U2 - 10.1109/MCSoC.2013.23

DO - 10.1109/MCSoC.2013.23

M3 - Conference contribution

AN - SCOPUS:84892635459

SN - 9780768550862

SP - 55

EP - 60

BT - Proceedings - IEEE 7th International Symposium on Embedded Multicore/Manycore System-on-Chip, MCSoC 2013

PB - IEEE Computer Society

ER -