Lineage-based probabilistic event stream processing

Zhitao Shen, Hideyuki Kawashima, Hiroyuki Kitagawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Many sensor network applications such as monitoring video camera streams or management of RFID data streams or tiny sensor data streams from Motes or SunSPOTs require the ability to detect composite events over high-volume data streams. Sensor data inputs from physical world are usually noisy, incomplete and unreliable because sensing devices are usually unreliable. Thus they are usually expressed with probability in ubiquitous sensor network environment. To manage this kind of data, the probabilistic event stream processing system is a natural consequence. In this paper, we propose a query language to support probabilistic queries for composite event stream matching. The language allows users to express Kleene closure patterns for complex event detection in physical world. We also propose a working framework for query processing over probabilistic event streams. Our method first detects sequence patterns over probabilistic data streams by using a new data structure, AIG which handles a record sets of active states with a NFA-based approach. After detecting active states, our method then computes the probability of each detected sequence pattern on its lineage. That is, query processing and confidence computation are decoupled. By the benefit of lineage, the probability of an output event can be directly calculated without considering the query plan. We conduct a performance evaluation of our method comparing with naive one which is called possible worlds approach. The result clearly shows the effectiveness of our approach. While our approach shows scalable throughput, naive approach degrades its performance rapidly. The experiments are conducted with the window size, the number of event types and the number of alternatives.

Original languageEnglish
Title of host publication2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008
Pages106-113
Number of pages8
DOIs
Publication statusPublished - 2009 Jul 23
Externally publishedYes
Event2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008 - Beijing, China
Duration: 2008 Apr 272008 Apr 30

Other

Other2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008
CountryChina
CityBeijing
Period08/4/2708/4/30

Fingerprint

Query processing
Sensor networks
Processing
Query languages
Sensors
Composite materials
Video cameras
Radio frequency identification (RFID)
Data structures
Throughput
Monitoring
Experiments

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Software

Cite this

Shen, Z., Kawashima, H., & Kitagawa, H. (2009). Lineage-based probabilistic event stream processing. In 2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008 (pp. 106-113). [4839090] https://doi.org/10.1109/MDMW.2008.12

Lineage-based probabilistic event stream processing. / Shen, Zhitao; Kawashima, Hideyuki; Kitagawa, Hiroyuki.

2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008. 2009. p. 106-113 4839090.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Shen, Z, Kawashima, H & Kitagawa, H 2009, Lineage-based probabilistic event stream processing. in 2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008., 4839090, pp. 106-113, 2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008, Beijing, China, 08/4/27. https://doi.org/10.1109/MDMW.2008.12
Shen Z, Kawashima H, Kitagawa H. Lineage-based probabilistic event stream processing. In 2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008. 2009. p. 106-113. 4839090 https://doi.org/10.1109/MDMW.2008.12
Shen, Zhitao ; Kawashima, Hideyuki ; Kitagawa, Hiroyuki. / Lineage-based probabilistic event stream processing. 2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008. 2009. pp. 106-113
@inproceedings{6c8d64b38291440082057b95982f361a,
title = "Lineage-based probabilistic event stream processing",
abstract = "Many sensor network applications such as monitoring video camera streams or management of RFID data streams or tiny sensor data streams from Motes or SunSPOTs require the ability to detect composite events over high-volume data streams. Sensor data inputs from physical world are usually noisy, incomplete and unreliable because sensing devices are usually unreliable. Thus they are usually expressed with probability in ubiquitous sensor network environment. To manage this kind of data, the probabilistic event stream processing system is a natural consequence. In this paper, we propose a query language to support probabilistic queries for composite event stream matching. The language allows users to express Kleene closure patterns for complex event detection in physical world. We also propose a working framework for query processing over probabilistic event streams. Our method first detects sequence patterns over probabilistic data streams by using a new data structure, AIG which handles a record sets of active states with a NFA-based approach. After detecting active states, our method then computes the probability of each detected sequence pattern on its lineage. That is, query processing and confidence computation are decoupled. By the benefit of lineage, the probability of an output event can be directly calculated without considering the query plan. We conduct a performance evaluation of our method comparing with naive one which is called possible worlds approach. The result clearly shows the effectiveness of our approach. While our approach shows scalable throughput, naive approach degrades its performance rapidly. The experiments are conducted with the window size, the number of event types and the number of alternatives.",
author = "Zhitao Shen and Hideyuki Kawashima and Hiroyuki Kitagawa",
year = "2009",
month = "7",
day = "23",
doi = "10.1109/MDMW.2008.12",
language = "English",
isbn = "9781424444847",
pages = "106--113",
booktitle = "2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008",

}

TY - GEN

T1 - Lineage-based probabilistic event stream processing

AU - Shen, Zhitao

AU - Kawashima, Hideyuki

AU - Kitagawa, Hiroyuki

PY - 2009/7/23

Y1 - 2009/7/23

N2 - Many sensor network applications such as monitoring video camera streams or management of RFID data streams or tiny sensor data streams from Motes or SunSPOTs require the ability to detect composite events over high-volume data streams. Sensor data inputs from physical world are usually noisy, incomplete and unreliable because sensing devices are usually unreliable. Thus they are usually expressed with probability in ubiquitous sensor network environment. To manage this kind of data, the probabilistic event stream processing system is a natural consequence. In this paper, we propose a query language to support probabilistic queries for composite event stream matching. The language allows users to express Kleene closure patterns for complex event detection in physical world. We also propose a working framework for query processing over probabilistic event streams. Our method first detects sequence patterns over probabilistic data streams by using a new data structure, AIG which handles a record sets of active states with a NFA-based approach. After detecting active states, our method then computes the probability of each detected sequence pattern on its lineage. That is, query processing and confidence computation are decoupled. By the benefit of lineage, the probability of an output event can be directly calculated without considering the query plan. We conduct a performance evaluation of our method comparing with naive one which is called possible worlds approach. The result clearly shows the effectiveness of our approach. While our approach shows scalable throughput, naive approach degrades its performance rapidly. The experiments are conducted with the window size, the number of event types and the number of alternatives.

AB - Many sensor network applications such as monitoring video camera streams or management of RFID data streams or tiny sensor data streams from Motes or SunSPOTs require the ability to detect composite events over high-volume data streams. Sensor data inputs from physical world are usually noisy, incomplete and unreliable because sensing devices are usually unreliable. Thus they are usually expressed with probability in ubiquitous sensor network environment. To manage this kind of data, the probabilistic event stream processing system is a natural consequence. In this paper, we propose a query language to support probabilistic queries for composite event stream matching. The language allows users to express Kleene closure patterns for complex event detection in physical world. We also propose a working framework for query processing over probabilistic event streams. Our method first detects sequence patterns over probabilistic data streams by using a new data structure, AIG which handles a record sets of active states with a NFA-based approach. After detecting active states, our method then computes the probability of each detected sequence pattern on its lineage. That is, query processing and confidence computation are decoupled. By the benefit of lineage, the probability of an output event can be directly calculated without considering the query plan. We conduct a performance evaluation of our method comparing with naive one which is called possible worlds approach. The result clearly shows the effectiveness of our approach. While our approach shows scalable throughput, naive approach degrades its performance rapidly. The experiments are conducted with the window size, the number of event types and the number of alternatives.

UR - http://www.scopus.com/inward/record.url?scp=67650665059&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67650665059&partnerID=8YFLogxK

U2 - 10.1109/MDMW.2008.12

DO - 10.1109/MDMW.2008.12

M3 - Conference contribution

AN - SCOPUS:67650665059

SN - 9781424444847

SP - 106

EP - 113

BT - 2008 9th International Conference on Mobile Data Management Workshops, MDMW 2008

ER -