TCP reassembler for layer7-aware network intrusion detection/prevention systems

Miyuki Hanaoka, Makoto Shemamura, Kenji Kono

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Exploiting layer7 context is an effective approach to improving the accuracy of detecting malicious messages in network intrusion detection/prevention systems (NIDS/NIPSs). Layer7 context enables us to inspect message formats and the message exchanged order. Unfortunately, layer7-aware NIDS/NIPSs pose crucial implementation issues because they require full TCP and IP reassembly without losing 1) complete prevention, 2) performance, 3) application transparency, or 4) transport transparency. Complete prevention means that the NIDS/NIPS should prevent malicious messages from reaching target applications. Application transparency means not requiring any modifications to and/or reconfiguration of server and client applications. Transport transparency is not to disrupt the end-to-end semantics of TCP/IP. To the best of our knowledge, none of the existing approaches meet all of these requirements. We have developed an efficient mechanism for layer7-aware NIDS/NIPSs that does meet the above requirements. Our store-through does this by forwarding each out-of-order or IP-fragmented packet immediately after copying the packet even if it has not been checked yet by an NIDS/NIPS sensor. Although the forwarded packet might turn out to be a part of an attack message, the storethrough mechanism can successfully defend against the attack by blocking one of the subsequent packets that contain another part of attack message. Testing of a prototype in Linux kernel 2.4.30 demonstrated that the overhead of our mechanism is negligible compared with that of a simple IP forwarder even with the presence of out-of-order and IP-fragmented packets. In addition, the experimental results suggest that the CPU and memory usage incurred by our store-through is not significant.

Original languageEnglish
Pages (from-to)2019-2032
Number of pages14
JournalIEICE Transactions on Information and Systems
VolumeE90-D
Issue number12
DOIs
Publication statusPublished - 2007 Dec

Fingerprint

Intrusion detection
Transparency
Copying
Program processors
Servers
Semantics
Data storage equipment
Sensors
Testing

Keywords

  • Layer7-aware NIDS/NIPS
  • Network intrusion detection/prevention systems
  • Network security
  • Tcp reassembler

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Software
  • Artificial Intelligence
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition

Cite this

TCP reassembler for layer7-aware network intrusion detection/prevention systems. / Hanaoka, Miyuki; Shemamura, Makoto; Kono, Kenji.

In: IEICE Transactions on Information and Systems, Vol. E90-D, No. 12, 12.2007, p. 2019-2032.

Research output: Contribution to journalArticle

@article{80e834396f554f9d8f0a6b13338df54b,
title = "TCP reassembler for layer7-aware network intrusion detection/prevention systems",
abstract = "Exploiting layer7 context is an effective approach to improving the accuracy of detecting malicious messages in network intrusion detection/prevention systems (NIDS/NIPSs). Layer7 context enables us to inspect message formats and the message exchanged order. Unfortunately, layer7-aware NIDS/NIPSs pose crucial implementation issues because they require full TCP and IP reassembly without losing 1) complete prevention, 2) performance, 3) application transparency, or 4) transport transparency. Complete prevention means that the NIDS/NIPS should prevent malicious messages from reaching target applications. Application transparency means not requiring any modifications to and/or reconfiguration of server and client applications. Transport transparency is not to disrupt the end-to-end semantics of TCP/IP. To the best of our knowledge, none of the existing approaches meet all of these requirements. We have developed an efficient mechanism for layer7-aware NIDS/NIPSs that does meet the above requirements. Our store-through does this by forwarding each out-of-order or IP-fragmented packet immediately after copying the packet even if it has not been checked yet by an NIDS/NIPS sensor. Although the forwarded packet might turn out to be a part of an attack message, the storethrough mechanism can successfully defend against the attack by blocking one of the subsequent packets that contain another part of attack message. Testing of a prototype in Linux kernel 2.4.30 demonstrated that the overhead of our mechanism is negligible compared with that of a simple IP forwarder even with the presence of out-of-order and IP-fragmented packets. In addition, the experimental results suggest that the CPU and memory usage incurred by our store-through is not significant.",
keywords = "Layer7-aware NIDS/NIPS, Network intrusion detection/prevention systems, Network security, Tcp reassembler",
author = "Miyuki Hanaoka and Makoto Shemamura and Kenji Kono",
year = "2007",
month = "12",
doi = "10.1093/ietisy/e90-d.12.2019",
language = "English",
volume = "E90-D",
pages = "2019--2032",
journal = "IEICE Transactions on Information and Systems",
issn = "0916-8532",
publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",
number = "12",

}

TY - JOUR

T1 - TCP reassembler for layer7-aware network intrusion detection/prevention systems

AU - Hanaoka, Miyuki

AU - Shemamura, Makoto

AU - Kono, Kenji

PY - 2007/12

Y1 - 2007/12

N2 - Exploiting layer7 context is an effective approach to improving the accuracy of detecting malicious messages in network intrusion detection/prevention systems (NIDS/NIPSs). Layer7 context enables us to inspect message formats and the message exchanged order. Unfortunately, layer7-aware NIDS/NIPSs pose crucial implementation issues because they require full TCP and IP reassembly without losing 1) complete prevention, 2) performance, 3) application transparency, or 4) transport transparency. Complete prevention means that the NIDS/NIPS should prevent malicious messages from reaching target applications. Application transparency means not requiring any modifications to and/or reconfiguration of server and client applications. Transport transparency is not to disrupt the end-to-end semantics of TCP/IP. To the best of our knowledge, none of the existing approaches meet all of these requirements. We have developed an efficient mechanism for layer7-aware NIDS/NIPSs that does meet the above requirements. Our store-through does this by forwarding each out-of-order or IP-fragmented packet immediately after copying the packet even if it has not been checked yet by an NIDS/NIPS sensor. Although the forwarded packet might turn out to be a part of an attack message, the storethrough mechanism can successfully defend against the attack by blocking one of the subsequent packets that contain another part of attack message. Testing of a prototype in Linux kernel 2.4.30 demonstrated that the overhead of our mechanism is negligible compared with that of a simple IP forwarder even with the presence of out-of-order and IP-fragmented packets. In addition, the experimental results suggest that the CPU and memory usage incurred by our store-through is not significant.

AB - Exploiting layer7 context is an effective approach to improving the accuracy of detecting malicious messages in network intrusion detection/prevention systems (NIDS/NIPSs). Layer7 context enables us to inspect message formats and the message exchanged order. Unfortunately, layer7-aware NIDS/NIPSs pose crucial implementation issues because they require full TCP and IP reassembly without losing 1) complete prevention, 2) performance, 3) application transparency, or 4) transport transparency. Complete prevention means that the NIDS/NIPS should prevent malicious messages from reaching target applications. Application transparency means not requiring any modifications to and/or reconfiguration of server and client applications. Transport transparency is not to disrupt the end-to-end semantics of TCP/IP. To the best of our knowledge, none of the existing approaches meet all of these requirements. We have developed an efficient mechanism for layer7-aware NIDS/NIPSs that does meet the above requirements. Our store-through does this by forwarding each out-of-order or IP-fragmented packet immediately after copying the packet even if it has not been checked yet by an NIDS/NIPS sensor. Although the forwarded packet might turn out to be a part of an attack message, the storethrough mechanism can successfully defend against the attack by blocking one of the subsequent packets that contain another part of attack message. Testing of a prototype in Linux kernel 2.4.30 demonstrated that the overhead of our mechanism is negligible compared with that of a simple IP forwarder even with the presence of out-of-order and IP-fragmented packets. In addition, the experimental results suggest that the CPU and memory usage incurred by our store-through is not significant.

KW - Layer7-aware NIDS/NIPS

KW - Network intrusion detection/prevention systems

KW - Network security

KW - Tcp reassembler

UR - http://www.scopus.com/inward/record.url?scp=68249140232&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=68249140232&partnerID=8YFLogxK

U2 - 10.1093/ietisy/e90-d.12.2019

DO - 10.1093/ietisy/e90-d.12.2019

M3 - Article

VL - E90-D

SP - 2019

EP - 2032

JO - IEICE Transactions on Information and Systems

JF - IEICE Transactions on Information and Systems

SN - 0916-8532

IS - 12

ER -