TCP reassembler for layer7-aware network intrusion detection/prevention systems

Miyuki Hanaoka, Makoto Shemamura, Kenji Kono

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Exploiting layer7 context is an effective approach to improving the accuracy of detecting malicious messages in network intrusion detection/prevention systems (NIDS/NIPSs). Layer7 context enables us to inspect message formats and the message exchanged order. Unfortunately, layer7-aware NIDS/NIPSs pose crucial implementation issues because they require full TCP and IP reassembly without losing 1) complete prevention, 2) performance, 3) application transparency, or 4) transport transparency. Complete prevention means that the NIDS/NIPS should prevent malicious messages from reaching target applications. Application transparency means not requiring any modifications to and/or reconfiguration of server and client applications. Transport transparency is not to disrupt the end-to-end semantics of TCP/IP. To the best of our knowledge, none of the existing approaches meet all of these requirements. We have developed an efficient mechanism for layer7-aware NIDS/NIPSs that does meet the above requirements. Our store-through does this by forwarding each out-of-order or IP-fragmented packet immediately after copying the packet even if it has not been checked yet by an NIDS/NIPS sensor. Although the forwarded packet might turn out to be a part of an attack message, the storethrough mechanism can successfully defend against the attack by blocking one of the subsequent packets that contain another part of attack message. Testing of a prototype in Linux kernel 2.4.30 demonstrated that the overhead of our mechanism is negligible compared with that of a simple IP forwarder even with the presence of out-of-order and IP-fragmented packets. In addition, the experimental results suggest that the CPU and memory usage incurred by our store-through is not significant.

Original languageEnglish
Pages (from-to)2019-2032
Number of pages14
JournalIEICE Transactions on Information and Systems
VolumeE90-D
Issue number12
DOIs
Publication statusPublished - 2007 Dec

Keywords

  • Layer7-aware NIDS/NIPS
  • Network intrusion detection/prevention systems
  • Network security
  • Tcp reassembler

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'TCP reassembler for layer7-aware network intrusion detection/prevention systems'. Together they form a unique fingerprint.

  • Cite this