A software implementation and evaluation for searching and extracting information of application layer from network traffic

Recently, new network services in the Internet have been proposed and studied, which use special information obtained from a router or a gateway. Although Layer-7 inspection software on a gateway is available, existing inspection software does not support application protocols for providing search and extraction of information, such as HTTP/1.1 gzip encode and chunk encode processing. In this paper, an open source software, SLIM (Smart Linux Interface Monitor) was implemented and evaluated. It provides TCP stream re-construction function and the HTTP/1.1 processing for supporting string extraction from Linux eth devices and pcap files using libpcap libraly. SLIM implements a TCP stream re-construction algorithm based on context-switch processing in order to reduce the required amount of memory. Simulation results show that SLIM achieves 21.3Mbps processing at a gateway, and when directly reading pcap files, it provides 86.8Mbps for storing PostgreSQL and 1.12Gbps for directly storing files. SLIM can analyze a 1.5TB enterprise traffic file and hundle 730,000 connections with 5.87GB memory consumption in offline mode. We confirmed that SLIM maintains its stable operation on a Laboratory gateway over three months.