NCAP - Distributed network capture with shared analysis

We describe NCAP, a new network capturing tool for distributed sensor systems. NCAP operates on messages rather than on packets, and so performs full IP reassembly at the point of measurement. The resulting data can either be managed as files or be transmitted as encapsulated UDP datagrams either unicast or multicast. The NCAP library is highly portable with C and Python interfaces, and has a plug-in mechanism whereby analysis logic can be written discretely and without regard to the handling of encapsulated datagrams or files. The primary application of NCAP is the Security Information Exchange, where cooperating distributed sensor operators now submit captured DNS traffic to a centralized location for subsequent long-running analysis. Examples of value added reprocessing and rebroadcast will be shown, as well as samples of captured traffic and of possible security problems illuminated by our analysis. These results will show that NCAP makes it possible to capture, share, and analyze live network data on a larger scale than has ever been done.