Packet Pre-filtering for Network Intrusion Detection

1. Packet Pre-filtering for Network Intrusion Detection Author： Ioannis Sourdis, Vasilis Dimopoulos, Dionisios Pnevmatikatos and Stamatis Vassiliadis Publisher： ANCS’06 Presenter： Zong-Lin Sie Date： 2011/01/05

2. Introduction • Observe that it is very rare for a single incoming packet to fully or partially match more than a few tens of IDS rules. • Select a small portion from each IDS rule to be matched in the pre-filtering step. • We also propose and evaluate the cost and performance of a reconfigurable architecture that uses multiple processing engines in order to exploit the benefits of pre-filtering.

3. Introduction • In the past years, many researchers have worked on reconfigurable IDS focusing mostly on the payload scan. [3,4,7,9-11,15,16] • In this paper we introduce a packet pre-filtering approach that header matching and a relatively low-cost pattern matching module can filter out the majority of the Snort rules and point out a small subset to be fully matched.

4. Packet Pre-filtering • Most hardware-based techniques suffer from the limitation that they search the payload for all patterns in the entire rule set while ignoring rule headers. • In essence, they search for thousands of patterns while the packet header might specify that we are interested in only a few tens or so patterns.

5. Packet Pre-filtering • Our key observation in packet pre-filtering is that matching a small part of each rule’s payload combined with matching the header information can substantially reduce the set of the possibly matching rules compared to using only header matching as in previously proposed approaches [5]. • The pre-filtering module is designed for reconfigurable hardware and therefore can update its supported IDS rule set via reconfiguration.

6. Packet Pre-filtering depend on rule’s definition

7. Packet Pre-filtering • Header Matching： ( using simple comparator ) Performs a more fine-grained grouping than Snort. source and destination ports ： additional parameters for TCP/UDP rules and the ICMP type for ICMP rules. The header fields are registered and forwarded to a pipelined comparator module. This module discovers all active rule sets and can also be used to inform the software of the best applicable rule set

8. Packet Pre-filtering • Partial Pattern Matching： Packet payload is scanned using partial search patterns. We select the first pattern and match a constant number of its prefix bytes. If the pattern is shorter than the selected number of prefix bytes then the full pattern is matched. The static pattern matching is performed utilizing DCAM, a pre-decoding technique [15].

9. DCAM[15] , FCCM 2004 • Basic CAM

10. DCAM[15] , FCCM 2004 • DCAM：

11. DCAM[15] , FCCM 2004 • DCAM detail：

12. DCAM[15] , FCCM 2004 • Increase performance：

13. Packet Pre-filtering • Bitmask： Each bit of the mask corresponds to a single rule. When the header and pattern matching performed in pre-filtering module is equivalent to a complete IDS rule, this rule should be directly reported and no further matching is required.

14. Packet Pre-filtering • Priority Encoder： Outputs sequentially all the positions of the active bits in the bitmask (possibly matching rules IDs).

15. Packet Pre-filtering • Priority Encoder： Stage N Stage N+1 • Fixed priority • Pipelined→ scales well as the #inputs • increases • Encodes/outputs every SET bit of the • bitmask • Binary tree like structure • Bitmask → leafs of the tree

16. Integrating Pre-filtering In An Intrusion Detecting System • Pre-filtering points out the rules to be fully matched • Specialized Engines: For each candidate rule: • A PE is reserved • A firmware is transferred to the PE • PE released  rule match , rule mismatch or End of packet • Coprocessors (Static patterns & Regular expression matching) perform payload scan • PEs select the coprocessor info and decide whether a rule matches or not

17. Integrating Pre-filtering In An Intrusion Detecting System • If ( candidate rules > PEs ) ? • # of PEs is the threshold defined by the system designer. ( i.e. 32 PEs in this design ) • In order to guarantee performance, the packet is reported, Admin policies determine the next step (i.e. drop)

18. Experimental results • Defcon11 traces • 9 trace files • ~10 millions packets • 4.6 million packets have payload • payload length: • Mean 698 bytes • Max 1460 bytes • SNORT v2.4 • 3,191 rules • 2,271 rules with payload description • ( 71.2% ) • 920 only header ( 28.8% ) • rules grouped into 381 rule sets

19. Simulation results • Pre-Filtering setup: • Header matching Scr/destIP+Port, Protocol • Payload Pattern match  2-10 chars prefix match • For prefix>2 chars: Average Candidate rules per packet= 1~3 ( per trace ) • Overall average: 1.8 rules per packet • Only header match ~45 rules per packet

20. Simulation results • Payload prefix match= 2 chars: max 63 candidate rules per packets • Payload prefix match>=4 chars: max 32 candidate rules per packets • What does this mean: • Max number of rules for further processing1% or 32 out of 3,200 rules • The Max degree of parallelism needed (processing engines, threads etc.)

21. Implementation Results • Present the implementation results of two packet pre-filtering designs. Xilinx Virtex4 FPGA devices contain up to 90,000 slices

22. Implementation Results • All the packet pre-filtering sub-modules are fine-grain pipelined and therefore the operating frequency of the designs is relatively high: • Datapath 8 bits/cycle: • Virtex2: 2.7 Gbps • Virtex4: 4 Gbps • Area 11K slices • (medium-small FPGA) • Datapath 32 bits/cycle: • Virtex2: 9.7 Gbps • Virtex4: 14 Gbps • Area 15K slices • (medium-small FPGA) • Priority encoder takes most of the area

23. Conclusions • Performance ： 99% of the IDS rules per incoming packet do not need further processing (in Defcon11 traces), without loosing detection precision. • Requirements： (1) Lightweight system, requires 10-15K slices, can fit in a medium-sized FPGA (2) Can be integrated in both HW or SW based systems