Models and Security Requirements for Intrusion Detection Systems

1. Models and Security Requirements for IDS

2. Overview • Sensitivity and detection as security requirements for IDS • IDS using the security framework based on sensitivity and detection • Combinatorial tools in intrusion detection

3. The system and attack model • The model of the system: • Scenario • What are the elements of the network? • Connectivity • How are these elements connected? • Action • What traffic is sent between these elements?

4. The system and attack model • Scenario • A large network, also called Autonomous System (AS) • AS can have many points of entry, called Border Gateways (BG) of the AS.

5. U U BG BG AS U BG BG The system and attack model • Connectivity • The traffic is generated by external users. • Each user (U) can send traffic to each BG.

6. The system and attack model • Action • The network traffic is a sequence of atomic packets. • The abstraction of a packet: p=(sid, time, poe, pl) sid – the identity of the sender (U) time – a timestamp of the action poe – point of entry (BG) pl – the payload – what is actually sent.

7. The system and attack model • Action (cont.) • At any time, the action in an AS is a stream of packets entering AS through any of its BGs. • Each packet in this stream can trigger an event in the AS.

8. The system and attack model • The model of an attack • Any sequence of c packets, c1, that successfully alters the state of the nodes (hosts) in an AS in order to achieve a specific (malicious) goal. • Let t be the state of the AS at the time instant t. The state may include, for example: • Available bandwidth • Internal states of all hosts within the AS.

9. The system and attack model • The model of an attack (cont.) • We can then define a polynomial time computable predicate (predicates are functions that take binary values) •  (1n,t,t) • n – a security parameter • 1n – input, unary string of length n

10. The system and attack model • The model of an attack (cont.) • Attack • A probability distribution A over all packet sequences ps=(p1,…,pl) • Samples with this distribution can be obtained efficiently (efficiently samplable distribution) • The probability that the experiment E(A) is unsuccessful is negligible, i.e. smaller than 1/p(n), for all positive polynomials p and all sufficiently large n.

11. The system and attack model • The model of an attack (cont.) • Attack (cont.) • The experiment E(D), for any distribution D: • A sequence p of packets is drawn from D • The sequence p is sent to the network • AS turns into the state t • The predicate  (1n,t,t) evaluates to the value b{0,1} • E(D) is successful if b=1.

12. The system and attack model • The model of an attack (cont.) • A class of attacks • C={A1,A2,…} • Normal traffic distribution • Efficiently samplable probability distribution N over the set of packets, such that the probability that the experiment E(N) is successful is negligible.

13. The system and attack model • The model of an IDS • An IDS is a triple of algorithms: • A representation algorithm R (data filtering, formatting, feature selection, etc.) • A data structure algorithm S (data collection, aggregation, knowledge base creation, etc.) • A classification algorithm C (detection in all forms – pattern-based, rule-based, anomaly-based, response, refinement, information tracing, visualization, etc.)

14. The system and attack model • The model of an IDS (cont.) • Two phases in the execution of an IDS: • An initialization phase • A detection phase. • The algorithm S is run in the initialization phase. • The algorithm C is run in the detection phase. • Both S and C use the algorithm R as a subroutine.

15. The system and attack model • The model of an IDS (cont.) • In the initialization phase: • The algorithm S uses the algorithm R to process a stream of packet data obtained from normal traffic distributions or known attack distributions. • The output from the algorithm S is a data structure that will be used in the detection phase. • It is assumed that the traffic generated in the initialization phase is not subject to an attack, unless it simulates a known attack.

16. The system and attack model • The model of an IDS (cont.) • In the detection phase: • The algorithm C is run on the input data structure and a sequence of traffic packets (possibly subject to a known or a new attack). • It returns an assessment of whether the input sequence of packets contains an attack (and if so whether this attack is new). • The algorithm R maps the sequence of packets entering the AS into a fixed-length tuple having a more compact form (e.g. a point in a high-dimensional space)

17. Security requirements for IDS • Given the following: • A security parameter n • Normal traffic distribution N • (Known) attack distributions A1,…,At • N, A1,…,At are efficiently samplable and pairwise disjoint.

18. Security requirements for IDS • An IDS is a triple of polynomial time algorithms R, S, C such that: • Given a sequence of rw packets p, algorithm R returns a d-tuple r. • Given distributions N, A1,…,At, algorithm S returns a data structure ds of size at most m[init]. • Given a data structure ds, a sequence m[det] packets p, a detection window dw and a class of attacks C1, algorithm C returns a classification value out.

19. Security requirements for IDS • IDS data: rw - representation window • the window of packets used in a single execution of R • usually a small value. m[init] - the length of the stream of packets used in the initialization phase.

20. Security requirements for IDS • IDS data (cont.): m[det] - the length of the stream of packets used in the detection phase, to be classified by algorithm S • Considered arbitrarily large, but polynomially dependent on n and rw. dw - Maximum distance between the first and the last packet of an attack sequence within the stream m[det].

21. Security requirements for IDS • In general, rw, d, m[init], m[det] and dw are all bounded by a polynomial in n. • A typical setting: rw=O(n) d=O(1) m[init]=na m[det]=nb rwdwm[det] a,b>1, potentially large constants.

22. Security requirements for IDS • An IDS can satisfy two requirements • Sensitivity • Detection

23. Sensitivity • We would like the output d-tuple of the algorithm R to capture differences between normal traffic and attack traffic. • Capturing these differences is formalized using the notion of computational distinguishability. • We require this distinguishability with respect to a single sample of the distributions, because an attack may be executed only once.

24. Sensitivity • Informal definition of sensitivity: • A is an attack distribution • N is a normal traffic distribution • The sensitivity of a representation algorithm R is defined on the basis of the distinguishability of the packet streams taken from the distributions A and N.

25. Sensitivity • Informal definition of sensitivity (cont.): • The measure of sensitivity is probabilistic: it describes the probability  that an attack distribution A can be distinguished from a normal traffic distribution N. • The definition of sensitivity can be generalized to families of distributions.

26. Detection • The representation algorithm R should give different outputs given fixed-window attack/normal traffic packet streams. • It does not clarify anything about the nature of this difference. • It does not give any constructive algorithm to distinguish which of two different outputs is of which type.

27. Detection • We would like the algorithms S and C to directly provide “good enough” detection properties on arbitrarily large traffic sequences as long as the algorithm R has “good enough” sensitivity properties on small and fixed traffic sequences.

28. Detection • The IDS operates in the following way: • In the first phase, the data structure algorithm S is given access to a stream of m packets and can run the representation algorithm on inputs of length rw. • S is allowed to query both the normal traffic distribution N and several (known) attack distributions A1,…,At. • At the end of the first phase, S returns the data structure ds.

29. Detection • Operation of the IDS (cont.): • A sequence of dw packets q is generated and the classification algorithm C returns an output out saying if q contains a sample from one of the known attacks A1,…,At, or a different (unknown) attack A or no attack at all. • The IDS is successful if this classification is correct.

30. Detection • Informal definition of detection: • If A is an attack distribution (potentially unknown), the IDS will detect that the given packet sequence q originates from A with probability , for any q. • This definition can also be generalized for classes of attack distributions.

31. Detection •  is always smaller than . • An IDS is considered a “good” detector if  is close to . • If A is not distinguishable from N (i.e. =0), then no pair of algorithms S,C can be a detector.

32. Analysis methodology • An ideal methodology to analyze an IDS would prove that it satisfies: • The sensitivity requirement (for some appropriate parameter values) • The detection requirement (for some appropriate parameter values) under the assumption that it satisfies the sensitivity requirement.

33. Analysis methodology • A mathematical proof that an IDS satisfies the sensitivity requirement is difficult to obtain, because of the unpredictable nature of a generic unknown attack. • Because of that, validating the sensitivity of the representation algorithm is performed by simulation.

34. Analysis methodology • Once the sensitivity property is validated for the representation algorithm R, the challenge is to formally prove that the given IDS is a detector.

35. IDS satisfying the framework • IDS-1 • The algorithm C is based on the approximate nearest neighbour search. • IDS-2 • The algorithm C is based on clustering – allows for more than one distribution for normal traffic – the class of detectable attacks with IDS-2 is larger than that of IDS-1.

36. IDS satisfying the framework • Approximate nearest neighbour search problem • V is a vector space of dimension d. •  is a distance function defined over V. • Given a set Q of kd-component vectors in V, an error parameter  and a d-component vector qV, we define the (1+)-approximate nearest neighbour of q as the vector v in Q such that (q,v)(1+)(q,w), for any wQ. • Problem: find the nearest neighbourin Q for any qV.

37. IDS satisfying the framework • Approximate nearest neighbour search problem (cont.) • A solution is a pair of algorithms (Init, Search): • On input an k-size set Q of d-length vectors and parameters  and , the algorithm Init returns a data structure ds. • On input data structure ds, a vector q and parameter , the algorithm Search returns a vector v. • With probability at least , vQ and v is a (1+)-approximate nearest neighbour of q.

38. IDS satisfying the framework • Approximate nearest neighbour search problem (cont.) • The algorithm Init must run in time polynomial in k and d. • The algorithm Search must run in time polynomial in d and log k. • Init is used in the initialization phase (off-line). • Search is used in the detection phase (on-line). • Such algorithms Init and Search exist.

39. Combinatorial tools in ID • We would like to have an IDS with arbitrary detection window. • We start with IDS1=(R1,S1,C1) with the representation window rw1 and detection window dw1=k. • IDS1 with its level of sensitivity can detect attacks having l effective packets.

40. Combinatorial tools in ID • We construct IDS2=(R2,S2,C2) from IDS1, with representation window rw2 and detection window dw2=m. • This can be done by means of a covering set system (l,k,m) – a combinatorial object.

41. Combinatorial tools in ID • Covering set system (covering design) • l,k,m – positive integers. • S – a set of cardinality m. • T={T1,…,Ts} – a set of subsets of S of cardinality k. • T is an (l,k,m)-covering set system for S if for any SiS of cardinality l, there exists a subset TjT such that SiTj.

42. Combinatorial tools in ID • Covering set system (cont.) • Space efficiency of the covering set system T is the cardinality s of T (can be a function of l, k, m). • Time efficiency of T is the running time (as a function of l, k, m) that an algorithm takes to construct T.

43. Combinatorial tools in ID • Starting from IDS1=(R1,S1,C1) with representation window rw1 and detection window dw1=k and given an (l,k,m)-covering set system for S={1,…,m} with time efficiency t and space efficiency s, it is possible to construct IDS2=(R2,S2,C2) with rw2=rw1 and dw2=m, for any m polynomial in k, where C2 runs in time O(t+stime(C1)). • R2=R1, S2=S1.

44. Further reading • G. Di Crescenzo, A. Ghosh, R. Talpade, Towards a Theory of Intrusion Detection, Proceedings of ESORICS 2005, LNCS 3679, pp. 267-286, 2005.