ANFIS Classifier for Network Intrusion Detection System

1. ANFIS Classifier for Network IntrusionDetection System دكترمحسن كاهاني http://www.um.ac.ir/~kahani/

2. Network Intrusion Detection سيستمهاي خبره و مهندسي دانش-دكتر كاهاني • Widespread use of computer networks • Number of attacks and New hacking tools and Intrusive methods • An Intrusion Detection System (IDS) is one way of dealing with suspicious activities within a network. • IDS • Monitors the activities of a given environment • Decides whether these activities are malicious (intrusive) or legitimate (normal).

3. Soft Computing and IDS سيستمهاي خبره و مهندسي دانش-دكتر كاهاني • Many soft computing approaches have been applied to the intrusion detection field. • Our Novel Network IDS includes • Neuro-Fuzzy • Fuzzy • Genetic algorithms • Key Contributions • Utilization of outputs of neuro-fuzzy network as linguistic variables which expresses how reliable current output is.

4. KDD cup 99 Dataset • Comparison of different works in IDS area • Needs of Standard dataset for evaluation of computer network IDSes. • Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining Collected and generated TCP dump data of simulated network in the form oftrain-and-testsets of features defined for the connection records. • We name this standard Dataset as KDD cup 99 dataset and will use it for our experiments. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

5. KDD cup 99 Dataset • 41 features derived for each connection. • A label which specifies the status of connection records as either normal or specific attack type. • Features fall in four categories • The intrinsic features e.g. duration of the connection , type of the protocol (tcp, udp, etc), network service (http, telnet, etc), etc. • The content feature e.g. number of failed login attempts etc. • The same host features examine established connections in the past two seconds that have the same destination host as the current connection, and calculate statistics related to the protocol behavior, service, etc • The similar same service features examine the connections in the past two seconds that have the same service as the current connection. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

6. Basic features of individual TCP connections سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

7. Content features within a connection suggested by domain knowledge سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

8. Traffic features computed using a two-second time window سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

9. KDD CUP 99 Sample Data 0,tcp,http,SF,200,4213,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,15,15,0.00,0.00,0.00,0.00,1.00,0.00,0.00,31,255,1.00,0.00,0.03,0.02,0.00,0.00,0.00,0.00,normal. 0,tcp,http,SF,293,4203,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,2,2,0.00,0.00,0.00,0.00,1.00,0.00,0.00,4,255,1.00,0.00,0.25,0.02,0.00,0.00,0.00,0.00,normal. 0,tcp,http,SF,296,6903,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,2,0.00,0.00,0.00,0.00,1.00,0.00,1.00,2,255,1.00,0.00,0.50,0.03,0.00,0.00,0.00,0.00,normal. 0,udp,domain_u,SF,104,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,2,0.00,0.00,0.00,0.00,1.00,0.00,1.00,56,56,1.00,0.00,1.00,0.00,0.00,0.00,0.00,0.00,normal. 0,udp,domain_u,SF,103,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,2,0.00,0.00,0.00,0.00,1.00,0.00,1.00,66,66,1.00,0.00,1.00,0.00,0.00,0.00,0.00,0.00,normal. 0,udp,domain_u,SF,89,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,2,0.00,0.00,0.00,0.00,1.00,0.00,1.00,76,76,1.00,0.00,1.00,0.00,0.00,0.00,0.00,0.00,normal. 0,udp,domain_u,SF,79,32,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,86,85,0.99,0.02,0.99,0.00,0.00,0.00,0.00,0.00,normal. 0,tcp,smtp,SF,1367,335,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,21,72,0.90,0.10,0.05,0.04,0.00,0.00,0.00,0.00,normal. 184,tcp,telnet,SF,1511,2957,0,0,0,3,0,1,2,1,0,0,1,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,1,3,1.00,0.00,1.00,0.67,0.00,0.00,0.00,0.00,buffer_overflow. 305,tcp,telnet,SF,1735,2766,0,0,0,3,0,1,2,1,0,0,1,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,2,4,1.00,0.00,0.50,0.50,0.00,0.00,0.00,0.00,buffer_overflow. 0,tcp,smtp,SF,1518,405,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,4,0.00,0.00,0.00,0.00,1.00,0.00,1.00,42,108,0.74,0.07,0.02,0.04,0.05,0.00,0.00,0.00,normal. 0,tcp,smtp,SF,1173,403,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,52,116,0.75,0.06,0.02,0.03,0.04,0.00,0.00,0.00,normal. 257,tcp,telnet,SF,181,1222,0,0,0,0,0,1,0,0,0,0,2,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,62,15,0.21,0.05,0.02,0.13,0.03,0.13,0.00,0.00,normal. 0,tcp,smtp,SF,2302,410,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,72,117,0.76,0.04,0.01,0.03,0.03,0.00,0.00,0.00,normal. 1,tcp,smtp,SF,1587,332,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,3,120,1.00,0.00,0.33,0.04,0.00,0.00,0.00,0.00,normal. 0,tcp,smtp,SF,1552,333,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,2,0.00,0.00,0.00,0.00,1.00,0.00,1.00,13,121,0.85,0.15,0.08,0.04,0.00,0.00,0.00,0.00,normal. 0,tcp,finger,SF,10,223,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,23,14,0.22,0.13,0.04,0.29,0.00,0.00,0.00,0.00,normal. 0,tcp,smtp,SF,971,335,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,16,120,0.94,0.12,0.06,0.03,0.00,0.00,0.00,0.00,normal. 1,tcp,smtp,SF,2007,335,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,3,0.00,0.00,0.00,0.00,1.00,0.00,1.00,26,129,0.92,0.12,0.04,0.03,0.00,0.00,0.00,0.00,normal. 0,tcp,finger,SF,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,3,16,0.67,0.67,0.33,0.31,0.00,0.00,0.00,0.00,normal. 0,tcp,smtp,SF,880,327,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,18,195,0.89,0.11,0.06,0.03,0.00,0.00,0.00,0.00,normal. 0,tcp,smtp,SF,4031,322,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,28,205,0.93,0.07,0.04,0.03,0.00,0.00,0.00,0.00,normal. 27,tcp,ftp,SF,916,2720,0,0,0,19,0,1,0,0,0,0,0,0,0,0,0,1,2,2,0.00,0.00,0.00,0.00,1.00,0.00,0.00,5,5,1.00,0.00,0.20,0.00,0.00,0.00,0.00,0.00,normal. 0,tcp,smtp,SF,2012,325,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,15,207,0.27,0.13,0.07,0.03,0.00,0.00,0.00,0.00,normal. 20,tcp,ftp,SF,239,774,0,0,0,4,0,1,0,0,0,0,0,0,0,0,0,1,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,55,34,0.62,0.04,0.02,0.00,0.00,0.00,0.00,0.00,normal. 23,tcp,ftp,SF,342,1072,0,0,0,6,0,1,0,0,0,0,0,0,0,0,0,1,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,65,40,0.62,0.03,0.02,0.00,0.00,0.00,0.00,0.00,normal. 1,tcp,smtp,SF,1609,364,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,4,0.00,0.00,0.00,0.00,1.00,0.00,1.00,75,187,0.37,0.03,0.01,0.03,0.00,0.00,0.00,0.00,normal. 21,tcp,ftp,SF,227,766,0,0,0,4,0,1,0,0,0,0,0,0,0,0,0,1,1,1,0.00,0.00,0.00,0.00,1.00,0.00,0.00,85,50,0.59,0.02,0.01,0.00,0.00,0.00,0.00,0.00,normal. 0,tcp,http,SF,54540,8314,0,0,0,2,0,1,1,0,0,0,0,0,0,0,0,0,2,2,0.00,0.00,0.00,0.00,1.00,0.00,0.00,111,111,1.00,0.00,0.01,0.00,0.00,0.00,0.01,0.01,back. 0,tcp,http,RSTR,53452,2920,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,3,3,0.00,0.00,0.33,0.33,1.00,0.00,0.00,112,112,1.00,0.00,0.01,0.00,0.00,0.00,0.02,0.02,back. 0,tcp,http,SF,54540,8314,0,0,0,2,0,1,1,0,0,0,0,0,0,0,0,0,3,3,0.00,0.00,0.33,0.33,1.00,0.00,0.00,113,113,1.00,0.00,0.01,0.00,0.00,0.00,0.02,0.02,back. 0,icmp,ecr_i,SF,1480,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,19,19,0.00,0.00,0.00,0.00,1.00,0.00,0.00,255,19,0.07,0.02,0.07,0.00,0.00,0.00,0.00,0.00,pod. 0,icmp,ecr_i,SF,1480,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,20,20,0.00,0.00,0.00,0.00,1.00,0.00,0.00,255,20,0.08,0.02,0.08,0.00,0.00,0.00,0.00,0.00,pod. 0,tcp,private,RSTR,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0.00,0.00,1.00,1.00,1.00,0.00,0.00,255,1,0.00,0.02,0.00,0.00,0.00,0.00,0.00,1.00,portsweep. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

10. KDD cup 99 Dataset • Attacks fall into four main categories • DOS (Denial of service): making some computing or memory resources too busy so that they deny legitimate users access to these resources. • R2L (Root to local): unauthorized access from a remote machine according to exploit machine's vulnerabilities. • U2R (User to root):  unauthorized access to local super user (root) privileges using system's susceptibility. • PROBE: host and port scans as precursors to other attacks. An attacker scans a network to gather information or find known vulnerabilities. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

11. KDD Cup 99 Dataset cont. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني • KDD dataset is divided into following record sets: • Training • Testing • Original training dataset was too large for our purpose10% training dataset, was employed here for training phase.

12. KDD Cup 99 Sample Distribution THE SAMPLE DISTRIBUTIONS ON THE SUBSET OF 10% DATA OF KDD CUP 99 DATASET THE SAMPLE DISTRIBUTIONS ON THE TEST DATA WITH THE CORRECTED LABELS OF KDD CUP 99 DATASET سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

13. ANFIS • ANFIS as an adaptive neuro-fuzzy inference system • Ability to construct models solely based on the target system sample (Learning) • Adopt itself through repeated training (Adaptation) • Above abilities among others qualifies ANFIS as a fuzzy classifier for IDS • Here we use ANFIS as Neuro-fuzzy classifier to detect intrusions in computer networks based on KDD cup 99 datasets. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

14. Generating Target fuzzy Inference System • Grid partitioning • all the possible rules are generated based on the number of MFs for each input • For example in a two dimensional input space, with three MFs in the input sets, the number of rules in grid partitioning will result in 9 rules. • Subtractive clustering • Subtractive Clustering is a fast, one-pass algorithm for estimating the number of clusters and the cluster centers in a set of data. • The clusters’ information obtained by this method is used for determining the initial number of rules and antecedent membership functions, which is used for identifying the FIS. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

15. Initial SYSTEM ARCHITECTURE • Features of KDD had all forms continuous, discrete, and symbolic. • Preprocessing: mapping symbolic valued attributes to numeric ones. • 150000 randomly selected points of the subset of 10% of data is used as training. • Randomly 40000 records of data selected as the checking data (used for validating model). • Five trails of 40000 sampled connections from the source of training dataset that does not overlap neither with training set nor each others, have been carried out as the testing data. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

16. Initial SYSTEM ARCHITECTURE • Subtractive Clustering Method with ra=0.5 (neighborhood radius) partitions the training data and generates an FIS structure. • Then for further fine-tuning and adaptation of membership functions, training dataset was used for training ANFIS while the checking dataset was used for validating the model identified. • The final ANFIS contains 212 nodes and a total number of 284 fitting parameters, of which 164 are premise parameters and 84 are consequent parameters. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

17. InitialSYSTEM ARCHITECTURE • Training ANFIS causes further fine-tuning and adaptation of initial membership functions. Initial and final membership functions of some input features are illustrated here. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

18. InitialSYSTEM ARCHITECTURE • ANFIS structure has one output, basically. • We need to gain an approximate class number by rounding off the output number of ANFIS. Γ is the parameter for rounding off which gives us the integer value. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

19. Standard metrics for evaluating network IDSes • Some Definition • Detection rate is computed as the ratio between the number of correctly detected attacks and the total number of attacks, • False alarm (false positive) rate is computed as the ratio between the number of normal connections that is incorrectly misclassified as attacks and the total number of normal connections. • Classification rate is defined as ratio between number of test instances correctly classified and the total number of test instances classified. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

20. Results • False Alarm, Detection and classification rate for training and checking data, Γ=0.5 • Error measures vs. epoch numbers for the training dataset سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

21. Results • Experiment 1 • All the records of labeled test dataset (corrected) as the testing data to evaluate our classifiers • False Alarm, Detection and Classification Rate for test data of first experiment; Γ=0.5 سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

22. Results • Experiment 2 • 5 trials of 40000 randomly selected 40000 samples. • Average of the resulting. • We compare our classifiers with different fuzzy algorithms. • Comparing False Alarm, Detection and complexity of different algorithms. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

23. Final System architecture سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

24. Proposed System(Data Sources) سيستمهاي خبره و مهندسي دانش-دكتر كاهاني The distribution of the samples in the two subsets that were used for the training SAMPLE DISTRIBUTIONS ON THE FIRST TRAINING AND CHECKING DATA RANDOMLY SELECTED OF 10% DATA OF KDD CUP 99 DATASET OF 10% DATA OF KDD CUP 99 DATASET

25. Proposed System(Data Sources) cont. سيستمهاي خبره و مهندسي دانش-دكتر كاهاني SAMPLE DISTRIBUTIONS ON THE SECOND TRAINING AND CHECKING DATA RANDOMLY SELECTED OF 10% DATA OF KDD CUP 99 DATASET OF 10% DATA OF KDD CUP 99 DATASET

26. Proposed System(ANFIS Classifiers) سيستمهاي خبره و مهندسي دانش-دكتر كاهاني The subtractive clustering method with ra=0.5 (neighborhood radius) has been used to partition the training sets and generate an FIS structure for each ANFIS. For further fine-tuning and adaptation of membership functions, training sets were used for training ANFIS. Each ANFIS trains at 50 epochs of learning and final FIS that is associated with the minimum checking error has been chosen. All the MFs of the input fuzzy sets were selected in the form of Gaussian functions with two parameters.

27. Proposed System(The Fuzzy Decision Module) سيستمهاي خبره و مهندسي دانش-دكتر كاهاني • A five‑input, single‑output of Mamdani fuzzy inference system • Centroid of area defuzzification • Each input output fuzzy set includes two MFs • All the MFs are Gaussian functions which are specified by four parameters. • The output of the fuzzy inference engine, which varies between -1 and 1, • Sspecifies how intrusive the current record is, • 1 to show completely intrusive and ‑1 for completely normal FUZZY ASSOCIATIVE MEMORY FOR THE PROPOSED FUZZY INFERENCE RULES

28. Proposed System(Genetic Algorithm Module) سيستمهاي خبره و مهندسي دانش-دكتر كاهاني A chromosome consists of 320 bits of binary data. 8 bits of a chromosome determines one parameter out of the four parameters of an MF.

29. Proposed System(Some Metrics) سيستمهاي خبره و مهندسي دانش-دكتر كاهاني • Cost Per Example • Where CM is a confusion matrix • Each column corresponds to the predicted class, while rows correspond to the actual classes. An entry at row i and column j, CM (i, j), represents the number of misclassified instances that originally belong to class i, although incorrectly identified as a member of class j. The entries of the primary diagonal, CM (i,i), stand for the number of properly detected instances. • C is a cost matrix • As well as CM,Entry C(i,j) represents the cost penalty for misclassifying an instance belonging to class i into class j. • N represents the total number of test instances, • m is the number of the classes in classification.

30. Proposed System(Fitness Function For GA) سيستمهاي خبره و مهندسي دانش-دكتر كاهاني • Two different fitness functions • Cost Per Example with equal misclassification costs • cost per examples used for evaluating results of the KDD'99 competition

31. Proposed System(Data Sources For GA) THE SAMPLE DISTRIBUTIONS ON THE SELECTED SUBSET OF 10% DATA OF KDD CUP 99 DATASET FOR THE OPTIMIZATION PROCESS WHICH IS USED BY GA سيستمهاي خبره و مهندسي دانش-دكتر كاهاني

32. Results سيستمهاي خبره و مهندسي دانش-دكتر كاهاني 10 subsets of training data for both series were used for the classifiers. The genetic algorithm was performed three times, each time for one of the five series of selected subsets. Totatally 150 different structures were used and the result is the average of the results of this 150 structures. Two different training datasets for training the classifiers and two different fitness functions to optimize the fuzzy decision-making module were used. ABBREVIATIONS USED FOR OUR APPROACHES

33. Results cont. CLASSIFICATION RATE, DETECTION RATE(DTR), FALSE ALARM RATE (FA) AND COST PER EXAMPLE OF KDD(CPE) FOR THE DIFFERENT APPROACHES OF ESC-IDS ON THE TEST DATASET WITH CORRECTED LABELS OF KDD CUP 99 DATASET CLASSIFICATION RATE, DETECTION RATE (DTR), FALSE ALARM RATE (FA) AND COST PER EXAMPLE OF KDD (CPE) FOR THE DIFFERENT ALGORITHMS PERFORMANCES ON THE TEST DATASET WITH CORRECTED LABELS OF KDD CUP 99 DATASET (N/R STANDS FOR NOT REPORTED) سيستمهاي خبره و مهندسي دانش-دكتر كاهاني