Exploiting Clustering Techniques for Web Session Inference

1. Exploiting Clustering Techniquesfor Web Session Inference A.Bianco, G. Mardente, M. Mellia, M.Munafò, L. Muscariello (Politecnico di Torino)

2. Outline • Web Session Model • Clustering techniques • The proposed algorithm • Performance of the algorithm • Session statistics

3. Web session definition • A single web client generates a succession of TCP flows and think times think time Toff think time Toff • A session here is defined as the set of TCP flows arriving close enough one to each other • For example a threshold can be used to discriminate between think times and inter arrivals of TCP flows

4. Algorithms • A threshold based approach needs a priori knowledge of the source • An adaptive algorithm should be capable to catch traffic variations • This is supposed to be less sensitive to traffic characteristics • Clustering is the chosen approach

5. Proposed algorithm • Three steps • A K-means is used on all samples to obtain a first clustering, K is chosen very large • A hierarchical clustering is used only on representatives of each cluster, K is reduced • A K-means is used on all samples again • To test the algorithm we need a priori known traffic, that is artificially generated

6. First Step: K-means • K is chosen large enough but significantly smaller than the number of samples • The K farthest flows determine the first partition • K-means is performed 1000 iterations on all samples • Each cluster is then represented using a subset of samples, one or two in our algorithm • The mean value (Centroid method) • The gth and (100-g)th percentiles (Single linkage method if g=0) g-th percentile (100-g)-th percentile

7. Second step: a hierarchical method • A hierarchical method is used on only representatives • This method merges clusters until a quality function determines that the optimal number of clusters Nc has been found

8. 70 60 50 40 gamma 30 20 10 0 -10 0 200 400 600 800 1000 1200 1400 Step Gamma function typical behaviour

9. Third Step: K-means • A K-means isperformed on all samples • This last step is not critical but rearranges samples’ positions within clusters that is flows within sessions • It is not CPU time consuming, than it is not critical to use it

10. Performance evaluation • Artificial traffic is generated according to an ON/OFF process • During ON periods a succession of flows is generated using i.i.d. inter-arrivals • In this model inferring is to recognize if an inter arrival is an OFF period or an inter arrival between flows within an ON period • Every time the algorithm does not guess correctly, an error is counted • Suppose all variables are exponentially distributed

11. K=1000 K=1500 10 K=2000 K=2500 1 Percentage of errors 0.1 0.01 0 200 400 600 800 1000 1200 1400 1600 1800 2000 T_{off} First step sensitivity (1/2) • If the initial number of clusters is chosen large enough the method is less error prone • The algorithm is much more sensitive to the value of the idle period

12. Single linkage g=15 Centroid Method g=25 10 10 g=1 g=35 g=5 g=45 1 1 Percentage of errors 0.1 0.1 0.01 0.01 0 0 200 200 400 400 600 600 800 800 1000 1000 1200 1200 1400 1400 1600 1600 1800 1800 2000 2000 T_{off} T_{off} First step sensitivity (2/2) • Performance is sensitive to the choice of the percentile g • When clusters are represented through flows at the border ofthe session the methodis less sensitive totraffic, i.e. g=1 • This is due to the fact that cluster has a long and narrow shape and those representatives well model this fact

13. clustering etha=T_{off}/16 etha=T_{off}/2 etha=T_{off}/32 etha=T_{off}/4 etha=T_{off}/64 10 10 etha=T_{off}/8 etha=T_{off}/128 Percentage of errors 1 1 0.1 0.1 0 0 200 200 400 400 600 600 800 800 1000 1000 1200 1200 1400 1400 1600 1600 1800 1800 2000 2000 T_{off} T_{off} Comparison with threshold based algorithms – exponential case • Threshold based algorithms work well if traffic characteristics are known • But they are very sensitive to the threshold value • If sessions are already well clustered because idle periods are large enough compared to flow’s inter arrivals, our algorithm is very good

14. clustering etha=T_{off}/16 etha=T_{off}/32 etha=T_{off}/2 etha=T_{off}/64 etha=T_{off}/4 10 10 etha=T_{off}/128 etha=T_{off}/8 Percentage of errors 1 1 0.1 0.1 0 0 200 200 400 400 600 600 800 800 1000 1000 1200 1200 1400 1400 1600 1600 1800 1800 2000 2000 T_{off} T_{off} Comparison with threshold based algorithms – Pareto case • Threshold based algorithms work well if traffic characteristics are known • But they are very sensitive to the threshold value • If sessions are already well clustered because idle periods are large enough compared to flow’s inter arrivals, our algorithm is very good

15. 0.06 0.3 First SYN -> Last TCP Tear-Down First SYN -> Last Data Segment 0.05 0.25 1 1 0.1 0.1 Compl. CDF 0.04 0.2 Compl. CDF 0.01 0.01 0.001 PDF PDF 0.03 0.15 0.001 0.0001 0.0001 1e-005 0.1 0.02 100 1000 10000 100 1000 10000 Session Length [s] Number of TCP connections per session 0.01 0.05 0 0 1 10 100 1 10 100 1000 10000 Number of TCP connections per session Session Length [s] Some statistics on aggregated sessions • The session sizes are heavy tailed (broadly) • Usually each session is made of a few TCP flows • Flow termination definition is not that important

16. 0.03 Server -> Client Client -> Server 0.025 1 0.1 0.02 Compl. CDF 0.01 0.001 PDF 0.015 0.0001 1e-005 0.01 10000 100000 1e+006 1e+007 Session data [bytes] 0.005 0 100 1000 10000 100000 1e+006 Session data [bytes] Some statistics on aggregated sessions • Similar results concerning server to client and client to server data • Similar distribution law, asymetries on volume only

17. 1 Apr.04 T_{off} 0.9 Oct.02 T_{off} Apr.04 T_{arr} 0.8 Oct.02 T_{arr} 0.7 0.6 CDF 0.5 0.4 0.3 0.2 0.1 0 0.1 1 10 100 1000 10000 Time [s] Flow’s and session’s inter-arrivals • The method infers session which are similar even when considering very different traces • Tarr and Toff are well identified

18. Conclusions • Clustering techniques could be easily used to infer web-session • The proposed algorithm is a mix a known clustering approaches • It is able to deal with huge amount of data • Sessions seems to be very well recognized