180 likes | 357 Views
Exploiting Clustering Techniques for Web Session Inference. A.Bianco, G. Mardente, M. Mellia, M.Munafò, L. Muscariello (Politecnico di Torino). Outline. Web Session Model Clustering techniques The proposed algorithm Performance of the algorithm Session statistics. Web session definition.
E N D
Exploiting Clustering Techniquesfor Web Session Inference A.Bianco, G. Mardente, M. Mellia, M.Munafò, L. Muscariello (Politecnico di Torino)
Outline • Web Session Model • Clustering techniques • The proposed algorithm • Performance of the algorithm • Session statistics
Web session definition • A single web client generates a succession of TCP flows and think times think time Toff think time Toff • A session here is defined as the set of TCP flows arriving close enough one to each other • For example a threshold can be used to discriminate between think times and inter arrivals of TCP flows
Algorithms • A threshold based approach needs a priori knowledge of the source • An adaptive algorithm should be capable to catch traffic variations • This is supposed to be less sensitive to traffic characteristics • Clustering is the chosen approach
Proposed algorithm • Three steps • A K-means is used on all samples to obtain a first clustering, K is chosen very large • A hierarchical clustering is used only on representatives of each cluster, K is reduced • A K-means is used on all samples again • To test the algorithm we need a priori known traffic, that is artificially generated
First Step: K-means • K is chosen large enough but significantly smaller than the number of samples • The K farthest flows determine the first partition • K-means is performed 1000 iterations on all samples • Each cluster is then represented using a subset of samples, one or two in our algorithm • The mean value (Centroid method) • The gth and (100-g)th percentiles (Single linkage method if g=0) g-th percentile (100-g)-th percentile
Second step: a hierarchical method • A hierarchical method is used on only representatives • This method merges clusters until a quality function determines that the optimal number of clusters Nc has been found
70 60 50 40 gamma 30 20 10 0 -10 0 200 400 600 800 1000 1200 1400 Step Gamma function typical behaviour
Third Step: K-means • A K-means isperformed on all samples • This last step is not critical but rearranges samples’ positions within clusters that is flows within sessions • It is not CPU time consuming, than it is not critical to use it
Performance evaluation • Artificial traffic is generated according to an ON/OFF process • During ON periods a succession of flows is generated using i.i.d. inter-arrivals • In this model inferring is to recognize if an inter arrival is an OFF period or an inter arrival between flows within an ON period • Every time the algorithm does not guess correctly, an error is counted • Suppose all variables are exponentially distributed
K=1000 K=1500 10 K=2000 K=2500 1 Percentage of errors 0.1 0.01 0 200 400 600 800 1000 1200 1400 1600 1800 2000 T_{off} First step sensitivity (1/2) • If the initial number of clusters is chosen large enough the method is less error prone • The algorithm is much more sensitive to the value of the idle period
Single linkage g=15 Centroid Method g=25 10 10 g=1 g=35 g=5 g=45 1 1 Percentage of errors 0.1 0.1 0.01 0.01 0 0 200 200 400 400 600 600 800 800 1000 1000 1200 1200 1400 1400 1600 1600 1800 1800 2000 2000 T_{off} T_{off} First step sensitivity (2/2) • Performance is sensitive to the choice of the percentile g • When clusters are represented through flows at the border ofthe session the methodis less sensitive totraffic, i.e. g=1 • This is due to the fact that cluster has a long and narrow shape and those representatives well model this fact
clustering etha=T_{off}/16 etha=T_{off}/2 etha=T_{off}/32 etha=T_{off}/4 etha=T_{off}/64 10 10 etha=T_{off}/8 etha=T_{off}/128 Percentage of errors 1 1 0.1 0.1 0 0 200 200 400 400 600 600 800 800 1000 1000 1200 1200 1400 1400 1600 1600 1800 1800 2000 2000 T_{off} T_{off} Comparison with threshold based algorithms – exponential case • Threshold based algorithms work well if traffic characteristics are known • But they are very sensitive to the threshold value • If sessions are already well clustered because idle periods are large enough compared to flow’s inter arrivals, our algorithm is very good
clustering etha=T_{off}/16 etha=T_{off}/32 etha=T_{off}/2 etha=T_{off}/64 etha=T_{off}/4 10 10 etha=T_{off}/128 etha=T_{off}/8 Percentage of errors 1 1 0.1 0.1 0 0 200 200 400 400 600 600 800 800 1000 1000 1200 1200 1400 1400 1600 1600 1800 1800 2000 2000 T_{off} T_{off} Comparison with threshold based algorithms – Pareto case • Threshold based algorithms work well if traffic characteristics are known • But they are very sensitive to the threshold value • If sessions are already well clustered because idle periods are large enough compared to flow’s inter arrivals, our algorithm is very good
0.06 0.3 First SYN -> Last TCP Tear-Down First SYN -> Last Data Segment 0.05 0.25 1 1 0.1 0.1 Compl. CDF 0.04 0.2 Compl. CDF 0.01 0.01 0.001 PDF PDF 0.03 0.15 0.001 0.0001 0.0001 1e-005 0.1 0.02 100 1000 10000 100 1000 10000 Session Length [s] Number of TCP connections per session 0.01 0.05 0 0 1 10 100 1 10 100 1000 10000 Number of TCP connections per session Session Length [s] Some statistics on aggregated sessions • The session sizes are heavy tailed (broadly) • Usually each session is made of a few TCP flows • Flow termination definition is not that important
0.03 Server -> Client Client -> Server 0.025 1 0.1 0.02 Compl. CDF 0.01 0.001 PDF 0.015 0.0001 1e-005 0.01 10000 100000 1e+006 1e+007 Session data [bytes] 0.005 0 100 1000 10000 100000 1e+006 Session data [bytes] Some statistics on aggregated sessions • Similar results concerning server to client and client to server data • Similar distribution law, asymetries on volume only
1 Apr.04 T_{off} 0.9 Oct.02 T_{off} Apr.04 T_{arr} 0.8 Oct.02 T_{arr} 0.7 0.6 CDF 0.5 0.4 0.3 0.2 0.1 0 0.1 1 10 100 1000 10000 Time [s] Flow’s and session’s inter-arrivals • The method infers session which are similar even when considering very different traces • Tarr and Toff are well identified
Conclusions • Clustering techniques could be easily used to infer web-session • The proposed algorithm is a mix a known clustering approaches • It is able to deal with huge amount of data • Sessions seems to be very well recognized