Statistical Modeling for Per-Hop QoS

1. Statistical Modeling for Per-Hop QoS Mohamed El-Gendy (mgendy@eecs.umich.edu) In collaboration with Abhijit Bose, Haining Wang, and Prof. Kang G. Shin Real-Time Computing Laboratory EECS Department The University of Michigan@Ann Arbor June 4th, 2003

2. Outline • Intro of DiffServ, PHB, and Per-Hop QoS • Motivations • Related Work • Approach to Statistical Characterization • Experimental Framework • Results and Analysis • A Control Example • Conclusions and Future Work

3. DiffServ and PHB • Scalable network-level QoS based on marked “traffic aggregates” • Traffic is conditioned and marked with DSCPat edge • Per-Hop Behaviors (PHBs) are applied to traffic aggregates at core • QoS is achieved through different PHBs: • Expedited Forwarding (EF) for delay assurance • Assured Forwarding (AF) for bandwidth assurance

4. DiffServ node Input traffic c/c’s I Output traffic c/c’s O (BW, D, J, L) PHB Configuration parameters C Per-Hop QoS • Throughput (BW), delay (D), jitter (J), and loss (L) experienced by traffic crossing a PHB

5. Motivations • Why modeling Per-Hop QoS? • PHB is the key building block of DiffServ • Wide variety of PHB realizations • PHB control and configuration • Necessary for end-to-end QoS calculation • Benefits of PHB modeling: • Facilitates the control and optimization of PHB performance • Enables contribution of per-hop admission control to e2e admission control decisions

6. Related Work • Study of TCP ACK marking in DiffServ [IWQoS’01] : • Used full factorial design and ANOVA • Compared many marking schemes for TCP acks • Suggested an optimal strategy for marking the acks for both assured and premium flows • Used ns simulation for analysis • AF performance using ANOVA [IETF draft]: • Compared different bandwidth and buffer management schemes for their effect on AF performance

7. Related Work • Performance of TCP Vegas [Infocom’00] : • Used ANOVA to test the effect of ten congestion and flow control algorithms • Clustered the ten factors into three groups according to the the three phases of the TCP Vegas operation

8. Approach to Statistical Characterization • Identify the factors in I and C that affect output per-hop QoS most • Construct statistical models of the per-hop QoS in terms of these important factors

9. Input Traffic Factors - I Dual Leaky Bucket (DLB) representation for I: • Average rate, peak rate, burst size, packet size, number of flows per aggregate, and traffic type • Ia : assured traffic, Ib: background traffic • Used the ratio between assured to best-effort traffic, instead of absolute value • Number of input interfaces to PHB node

10. Alternative PHB Realizations • Different PHB realizations have different functional relationships between inputs and outputs EF-EDGE EF-CORE EF-CBQ

11. Configuration Parameters- C Configuration parameters depend on PHB realization: • EF-EDGE: token rate, bucket size, and MTU • EF-CORE: queue length • EF-CBQ: service rate, burst size, and avg packet size • AF: min. threshold, max. threshold, and drop probability

12. Statistical Analysis Analysis of Variance (ANOVA) • Models Output response as a linear combination of the main effects and their interactions • Allocation of variation Calculate the percentage of variation in the output response due to factors at each level, their interactions, and the errors in the experiments • ANOVA Statistically compare the significance of each factor as well as the experimental error

13. ANOVA • For any three factors (k = 3), A, B, and C with levels a, b, and c, and with r repetitions, the response variable y can be written as:

14. ANOVA • Squaring both sides we get: SSE: sum of squared errors

15. ANOVA

16. ANOVA Model Assumptions • Assumptions: • Effects of input factors and errors are additive • Errors are identical, independent, and normally distributed random variables • Errors have a constant standard deviation • Visual tests: • No trend in the scatter plot of residuals vs. predicted response • Linear normal quantile-quantile (Q-Q) plot of residuals No assumptions about the nature of the statistical relationship between input factors and response variables

17. Statistical Analysis - Regression Polynomial regression • A variant of multiple linear regression • Any complex function can be expanded into piecewise polynomials with enough number of terms • Transformations to deal with nonlinear dependency • Coefficient of determination (R2) as a measure of the regression goodness

18. Regression Linear model for one dependent variable y and k independent variables x : Polynomial model for two independent variables x1, x2: Transformation to fit into linear model:

19. Experimental Framework Framework components: • Traffic Generation Agent • Generates both TCP and UDP traffic • Policed with a built-in leaky bucket for profiled traffic • BW, D, J, L are measured within the agent itself • Controller and Remote Agents • Control the flow of the experiments according to a distributed scenario file • Executes and keep track of the experiments steps and other components

20. Experimental Framework, cont’d • Network and Router Configuration Agents • Configure traffic control blocks on router according to experiment scenarios • Receive scenario commands from the Controller agent • Current implementation works on Linux traffic control • Analysis Module • Performs ANOVA, model validation tests, and polynomial regression on output data

21. Experimental Framework, cont’d Network Setup • Using ring topology for one-way delay measurements

22. Full Factorial Design of Experiments • If we have k factors, with ni levels for the i-th factor, and repeat r times Total number of experiments = LARGE!! • Use factor clustering and automated experimental framework

23. Scenarios of Experiments EF PHB • Factor sets: Ia, Ib, C • PHB configurations: EF-EDGE, EF-CORE, EF-CBQ • Operating mode: over-provisioned (OP), under-provisioned (UP), fully-provisioned (FP)

24. Scenarios of Experiments, cont’d AF PHB • Use AF11 as assured traffic • Use AF12 and AF13 as background traffic • Change max. threshold, min. threshold, and drop probability for AF11 only

25. EF PHB – OP, EF-EDGE w/o BG traffic BW surface response: significant factors are assured rate ( ar ) and number of assured flows ( an ), R2 = 96%

26. EF PHB – OP, EF-EDGE w/o BG traffic J model and visual tests Significant factors are: assured rate ( ar ), number of assured flows ( an ), and assured packet size (apkt)

27. EF PHB – UP, EF-EDGE w/o BG traffic • ANOVA results for L Significant factors are: assured rate ( ar ), number of assured flows ( an ), and the token bucket rate ( efr ) • Regression model for L

28. EF PHB – OP, EF-EDGE w/ BG traffic • ANOVA results for BW, D, and J Significant factors are: BG packet size ( bpkt ), number of BG flows ( bn ), and ratio of assured to BG traffic ( Rab )

29. EF PHB – EF-CORE w/o BG traffic • J surface response: Significant factors are assured packet size ( apkt ) and number of assured flows ( an ), R2 = 64%

30. EF PHB – EF-CORE w/o BG traffic • J visual tests

31. EF PHB – OP, EF-CBQ w/o BG traffic • ANOVA results for BW, D, and J Significant factors are assured rate ( ar ), assured packet size ( apkt ) and number of assured flows ( an )

32. EF PHB – OP, EF-CBQ w/ BG traffic • ANOVA results for L Significant factors are BG packet size ( bpkt ), number of BG flows ( bn ), and ratio of assured to BG traffic ( Rab )

33. EF PHB – OP, EF-CBQ w/ BG traffic • D regression model R2 = 89% • D approximate model

34. AF PHB • ANOVA results for BW, D, J, and L Significant factors are: assured rate ( ar ), assured peak rate ( ap ), assured packet size ( apkt ), max. threshold ( maxth ) , and min. threshold ( minth )

35. Discussion • BW shows a square root relationship with factors in Ia in EF-CBQ only, and direct relation in the other EF realizations • D shows a direct relation with Ia in EF-EDGE, and EF-CORE, and inverse relation in EF-CBQ • D shows a logarithmic (multiplicative) relation with Ib • J shows inverse relation with Ia and a direct relation with Ib • J depends on the number of flows in the aggregate as well as the difference in packet size with other flows/aggregates

36. Errors • Experimental errors: due to experimental methods; captured in ANOVA • Model errors: due to factor truncation • Statistical and fitting errors: due to regression; captured in coefficient of determination (R2 )

37. A PHB Control Example • For OP, EF-CBQ w/ BG traffic: • For bpkt = 600 B, bn = 1, Rab = 2  D = 0.4136 msec • For bpkt = 1470 B, bn = 3, D = 0.4136 msec  Rab = ?? • Use the delay model to find Rab = 0.494 with accuracy of (1-R2) = 11%

38. Conclusions • Simple statistical models are derived for per-hop QoS using ANOVA and polynomial regression • Statistical full factorial design of experiments is an effective tool for characterizing QoS systems • Using automated experimental framework is shown to be effective in such studies • Different PHB realizations show differences in dependency of per-hop QoS on input factors

39. Extensions and Future Work • More rigorous control analysis and study of suitable control algorithms • The framework presented is general to be applied for studying edge-to-edge (Per-Domain Behavior or PDB) in DiffServ • Validate the models derived with analytical methods such as network calculus • Use real-time measurements to update models and control criterion while operation

40. Multi-Hop Case • First approach

41. Multi-Hop Case • Second approach

42. Questions ?