Performance Evaluation: 30 years in review

1. Performance Evaluation: 30 years in review Martin Reiser GMD – German National Research Center for Information Technology reiser@gmd.de MMB’99, 22.–24.September 1999, Trier Presentation 23. September, 9:00

2. Host Switch Link Memory ALU Channels Very complex systems • where are the limits? • will it work? • if so, how well? • can it work better? • what will I need tomorrow, in a month, year …? but why bother anyway?

3. 80! ? … m trunk lines many subscribers A. K. Erlang, The Post Office Electrical Engineers's Journal 1918 30 years?

4. Cutting the Gordian knot Message Independence Central Server Kleinrock, 1964 Gaver, 1967; Buzen, 1971

5. ( ) · E p i , r ; j , s l r ( ) · G N Jobs s Big bang: BCMP open class r class change i i G(•) closed class s FCFS LCFS/PR j j PS IS unlimited waiting room queue-dependent rates Baskett, Chandy, Muntz and Palacios, 1975

6. The PF solution is robust station i, class r: Routing details don’t matter, only mean number of visits does! Distribution of service-times does not matter, only mean values enter into the solution! Õ Õ n i , r = P ( n ) C ( n ) W i i , r i r long live the Central Server

7. The classical models Delay analysis of data networks (Kleinrock 64) Sharing the CPU, i.e. PS (Kleinrock et. al. 64 +) Time-sharing model (Scherr 67) Central Server (Buzen 71) Polling, token-ring (Konheim et. al. 74 +) Aloha, CSMA …(Kleinrock, Lam, Tobagi 73 +)

8. to meshed networks From trees 3274 370 3705 3274 3705

9. to MVA = t + - ( ) T ( W ) 1 n ( W 1 ) i i i W l = å T ( W ) i = l n ( W ) T ( W ) i i From window flow-control

10. MPX to MAP a l 2 l 1 … silence b talk spurt From Poisson arrivals call

11. Methods

12. Methods

13. Problems

14. Problems

15. PE specialists … work on PE problems cached in PE terms … build tools … develop (and solve) models of real systems … are integral parts of development teams IS professionals … use PE methodology in their daily work (planning, managing, tuning) really?

16. Success stories PE-based engineering and development Capacity planning Network management Software performance engineering Failures ?

17. PE-basierte F+E • Skilled PE specialist group • is integrated in R&D team • is of sufficient size (5 – 10%) • is highly respected • and contributes to design! All important design decisions are based on PE results and analysis Early prototype support PE predictions Case study: IBM Token Ring

18. VLSI Design Generally accepted specification process Simulation data from specification • on several levels of detail Automatic mask generation Generation of test patterns integrated process environment

19. PE in standards Early attempts in SDLC/HDLC Success story LAN standard IEEE 802 Success story ATM fix before build

20. What is PE? …………………………………………………yes Is it a science ……………………………some Is it used by engineers? ………………little Is it used in IS planning/tuning? …………cottage industry Is it a business? Is it regular part of ……not really university curriculum? Quo vadis ?

21. Reach out to main stream? Must make quantum leap in SPE Must integrate tool into develop-ment processes • robust • Sprache des Benutzers Stay where we are? • accepted in EE, especially communications • continue existence as niche science

22. SPE roadmap Path-length Usage CASE me- Rep. estimation scenario thodology process editor System Capacity structure planning editor Workload data Analytic system structure Network capacity model planning In user’s language

23. 3 grand challenges Understand self-similar traffic • do we need a new theory? • se we merely shifting loads? Introduce analytical methods in computer architecture (“a gold mine” S. Adve) to enter into the main stream of the CASE process and tools