1 / 16

Performance Modeling in the Large: A Case Study

Performance Modeling in the Large: A Case Study. B.D. Theelen. Contents. Introduction Parallel Object-Oriented Specification Language (POOSL) Case Study: Internet Router Modeling the Internet Router Exhaustive Performance Analysis Performance Evaluation by Simulation Accuracy Analysis

jackson-daugherty
Download Presentation

Performance Modeling in the Large: A Case Study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Performance Modeling in the Large:A Case Study B.D. Theelen www.ics.ele.tue.nl/~btheelen

  2. Contents • Introduction • Parallel Object-Oriented Specification Language (POOSL) • Case Study: Internet Router • Modeling the Internet Router • Exhaustive Performance Analysis • Performance Evaluation by Simulation • Accuracy Analysis • Conclusions www.ics.ele.tue.nl/~btheelen

  3. System-Level Design Methods • Focus on developing executable models, which • Describe a system in the earliest phases of the design process • Abstract from many implementation details • Are relatively cheap to create, validate, test, debug and simulate • Allow the analysis of system properties before the system is actually being realized in terms of hardware and software • Correctness properties: absence of deadlock, … • Performance properties: throughput, packet-loss, jitter, … • Enable to take well-founded design decisions based on the obtained system property analysis results • Require the use of a well-defined modeling language Parallel Object-Oriented Specification Language (POOSL) www.ics.ele.tue.nl/~btheelen

  4. POOSL • System-level modeling language for complex real-time distributed hardware/software systems • Small set of very expressive primitives • Formal (mathematically defined) semantics • Probabilistic features for performance analysis • Process part based on real-time extension of CCS • Process objects model the elementary time-related behavior • Channels enable synchronous message passing between process objects • Clusters allow for describing hierarchy • Data part based on imperative object-oriented languages • Data objects model data structures and operations on them www.ics.ele.tue.nl/~lvbokhov/poosl www.ics.ele.tue.nl/~btheelen

  5. Alcatel 7770 RCP (Routing Core Platform) Internet Router 640 Gb/s – 10 Tb/s OC-3 – OC-192 8.5 kW $ 290.000 www.ics.ele.tue.nl/~btheelen

  6. Overview of the Internet Router Sources Input Buffers Switch Core Output Buffers Sinks 1 1 1 1 1 1 N M Outputs Inputs 1 N N N 1 N N M Is the specified flow-control mechanismsuitable for future product variants? www.ics.ele.tue.nl/~btheelen

  7. System Parameters N and M • Sources NxM Source Activities • Input Buffers N Input Dispatchers NxN Queue Output Handlers • Switch Core NxN Switch Activities • Output Buffers N Queue Input Handlers N Queue Output Handlers • Sinks N Sink Activities 2N2 + (M+ 4) N Activities Huge number of concurrent activities www.ics.ele.tue.nl/~btheelen

  8. Scalability of the Model • Instantiating a process object is performed by drawing the object • Modeling each component with a separate process object is not favorable for large N and M • Initialize parameterized number of similar concurrent activities  Initialize()() InitializeActivity(1)().  InitializeActivity(Identity: Integer)() par Activity(Identity)() /* Invokes the actual Activity */ and if (Identity < N) then InitializeActivity(Identity + 1)() fi rap. (Optimization: dynamic creation and termination of concurrent activities) www.ics.ele.tue.nl/~btheelen

  9. Executable Model • Create executable model with SHESim tool • Freely available on: www.ics.ele.tue.nl/~mgeilen/shesim www.ics.ele.tue.nl/~btheelen

  10. Validation • Is the model adequate for analyzing the performance metrics? • Validate, test and debug model using SHESim tool www.ics.ele.tue.nl/~btheelen

  11. Formalization of Performance Properties • The flow-control mechanism should contribute to • Minimizing packet-loss • Minimizing buffer capacity • Minimizing latency • Maximizing throughput • Performance metrics to analyze • Probability of losing packets • Average and maximum buffer occupancy • Average and maximum latency • Throughput Use POOSL formalism to extend model for performance analysis www.ics.ele.tue.nl/~btheelen

  12. UnderstandablePOOSL Model&Extensions for Performance Analysis Formal Semantics 1 Out!Packet(p) Markov Decision Process&Reward Structure 1  0.9 3 S1 S2 S3 S4 0.1 External Scheduler Discrete-Time Markov Chain & Reward Structure Performance Evaluation with POOSL Markov Chain is implicitly defined by expressive modeling language and not by hand www.ics.ele.tue.nl/~btheelen

  13. Performance Evaluation with POOSL • Analytically: Ergodic Theorem for Markov Chains • Example: time-average buffer occupation • Simulation: Central Limit Theorem for Markov Chains • Dependable random variables • Example: packet-loss probability • Enables accuracy analysis with confidence intervals www.ics.ele.tue.nl/~btheelen

  14. Performance Evaluation by Simulation • Fast simulation with Rotalumis tool • Freely available on: www.ics.ele.tue.nl/~lvbokhov/poosl/rotalumis • Simulation = generation of one trace through Markov chain • How long should a simulation take to obtain accurate results? • Analyze accuracy of simulation results with confidence intervals • Automatic termination of simulation www.ics.ele.tue.nl/~btheelen

  15. Accuracy Analysis • For N=512 and M=128, a 99% confidence interval[0.001046, 0.001052] is obtained for the average latency • Estimation of average latency is 0.001049 seconds • We are 99% sure that the real average latency lies in [0.001046, 0.001052] • An upper bound for the relative error is 1% • Takes 0.0072 seconds of simulated real-time • Takes 11.9 hours of simulation time • POOSL classes for accuracy analysis of long-run averages • Long-run sample average: packet-loss probability, latency • Long-run sample variance: jitter • Long-run time average: buffer occupancy, processor utilization • Long-run time variance: buffer occupancy Rare events (losing a packet) may change the status of a metric www.ics.ele.tue.nl/~btheelen

  16. Conclusions • POOSL allows for a compact abstract description of real-life systems to adequately investigate their performance properties • Performance analysis requires extending the original model • POOSL model implicitly defines a Markov chain • Formal semantics enables computation of performance • POOSL tools offer good validation and simulation possibilities • Confidence and accuracy of simulation results should be known • POOSL library classes for accuracy analysis • Future research • Long-run rate average (throughput) and rate variance (burstiness) • Rare events • Support for performance metric monitors www.ics.ele.tue.nl/~btheelen

More Related