1 / 25

Continuous Performance Testing in Virtual Time

Nikos Baltas & Tony Field Department of Computing Imperial College London. Continuous Performance Testing in Virtual Time. Motivation. Performance analysis: Early stages: performance modelling Late stages: profiling & performance testing

keiji
Download Presentation

Continuous Performance Testing in Virtual Time

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nikos Baltas & Tony Field Department of Computing Imperial College London ContinuousPerformance Testing in Virtual Time Imperial College - Department of Computing

  2. Motivation • Performance analysis: • Early stages: performance modelling • Late stages: profiling & performance testing • Intermediate stages: extract models from code, calibrate them and analyse... • Can we simplify this? • Can we performance test incomplete applications? Imperial College - Department of Computing

  3. Imperial College - Department of Computing Idea • Integrate models and code • Don't model existing code – execute it in virtual time! • Use performance models that describe the expected response times of missing (or existing) methods • Combine measurements and model simulations to guide the thread progress of the application

  4. Imperial College - Department of Computing Virtual Time Execution

  5. VEX • Virtual Execution framework (VEX) • Java applications • Components • Thread control • Profiling • I/O handling Imperial College - Department of Computing

  6. Imperial College - Department of Computing VEX simulation Model existing code by profiling it on the simulation host Map the (possibly scaled) measurements on a virtual timeline Control the thread progress to enforce a fair scheduling policy in virtual time

  7. Imperial College - Department of Computing Integration of models to VEX Don't just scale method durations: replace them Use time acquired by the simulation of performance models (open queuing networks) Decouple profiled time from virtual time measurements Use for either implemented methods or code stubs Continuous performance testing

  8. Imperial College - Department of Computing Example

  9. Imperial College - Department of Computing Issues How does the model-simulation time relate to the virtual time of executed code? How do we control the model simulation? What if the code of the model-described method interacts with the application?

  10. Imperial College - Department of Computing Remote and local resources

  11. Imperial College - Department of Computing Model simulation in VEX Thread adds job Thread state depends on job state in queue: L-Think: Running/able R-Think: Timed-waiting Queuing: Waiting L-Serv: Running/able R-Serv: Timed-waiting Thread and job join at a Sink node

  12. Imperial College - Department of Computing Model description: JMT [1] [1] M. Bertoli, G. Casale, and G. Serazzi, “JMT: performance engineering tools for system modeling,” SIGMETRICS Perform.Eval.Rev., vol. 36, no. 4, pp. 10–15, 2009.

  13. Imperial College - Department of Computing Functional consistency Performancemodel for M() Join with model resume T1 blocked on obj monitor synchronized(obj) { enter/exit monitor M() { } M() Deadlock! violate virtual scheduleand resume anyway wait for T1 to resume T2 resume synchronized(obj) { suspend } exit monitor obj suspend

  14. Imperial College - Department of Computing Usage java -Xbootclasspath/p:vex.jar –javaagent:jine.jar –agentpath:libjinevex.so Class @virtualtime.ModelPerformance( jmtModelFilename=“file.jsimg”, replaceMethodBody=true, sourceNodeLabel=“Source0”) void foo() { ….. bar(); ….. }

  15. Imperial College - Department of Computing Evaluation Case study to demonstrate usage and verify functionality of approach Sample results and parameters Validate process by comparing predicted results to real times Study: remote DB server with local cache

  16. Imperial College - Department of Computing Pure model

  17. Imperial College - Department of Computing Partially complete code

  18. Complete code Imperial College - Department of Computing

  19. Results Imperial College - Department of Computing

  20. Imperial College - Department of Computing Conclusion Integration of models and code within VEX Models and code can be used interchangeably Continuous performance testing without extracting models from code Prediction errors due to disregard for background system load and I/O approximation

  21. Imperial College - Department of Computing Future work Handle more behaviours (asynchronous I/O) Lower-level VEX and/or distributed virtual execution Application on larger scale projects Release VEX under open source license Extensions to other languages (C++)

  22. Imperial College - Department of Computing Questions? Thank you

  23. Imperial College - Department of Computing Virtual Time Execution II

  24. Imperial College - Department of Computing Remote and local resources II

  25. Imperial College - Department of Computing Local resources visualisation

More Related