1 / 21

Software Architecture based Performance and Reliability Evaluation

Software Architecture based Performance and Reliability Evaluation Vibhu S. Sharma Ph.D. Scholar CSE, IITk Client 1 Client 2 Web-server Application Server DB Client nc A web-based software system Software Architecture

Antony
Download Presentation

Software Architecture based Performance and Reliability Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Software Architecture based Performance and Reliability Evaluation Vibhu S. Sharma Ph.D. Scholar CSE, IITk

  2. Client 1 Client 2 Web-server Application Server DB Client nc A web-based software system Software Architecture • Software architecture is the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution. - IEEE 1471-2000. • Software architecture styles are like templates which are followed by commonly used systems: • Layered style : Web-based systems • Pipe and Filter style : Compilers • . . . IIT Kanpur

  3. 1-p12 1 p12 2 p23 3 1 C 1-p23 ESA Antenna array configuration tool architecture S0 p12 p23 p(nl-2)(nl-1) p(nl-1)nl 1 S1 S2 . . . . . . . . . . Snl-1 Snl 1-p12 1-p23 1 1- p(nl-1)nl Snl+1 Snl+2 . . . . . . . . . . S2nl-1 S2nl 1 1 1 1 1 S2nl+1 1 Modeling the workflow of the Software Architecture • We use finite absorbing Discrete Time Markov Chains (DTMC) for representing the software architecture workflow. • Examples: • These DTMCs can be used to find the average visit counts to various components for a typical request as well as the limiting probabilities. • Very useful for reliability as well as performance prediction. A typical layered architecture IIT Kanpur

  4. 1 pf1 (1-pf1)p12 3 pf3 2 pf2 F 4 pf4 5 pf5 S Architecture based Reliability prediction • Issues/Questions: • What is the reliability of a software system composed of unreliable components? • How does increasing the reliability of a particular component affects overall reliability ? • How does changing the architecture affect reliability ? • Reliability modeling using DTMCs • Model the component failure as a state in the DTMC and associate probabilities of transition (unreliabilities). • Overall reliability is the probability of ending in the correct state. • Another option is using a reward based method IIT Kanpur

  5. S0 p12 p23 p(nl-2)(nl-1) p(nl-1)nl 1 S1 S2 . . . . . . . . . . Snl-1 Snl 1-p12 1-p23 1 1- p(nl-1)nl Snl+1 Snl+2 . . . . . . . . . . S2nl-1 S2nl 1 1 1 1 1 S2nl+1 1 Architecture based performance prediction • Issues/Questions: • What is the maximum number of clients the system can able to handle before it saturates ? • What effect does varying the number of clients have on the throughput and the average response time ? • How should software components be allocated to the hardware nodes ? • What changes should be made in the system to improve performance ? • We take an example of layered software architecture. • A Discrete Time Markov Chain (DTMC) is used to characterize the control flow in the layered software architecture. • The DTMC is used to calculate the visit counts to various layers and then a closed queueing network (QN) model is constructed and solved. IIT Kanpur

  6. Some Analysis Results • Average response time and throughput and bottleneck analysis • Effect of change in individual reliabilities & workload on overall reliability IIT Kanpur

  7. Reliability and Performance Tradeoff • Given a set of components and a set of machines to deploy these onto • What are the performance & reliability characteristics of the setup under a given load ? • Which architecture should be chosen for optimal performance and reliability ? • What is the tradeoff ? • Software failures of different types cause different effects on the performance of the system. • Hardware availability also needs to be taken into account. • Some configurations will result in high throughput with decreased reliability and vice-versa. IIT Kanpur

  8. Combining Static Analysis and Testing Frameworks to find Software Defects Vipindeep V

  9. Write Code Testing Compile/Build Customer Evaluate Defect Prevention during Software Development Static analysis Test Suite Program Specification IIT Kanpur

  10. Testing + Static Analysis – How? • Static Analysis Vs Testing • Different areas altogether, but motive – identify defects • Combining the approaches… improved program checking • Motivations for the Proposed Approach • Static analysis concerns • Testing is done.. Bugs are revealed • Ideal test suite => identify all bugs • Non-ideal scenario of Testing • Identify defects, partially • Some paths/blocks are “not (properly) tested” .. Why?? • Static analysis effort on unexplored parts of the code • Our proposed approach • Focused Static Analysis with Path Pruning using Coverage Data IIT Kanpur

  11. Overview of the approach IIT Kanpur

  12. Pros and Cons • Advantages: • New real errors can be identified which escaped in _all_paths analysis • Rigorous testing effort counted for better static analysis • Reduce execution time • Reduced warnings => Reduced Noise • Disadvantages: • Some errors escape in the pruned path analysis • Depends on testing framework IIT Kanpur

  13. Preliminary Results • Applied the approach with FindBugs, Jlint for JAVA • Results from testing were encouraging • Tested the applicability using Random Path Selection IIT Kanpur

  14. Conclusions • Works well in practice • Finds enough real defects to be useful • Noise is low enough that people can use it • Scales well, so works on large very large code bases • More issues to explore.. • Better algorithm to use testing data • Path profiles, Crash dumps – focused static analysis • Use static analysis results for focusing the testing effort • Combining static and dynamic analysis • Usability related issues IIT Kanpur

  15. State Based Testing Automation Atul Gupta Ph.D. Scholar

  16. State Based Testing • Program behavior can be modeled as a state graph • A formal testing approach at all levels-unit, component or system level. • Effective testing strategy to O-O software • Test Cases and Test Oracles can be automatically generated and evaluated Atul Gupta PhD Scholar IIT Kanpur

  17. vend( ) allowVend = 0 curQtrs = 1 allowVend = 0 curQtrs = 0 addQtr( ) addQtr( ) addQtr( ) WAIT Q1 Q2 retQtrs( ) allowVend = 1 curQtrs >= 2 retQtrs( ) An Example : A CoinBox Class • Class CoinBox { • unsigned totalQtrs; • unsigned curQtrs; • unsigned allowVend; • public: • CoinBox( ) { • totalQtrs = 0; • curQtrs = 0; • allowVend = 0; • } • void retQtrs( ) { • curQtrs = 0; • } • void addQtr( ) { • curQtrs = curQtrs +1; • if (curQtrs > 1) • allowVend = 1; • } • void vend( ) { • if (allowVend) { • totalQtrs = totalQtrs + curQtrs; • curQtrs = 0; • allowVend = 0; • } • } • }; IIT Kanpur

  18. All-Transition Coverage (AT) All-Transition-pair Coverage (ATP) Full Predicate coverage (FP) Transition-Tree Coverage (TT) All-Round-Trip path coverage (ART) Complete Sequence (CS) vend( ) allowVend = 0 curQtrs = 1 allowVend = 0 curQtrs = 0 addQtr( ) addQtr( ) addQtr( ) WAIT Q1 Q2 retQtrs( ) allowVend = 1 curQtrs >= 2 retQtrs( ) State Based Testing Criteria IIT Kanpur

  19. vend( ) allowVend = 0 curQtrs = 1 allowVend = 0 curQtrs = 0 addQtr( ) addQtr( ) addQtr( ) WAIT WAIT WAIT WAIT WAIT Q1 Q1 Q2 Q2 Q2 addQtr( ) retQtrs( ) allowVend = 1 curQtrs >= 2 retQtrs( ) addQtr( ) retQtrs( ) addQtr( ) retQtrs( ) vend( ) Generating Test Sequences IIT Kanpur

  20. Research Issues • How to generate test cases automatically from a state model • How to execute them in an automated manner • How to evaluate the results automatically IIT Kanpur

  21. Our Approach • Identify State Model of a Class • Generate test sequences based on some Coverage Criteria • Convert these sequences into JUnit format for Automatic Execution and Result evaluation AIM : Complete end-to-end Automation IIT Kanpur

More Related