Download
slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
NASA OSMA SAS '02 PowerPoint Presentation
Download Presentation
NASA OSMA SAS '02

NASA OSMA SAS '02

321 Views Download Presentation
Download Presentation

NASA OSMA SAS '02

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. NASA OSMA SAS '02 Software Reliability Modeling:Traditional and Non-ParametricDolores R. WallaceVictor LaingSRS Information ServicesSoftware Assurance Technology Centerhttp://satc.gsfc.nasa.gov/dwallac, vlaing@pop300.gsfc.nasa.gov NASA OSMA SAS02

  2. The Problem • Critical NASA systems must execute successfully for a specified time under specified conditions -- Reliability • Most systems rely on software • Hence, a means to measure software reliability is essential to determining readiness for operation • Software reliability modeling provides one data point for reliability measurement NASA OSMA SAS02

  3. Software Reliability Modeling(SRM) – Traditional • Captures hardware reliability engineering concepts • Mathematically models behavior of a software system from failure data to predict reliability growth • Invokes curve-fitting techniques to determine values of parameters used in the models • Validates models with data with statistical analysis • Using parametric values, predicts future measurements, e.g., • Mean time to failure • Total number faults remaining • Number faults at time t NASA OSMA SAS02

  4. Synopsis • FY01 • Identify mathematics of hardware reliability not used in software • Identify differences between hardware, software affecting reliability measurement • Identify possible improvements • FY02 • Demonstrate practicality of SRM at GSFC • Fault correction improvement – Schneidewind • Non-parametric model - Laing NASA OSMA SAS02

  5. SRM: Data Collection • Resistance to data collection • Data content • Accuracy of content • Dates of failure, correction • Calendar time not execution time • Activities/ phase when failures occur • Data manipulation • Frequency counts • Interval size and length • Time between failure NASA OSMA SAS02

  6. IntervalCounter Sample had 35 weeks – simplified fault count NASA OSMA SAS02

  7. SMERFS^3 3-D OUTPUT NASA OSMA SAS02

  8. Practical Method • SATC Services • SATC executes models and prepares analysis • SATC provides training and public domain tool • Improvements • Recommendations to projects for data collection • IntervalCounter to simplify data manipulation NASA OSMA SAS02

  9. Fault Correction Adjustments • Reliability growth occurs from fault correction • Failure correction proportional to rate of failure detection • Adjusted model with delay dT (based on queuing service) but same general form as faults detected at time T • Process: use SMERFS Schneidewind model to get parameters; apply to revised model via spreadsheet • Results • Show reliability growth due to fault correction • Predict stopping rules for testing NASA OSMA SAS02

  10. SMERFS^3 – Excel Approach* • Best approach: combine SMERFS^3 with Excel. • SRT provides model parameter estimation. • Copy and paste parameters from SRT into spreadsheet. • Excel extends capabilities of SRT by allowing user to provide equations, statistical analysis, and plots. * CASRE or other software reliability modeling tool may be used with EXCEL Recommended approach until the SRM tools incorporate this new model. NASA OSMA SAS02

  11. Non-parametric Reliability Modeling • Hardware • - Wears out over time • - Increasing failure rate • Software • - Do not wear over time • - Decreasing failure rate NASA OSMA SAS02

  12. Continued • Hardware Reliability Modeling • - “Large” independent random sampling • - Model reliability • - Make predictions • Software Reliability Modeling • - “Small” observed dependent sample (of size one?) • - Not based on independent random sampling • - Model reliability • - Make predictions? • Do we search for the silver bullet of SWR models? NASA OSMA SAS02

  13. Reliability Trending • Hardware Reliability • 100%Maximum • 0%Minimum • 0 1 2 3 4 …  • Time • Software Reliability • 100% Maximum • 0% Minimum • 0 1 2 3 4 …  • Time NASA OSMA SAS02

  14. Software Reliability Bounds • 100% Maximum • Estimated Bound • Estimated Model • 0% Minimum • 0 1 2 3 4 …  • Time NASA OSMA SAS02

  15. Calculation of Estimated Models and Bounds • Dynamic Metrics • - Failure rate data • - Problem reports • Static Code Metrics • - Traditional • - Source Lines of Code (SLOC) • - Cyclomatic Complexity (CC) • - Comment Percentage (CP) • - Object-Oriented • - Coupling Between Objects (CBO) • - Depth of Inheritance Tree (DIT) • - Weighted Methods per Class (WMC) NASA OSMA SAS02

  16. Combining Dynamic and Static Metrics • The Proportional Hazards Model (PHM) • PHM Non-Parametric Component (Static) • R(t|z) = {R0(t)}g(z) • Parametric Component (Dynamic) • - Where zβ = z1β1 + z2β2 + … + zpβp , βi’s are unknown • regression coefficients and zi’s are static code metrics data NASA OSMA SAS02

  17. Tool Schema • Input Data z = (z1, z2, … zp) • DatabaseObserved Data • Data Processing R(t|z) = {R0(t)}g(z) • Weighted Average Raw Data • Output Data Estimated Model Estimated Bound • - Process Below Bounds • Action - Corrective Action • - Process Above Bounds • - No Corrective Action NASA OSMA SAS02

  18. SUMMARY • Software reliability modeling • Provides useful measurements for decisions • Does not require expert knowledge of the math! • Is relatively easy with use of software tools • Fault correction improvement • Adapts model to be more like software • Demonstrates combined use of traditional SRM tools with spreadsheet technology • Non-parametric modeling • New approach shows promise • Prototype to be expanded NASA OSMA SAS02

  19. AIAA Recommended Steps(specific to SRM) • Characterizing the environment • Determining test approach • Selecting models • Collecting data • Estimating parameters • Validating the models • Performing analysis NASA OSMA SAS02

  20. Fault Correction Modeling • Software reliability models focus on modeling and predicting failure occurrence • There has not been equal priority on modeling the fault correction process. • Fault correction modeling and prediction support to • predict whether reliability goals have been achieved • develop stopping rules for testing • formulate test strategies • rationally allocate test resources. NASA OSMA SAS02

  21. Equations: Prediction and Comparison Worksheets Time to Next Failure(s) Predicted at Time t Remaining Failures Predicted at Time t: r(t) = (/) – Xs,t Cumulative Number of Failures Detected at Time T: D(T) = (α/β)[1 – exp (-β ((T –s + 1)))] + Xs-1 Cumulative Number of Failures Detected Over Life of Software TL: D(TL) = / + Xs-1 Equations developed by Dr. Norman Schneidewind, Naval Postgraduate School, Monterey, CA NASA OSMA SAS02