html5-img
1 / 29

Software Verification and Validation Process and Tracing Software/System Requirements

Software Verification and Validation Process and Tracing Software/System Requirements . Tom Foley Altran Solutions. Invensys OpsManage11 Nashville, TN November 8, 2011. I. Software Verification and Validation Process. Introduction.

fuller
Download Presentation

Software Verification and Validation Process and Tracing Software/System Requirements

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software Verification and Validation Processand Tracing Software/System Requirements Tom Foley Altran Solutions Invensys OpsManage11 Nashville, TN November 8, 2011

  2. I. Software Verification and Validation Process

  3. Introduction • Digital I&C systems require additional design and qualification approaches above and beyond analog I&C systems. The major differentiator between the digital I&C systems and the analog I&C systems is the software.   • To obtain high confidence in the software quality, a high quality software development engineering process is necessary that incorporates disciplined specification and implementation of design requirements. • As part of high quality engineering assurance, rigorous software Verification and Validation (V&V) life cycle methodologies are necessary. • Lifecycle methodologies are compliant with IEEE Std 1012-1998, which states The SW V&V is the process of determining whether: • Requirements for a system or components are complete, correct, consistent, and accurate; • Products of each development phase fulfill the requirements or conditions imposed by the previous phase; • Final systems or components comply with specific requirements.

  4. V&V Objectives Per IEEE 1012-1998, • SW V&V is a technical discipline of systems engineering.   • The purpose of SW V&V is to help the development organization build quality into the SW during the SW development life cycle. • V&V processesprovide an objective assessment of software products and processesthrough the SW life cycle. • The assessmentdemonstrateswhether the SW requirements are correct, complete, accurate, consistent, and testable.

  5. SW V&V Plan and Execution To ensure that the quality is built into the design: • SW V&V Plan is produced; • The SW V&V Plan is followed (executed) in the SW life cycle process; • The process produces results (V&V Summary Reports and Final Report).

  6. V&V Guiding Documents(from a top level hierarchical order) • 10 CFR 50 Appendix B • NUREG-0800, BTP 7.14 • RG 1.168-2004 • IEEE-1012-1998 Standards • Software Quality Assurance Plan • Software Verification and Validation Plan • Specific Procedures or Instructions

  7. Verification and Validation ProblemSituation • Validation: • “Are we building the right system?” • Does our problem statement accurately capture the real problem? • Did we account for the needs of all the stakeholders? • Verification: • “Are we building the system right?” • Does our design meet the spec? • Does our implementation meet the spec? • Does the delivered system do what we said it would do? • Are our requirements models consistent with one another? Problem Statement Validation Verification Implementation Statement System

  8. V&V Processes & Activities • Process: Management • Activity: Management of V&V • Process: Development • Activity: Concept Phase V&V • Activity: Requirements Phase V&V • Activity: Design Phase V&V • Activity: Implementation Phase V&V • Activity: Test Phase V&V • Activity: Installation Phase and Checkout V&V • Process: Operation and Maintenance V&V

  9. V&V Tasks • Software Design Review and Verification • Software Traceability Analysis • Software Evaluation • Software Interface Analysis • Software Criticality Analysis • Software Integration Test • System/Acceptance Test • Risk Analysis • Security Analysis (Cyber Security) • V&V Reporting

  10. Applying V & V to a Software Life Cycle (Development Process) 10

  11. installation acquisition planning concept requirements test design implementation The V-model – Overview of the V&V Process

  12. To obtain high confidence and reliability in safety-related software and system, it is required that V&V organization be independent from the development organization technically, financially, and managerially. Regulatory Guide 1.168 and IEEE Std 1012-1998 Guidance on V&V Independence

  13. Technical independence (“fresh viewpoint”) is an important method to detect subtle errors overlooked by those too close to the solution.The Independent V&V effort should formulate its own understanding of the problem and how the proposed system is solving the problem.For software tools, technical independence means that the V&V effort uses or develops its own set of test and analysis tools separate from the developer’s tools. V&V Technical Independence

  14. V&V Summary Report Requirements • Software V&V activity identification • Background information of standards, and source documents with revisions • Synopsis of the review and confirmation that the plan was followed • Anomalies found • Recommendations • The individual who performed the task • The portion of the task performed by the individual (if multiple performers) • The date that the task was performed

  15. V&V Final Report The final V&V Report provides a summary regarding implementation of the SVVP at the conclusion of V&V effort, which may include: • Identification of all V&V documentation • Summary of all Life Cycle V&V Activities • Summary of all V&V task results • Summary of all V&V activity anomaly issues and their resolutions • Assessment of overall software quality based upon a review of the V&V activity • Lessons learned/process improvements and • Recommendations, if any, regarding the overall development process for future nuclear safety-related application software projects

  16. II. Tracing Software/System Requirements

  17. Introduction • Operational software costs 10 to 100 times more to fix. • Statistics show that about half of the software errors detected were attributable to errors not discovered during the requirements or the design phase. • It is imperative that design requirements are complete and traceable in the early phases of software development so that errors can be detected and risk can be minimized. • The requirements tracing activity in nuclear digital I&C is part of the design and quality engineering processes that ensure system and software requirements are complete, testable, and implemented correctly.

  18. Defining Requirement • Per IEEE Std 0610-12-1990 (IEEE Standard Glossary of Software Engineering Terminology), A Requirement is • (1) A condition or capability needed by a user to solve a problem or achieve an objective. (2) A condition or capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed documents. (3) A documented representation of a condition or capability as in (1) or (2).

  19. What are Requirements for? • To show results the User want from the system. • To show traceability back to sources and the history of changes. • To show what the organization needs. • To show what the system must do. • To form a basis for the design and design optimization. • To enable a logical approach to change management. • To partition the work out to Suppliers. • To act as a foundation for testing and payment. • To test the system or any of its parts during development. • To communicate the basics about the system in non-technical terms to all participants.

  20. Well-Formed Requirements Attributes • A well-formed requirement is a statement of system functionality (capability) that can be validated, that must be met or possessed by a system to solve a customer problem or to achieve a customer objective, and that is qualified by measurable conditions and bounded by constraints (IEEE Std 1023-1998). • One of the requirements properties is that each requirement should be implementation independent (IEEE Std 1023-1998). • Per IEEE Std 830-1993, an SRS should be a) Correct; b) Unambiguous; c) Complete; d) Consistent; e) Ranked for importance and/or stability; f) Verifiable; g) Modifiable; and h) Traceable. • US Regulatory Position (RG 1.172-1997) is that a good requirement must posses the above characteristics as defined in IEEE Std 830-1993.

  21. RequirementsTraceabilityDefinition • Requirements traceability refers to the ability to describe and follow the life of a requirement, in both a forwards and backwards direction (i.e., from its origins, through its development and specification, to its subsequent deployment and use, and through all periods of on-going refinement and iteration in any of these phases.)

  22. Requirements Traceability Analysis (e.g., in the Requirements Phase) • Provides tools and methods to trace the software requirements (e.g., SRS) to system requirements (concept documentation, e.g., SyRS) and system requirements to the software requirements. • Analyzes identified relationships for correctness, consistency, completeness, and accuracy.

  23. A simple requirements traceability flow SW Validation Test [Functional Tests (or Black Box) and Structural Tests (or White Box)] System Test SyRS SRS SDD Code Code Tests

  24. Traceability and Test Coverage • The traceability shows where the items fromStep N-1 are addressed in step N and in the validation program. But itdoes not show if the item has been completelycovered. • A complementarycoverageanalysisisthennecessary. • The traceabilityanalysis and the coverageanalysis are complementary to ensurethat all the initial requirements are completelysatisfied in the final product. • An exampleis the logicdiagram – to assure thateachlogicpathisaddressed in testing. • Weshould devise traceabilityenvironmentswith a right mix of automated and non-automatedaids.

  25. yes No,Impact analysis OK? ModificationNew rev. of SDD ModificationNew rev. of code Traceability and Design Change (Configuration Control) There canbeseveral rounds of testingDo not lose the control of the configuration duringtesting!

  26. Traceability and Design Change • Traceability supports Change Impact Analysis • Traceability supports Regression Test Selection • Note: “Items that could change because of design changes, review, or audit should be configuration items subject to formal change control.” (RG 1.169-1997)

  27. Summary • High confidence in digital I&C software/system can be obtained through disciplined specification and implementation of design and rigorous V&V – life cycle process • It is a regulatory requirement to perform rigorous Independent V&V on safety-related digital I&C systems • A rigorous V&V process is specified in IEEE Std 1012-1998, which is endorsed by RG 1.168-2004 • Requirements Traceability Analysis is an important V&V approach to fulfill the V&V objective. Traceability supports Change Impact Analysis; and traceability supports Regression Test Selection. • Digital I&C system safety can be ensured with documented evidence of a defined and acceptable V&V process being followed and results generated

  28. Thank you! QUESTIONS?

More Related