Requirements Validation

Requirements Validation Section Four Version: 1.0 Mehr 1386 Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

What is V&V? What • Purpose of V&V is to deliver a product that works properly and performs to customer expectations. • Verification is the process of confirming deliverable hardware and software are in compliance with the functional, performance, and design requirements.(Have we built the system right?) • Validation is the process of providing evidence that a systems meets the needs of the User.(Have we built the right system?) What we are trying to do is to establish confidence in the system, support equipment, test equipment, and people. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Requirements validation What • Concerned with demonstrating that the requirements define the system that the customer really wants. (Define the right system) • Pickup any problem before resources are committed to addressing the requirements. • Requirements error costs are high so validation is very important • Fixing a requirements error after delivery may cost up to 100 times the cost of fixing an implementation error Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Requirements checking What • Validity. Does the system provide the functions which best support the customer’s needs? • Consistency. Are there any requirements conflicts? • Completeness. Are all functions required by the customer included? • Realism. Can the requirements be implemented given available budget and technology • Verifiability. Can the requirements bechecked? Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Characteristics of requirements What • Correct • Unambiguous • Complete • Testable • Consistent • Deal only with the problem • Modifiable • Traceable Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Correct When • Each requirement statement accurately represents the functionality required of the system to be built. • Example (of an incorrect requirement): • Problem domain (real life) states that policeman ID numbers are in the range [10000…) and the requirements document specifies that each policeman has an ID number. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Unambiguous When • The difficulty of ambiguity stems from the use of natural language which in itself is inherently ambiguous. • There is one and only one interpretation for every requirement. • Requirement statements should be short, explicit, precise and clear. • A glossary should be used when a term used in a particular context could have multiple meanings (I.e. “the user”). • Formal requirements languages help reduce ambiguity. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Unambiguous (cont. ) How • Example of ambiguity • The TVRS shall store changes made in the details of a traffic report as soon as the data is entered. • Disambiguation: • The TVRS shall store changes made in the details of a traffic report if and only all input fields are valid and user approved saving of data Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Complete When • A requirements document is complete if it includes all of the significant requirements, relating to • functionality, • performance, • design constraints attributes • external interfaces. • No sections are marked “To be determined” (TBD). • Conforms to the company standards. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Testable When • A requirements document is testable (verifiable) if and only if every requirement statement in it is testable. • A requirement is testable if and only if there is some finite cost-effective way in which a person or machine can check to see if the software product satisfies that requirement. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Testable (cont.) How • Example of a non-verifiable requirement: • The TVRS shall complete storage of data within a reasonable time of the user confirming a “Save” sequence. • Example of a verifiable requirement: • The TVRS shall complete storage of data within 5 seconds of the user confirming a “Save” sequence, 80% of the time. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Consistent When • Three types of conflicts: • Different terms used for the same object: • F323 and a “Policeman Details Form” might be used to describe the same form. • Characteristics of objects: • In one part of the requirements document: • “A policeman ID shall consist of decimal digits only”, • while in another part • “Incase the policeman ID consists of non-alphanumerical characters, display an error message”. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Consistent (cont.) When • Logical or temporal faults: “A follows B” in one part, “A and B occur simultaneously” in another. • “TVRS shall support removal of a policeman record from the personal database” vs. “TVRS shall support read-only access to policeman details”. Do clients know what a database is? Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Deal only with the problem When • Requirements should state “what” is required at the appropriate system level, not “how”. • In some cases, a requirement may dictate how a task is to be accomplished. • Avoid telling the designer “how” to do this job, instead state “what” has to be accomplished. • Requirements should be understood by the clients as well as the developers. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Modifiable When • A requirements document is modifiable if its structure and style are such that changes can be made easily, completely and consistently: • Easy to use organization – table of contents, index and cross references. • No redundancy – a requirement should not be in more than one place. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Traceable When • Each requirement should be contained in a single, numbered paragraph so that it may be referred to in other documents: • Backward traceability - implies that we know why every requirement exists • Each requirement explicitly references its source in previous documents. • Forward traceability – all documents to follow will be able to reference each requirement. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Traceable (cont.) How • Example: • 2.1 Functional requirements: • 2.1.1 TVRS initialization: • … • 2.1.15 TVRS shall display the user login window (see section 2.1.2.2) • 2.1.2 TVRS user interfaces: • 2.1.2.1 All user interaction with the TVRS shall occur by means of a graphical user interface. • 2.1.2.2 User login window: • … Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Requirements validation techniques What • Requirements reviews • Prototyping • Acceptance tests • Model Validation and Automated consistency analysis Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Requirements reviews What • Requirements review is a systematic manual analysis of the requirements • Regular reviews should be held while the requirements definition is being formulated • Both client and contractor staff should be involved in reviews • Reviews may be formal (with completed documents) or informal. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

General Guidelines for Review What • Good communications between developers, customers and users can resolve problems at an early stage • Always conduct reviews in a meeting format, although the meeting participants might prepare some reviews on their own. • Continuously check what is produced to make sure the product quality is as high as possible. Checkpoints are provided for this purpose; refer to the checkpoints for each analysis activity. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Roles in Review Meetings Who • a person has essential knowledge of the business or technology domain and detailed knowledge of the applied facilitation and modeling techniques: • Requirements Reviewer • Requirements Specifier • System Analyst • possibly at milestones such as the beginning or end of a Phase: • Stakeholders - customers and end-users (Where possible) • Change Control Manager (Where reviewing Change Requests) • Test Designer (Optional) • Software Architect (Optional, usually in Inception and Elaboration) • Project Manager (Optional, usually at Phase Start) Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Review checks What • Verifiability. Is the requirement realistically testable? • Comprehensibility. Is the requirement properly understood? • Traceability. Is the origin of the requirement clearly stated? • Adaptability. Can the requirement be changed without a large impact on other requirements? Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

RUP Checkpoints: Requirements Attributes What • Has the correct set of requirements attributes been used as specified in the Requirements Management Plan? • Have attributes been set up for each requirement type to account for the following, where applicable, for each requirement? • Tracking status? • Benefit? • Rationale? • Level of effort to implement? • Type and amount of each type of risk involved in implementing? • Schedule risk? • Resource risk? • Technical risk? • Stability of the requirement? • Target release? • Assignment? • Marketing input? • Development input? • Revision history? • Location? • Reasons for change? • Inconclusive requirements? • Have all traceabilities been set up as specified for the project in the Requirements Management Plan? Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Requirements validation techniques What • Requirements reviews • Prototyping • Acceptance tests • Model Validation and Automated consistency analysis Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

System prototyping What • Prototyping is the rapid development of a system • Using an executable model of the system to check requirements. • In the past, the developed system was normally thought of as inferior in some way to the required system so further development was required • Now, the boundary between prototyping and normal system development is blurred and many systems are developed using an evolutionary approach Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Uses of system prototypes What • The principal use is to help customers and developers understand the requirements for the system • Requirements elicitation. Users can experiment with a prototype to see how the system supports their work • Requirements validation. The prototype can reveal errors and omissions in the requirements • Prototyping can be considered as a risk reduction activity which reduces requirements risks Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Prototyping benefits Why • Misunderstandings between software users and developers are exposed • Missing services may be detected and confusing services may be identified • A working system is available early in the process • The prototype may serve as a basis for deriving a system specification • The system can support user training and system testing Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Prototyping benefits Why • Improved system usability • Closer match to the system needed • Improved design quality • Improved maintainability • Reduced overall development effort Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Prototyping in the software process What • Evolutionary prototyping • an initial prototype is produced and refined through a number of stages to the final system • Throw-away prototyping • a practical implementation of the system is produced to help discover requirements problems and then discarded. The system is then developed using some other development process Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Prototyping objectives What • Evolutionary prototyping • to deliver a working system to end-users. • The development starts with those requirements which are best understood. • Throw-away prototyping • to validate or derive the system requirements. • The prototyping process starts with those requirements which arepoorly understood Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Evolutionary prototyping What • Must be used for systems where the specification cannot be developed in advance e.g. AI systems and user interface systems • Based on techniques which allow rapid system iterations • Verification is impossible as there is no specification. • Validation means demonstrating the adequacy of the system. Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Evolutionary prototyping What Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Evolutionary prototyping advantages Why • Accelerated delivery of the system • Rapid delivery and deployment are sometimesmore important than functionality or long-term software maintainability • User engagement with the system • Not only is the system more likely to meet user requirements, • they are more likely to commit to the use of the system Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Evolutionary prototyping How • Specification, design and implementation are inter-twined • The system is developed as a series of increments that are delivered to the customer • Techniques for rapid system development are used such as CASE tools and 4GLs • User interfaces are usually developed using a GUI development toolkit Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Evolutionary prototyping problems What • Management problems • If management processes assume a waterfall model of development • Specialist skills are required which may not be available in all development teams • Maintenance problems • Continual change tends to corrupt system structure so long-term maintenance is expensive • Contractual problems Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Prototypes as specifications What • Some parts of the requirements (e.g. safety-critical functions) may be impossible to prototype and so don’t appear in the specification • An implementation has no legal standing as a contract • Non-functional requirements cannot be adequately tested in a system prototype Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Throw-away prototyping What • Used to reduce requirements risk • The prototype is developed from an initial specification, delivered for experiment then discarded • The throw-away prototype should NOT be considered as a final system • Some system characteristics may have been left out • There is no specification for long-term maintenance • The system will be poorly structured and difficult to maintain Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Prototype delivery Why not • Developers may be pressured to deliver a throw-away prototype as a final system • This is not recommended • It may be impossible to tune the prototype to meet non-functional requirements • The prototype is inevitably undocumented • The system structure will be degraded through changes made during development • Normal organisational quality standards may not have been applied Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Rapid prototyping techniques What • Various techniques may be used for rapid development • Dynamic high-level language development • Database programming • Component and application assembly • These are not exclusive techniques - they are often used together • Visual programming is an inherent part of most prototype development systems Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Component and application assembly What • Prototypes can be created quickly from a set of reusable components plus some mechanism to ‘glue’ these component together • The composition mechanism must include control facilities and a mechanism for component communication • The system specification must take into account the availability and functionality of existing components Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Visual programming What • Scripting languages such as Visual Basic support visual programming where the prototype is developed by creating a user interface from standard items and associating components with these items • A large library of components exists to support this type of development • These may be tailored to suit the specific application requirements Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Visual programming with reuse What Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Problems with visual development What • Difficult to coordinate team-based development • No explicit system architecture • Complex dependencies between parts of the program can cause maintainability problems Amirkabir University of Technology, Computer Engineering Faculty, Intelligent Systems Laboratory

Requirements Validation