V&V&S

1. Verification, Validation and Synthesis: doing away with defects V&V&S

2. Quality and quality assurance methods Quality: process quality = meeting the specification. system quality = playing the role required by the environment. Methods: 1. Constructive methods that aim to generate the right results in the first place => The SDL methodology, program generation. 2. Corrective methods that aim to detect errors and make corrections. => Verification and validation.

3. Verification and Validation Barry Boehm (Boehm, 1981): Verification: To establish the truth of correspondence between a software product and its specification (from the Latin veritas, “truth”). Validation: To establish the fitness or worth of a software product for its operational mission (from the Latin valere, “to be worth”). Verification: Are we building the product right? Validation: are we building the right product?”

4. Developers needs needs needs Verification and Validation in TIMe verify domain verify speci- fication Application design Framework design Architecture design verify Validate imple-mentation instance Market and users config. validate system

5. Results Requirements Design Implementation Techniques Corrective methods (V&V) Constructive methods Defects and errors Synthesize -design -implement. Reviews analysis Testing

6. Analysis • Syntax analysis: by editor • Static semantic analysis: by analyzer • Dynamic semantic analysis: by simulation and reachability analysis (modelchecking) • Verification: by manual inspection and checking formal relationships between models • Validation: by manual inspection, by testing and formal compatibility checks

7. Static signal check 1 • Derive valid input and output signal sets from process graph • Check signal sets against signal lists on each connection: P2in = L2 + L3 + L5 P2out = L1 + L4 + L6 • Check signal destinations: P2 out-to-p1 = L1 P2 out-to-p3 = L4 P2 out-to-p3 = L6

8. Static signal check 2 • Valid input and output signal sets against gate signal lists once for each type • Signal lists against each other for each connection • Signal destinations

9. Static signal check 3 • Find logical signal routes by signal list partitioning conserving flows • Signal sets against signal lists • Signal destinations Who talks together here?

10. Dynamic analysis Liveness = something good will eventually happen (system specific) safety = nothing bad will ever happen (system independent) 1. Reachability analysis Full (up to 10**5 states) Controlled partial (up to 10**8 states) Random simulation (larger) 2. Structural reasoning Invariants Logical inference on equations, rules, axioms Model checking 3. Execution Simulation Testing

11. Reachability analysis • Iteratively generate all reachable system states • Analyse each state for • deadlocks; • incompleteness; • other errors. • Analyse the generated graph • partial deadlock; • liveness; • etc.

12. [Rb,Eb] [Gb,Sa,Qa] [Aa] A B [Ga, Sb,Qb] [Ra,Ea] [Ab] A B 1 1 Ra Sa Rb Sb Sb Ga Sa Gb 3 8 3 8 Gb Qa Ga Qb 1 1 Aa Ab 5 5 Ea Eb Qb Qa 1 1 Are there any problems here?

13. Step 1: Make transition charts

14. Step 2: Generate reachability graph Find all possible behaviours considering that a signal transfer takes two steps: • send • consume

15. The errors: Unspecified receptions and deadlocks may occur The reason: mixed initiatives not properly handled Graph “explodes” because independent behaviours are interleaved and signals are queued Many paths lead to the same error Findings Is there a simpler way??

16. [Rb,Eb] [Gb,Sa,Qa] [Aa] A B [Ga, Sb,Qb] [Ra,Ea] [Ab] A B 1 1 Ra Sa Rb Sb Sb Ga Sa Gb 3 8 3 8 Gb Qa Ga Qb 1 1 Aa Ab 5 5 Ea Eb Qb Qa 1 1 Working directly in SDL Association role behaviour = the visible behaviour of one object when observed from another (associated) object. Mixed initiative state = a state with inputs from two independent sources (i.e. association roles) mixed initiative state invisible from B

17. Role behaviour and input consistent roles • Deriving a role behaviour: • Replace invisible inputs by “none” (or just make a mental note) • Remove invisible outputs (or just ignore them) • The result is a process graph with only visible inputs and outputs. (or just a mental view on the original graph) • Input consistency: • An invisible transition is a transition with a none input. • An invisible transition is input consistent iff the next-state explicitly accepts all the visible inputs of the (present) state. • The role is input consistent iff every invisible transition in the role is input consistent. A 1 invisible transitions none Sa Sb Ga 1 and 3 are not input consistent because 3 does not accept Sa 3 8 Gb Qa 1 5 5 and 1 are input consistent because 1 accepts more than 5 none Qb 1

18. How to ensure input consistency ? • When checking - for each role: • Find invisible transitions • Check that every invisible transition is input consistent • Consider transitions from mixed initiative states first. • When designing: • for each state, in particular mixed initiative states, check for each role that all transitions are input consistent. Mixed initiative state A invisible transition 1 none Sa Sb Ga 1 and 3 are not input consistent 3 8 Gb Qa 1 5 none 5 and 1 are input consistent Qb The intuition behind this is that the associated process will not immediately observe an invisible transition and therefore the same visible outputs should be accepted in both states 1

19. S-Rule: Input consistent process Design SDL processes such that all their role behaviours, i.e. the behaviour visible to processes in their environment, are input consistent. In particular check the vicinity of mixed initiative states

20. SDL and MSC • The SDL behaviour shall “contain” all the MSC behaviours in addition the SDL behaviour may contain: • cases not treated by the MSCs • behaviour on other interfaces • behaviour in other services MSC (instance) behaviour tree SDL SDL (object) behaviour tree MSCs “contains” structure

21. Verifying SDL • generate all possible traces defined by the MSCs • check that all traces may execute in the global behaviour tree for the corresponding SDL (using the Validator) MSC behaviour tree MSCs SDL SDL behaviour tree

22. msc User_accepted User AC System Idle Code OK Unlock Card out Door unlocked msc User_rejected User AC System Idle Code Card out NOK Idle Synthesizing behaviour 1 process ACSystem Idle Code Code CardOut CardOut OK NOK Unlock Door unlocked Idle How to correct this?

23. ...by combining and branching Idle Idle Code OK NOK Code Code Validate CardOut CardOut CardOut CardOut OK NOK OK NOK Unlock Unlock Door unlocked Idle Door unlocked Idle

24. Synthesizing behaviour 2 MSC EnterZone Process Type UserServer User Authenticator Authorizer UserServer DCL Cardid, PIN, zone integer; Idle Card [cardid] Enter PIN Card (Cardid) ValidatePIN [cardid,PIN] Givepin [PIN] Enter PIN NOK exc Illegal PIN wait for PIN OK ValidateRights [cardid,zone] GivePIN (PIN) NOK exc ValidatePIN (cardid, PIN) AccessDenied OK AccessGranted Validate PIN

25. Synthesizing behaviour 3 MSC EnterZone Process Type UserServer User Authenticator Authorizer UserServer Validate PIN Card [cardid] NOK OK Enter PIN ValidatePIN [cardid,PIN] Givepin [PIN] ValidateRights (cardid, zone) IllegalPIN NOK exc Illegal PIN Idle ValidateRights OK ValidateRights [cardid,zone] NOK OK NOK exc AccessDenied AccessDenied AccessGranted OK AccessGranted Idle Idle

26. Synthesizing behaviour 4 General approach: • Split each MSC into instances, and make corresponding SDL fragments. • Combine all the SDL fragments for a given process into a more complete behaviour • Remove non-determinism by combining and adding decisions or states • Add unspecified behaviour details • Add exceptions • Consider all possibilities in each state: ensure that all roles are input consistent!

27. Decompose Verify MSC Synthesize Validate interface Verify SDL Specification and design Specification ••• Design •••

28. Validating interfaces • Static signal checks: are all output signals received? • Aligning MSC: do both sides align with the same MSCs? • Comparing role behaviours manually: are required roles contained in provided roles? • Simulating with SDT: will the interface work for all simulated cases? • Reachability analysis with SDT: will the validator find errors using state space exploration? structure Type Type

29. Aspects: verifying MSC decomposition verifying SDL behaviour: object behaviour composition behaviour Principle: every trace defined by the MSCs shall be possible in the SDL context behaviour Verify verifying MSC decomposition aligning structure Verifying SDL behaviour Verifying SDL behaviour

30. MSC MSC application infrastructure support sw hw MSC MSC and testing system Abstract test suite Executable test suite system under test Tester verdict

31. Testing what? Application (SDL). Application support: • Protocols. • Input-output modules. General support: • Routing. • Operating system. • SDL runtime support. • Error handling. • Test and debug facilities. Performance (non-functional): • Real time response, delays. • Traffic handling capacity. • Load control.