Chapter 2.11

1. Chapter 2.11 Program Validation

2. = Reliable Hardware AND Reliable Software AND Compatible Hardware and Software Reliable System

3. Good Design Diagnostic Circuits Build-in Redundancy Diagnostic Programs Reliable Hardware

4. should be ADEQUATE (accomplishes the tasks it is used for) and ROBUST (survives all abnormal inputs) Reliable Software

5. Before being used a program should be VALIDATED for ADEQUACY and ROBUSTNESS Software Validation

6. Validation can never improve the quality of software, it can only prevent the use of erroneous programs Software ValidationWARNING !

7. Syntax errors Detected by the compiler Run-time errors The program stops with an error message Special recovery software is activated Erroneous results Consequences can be catastrophic !!! Software engineering tries to push errors upwards in the list Program Errors

8. Mandatory Declarations BEGIN ... L0AD := 1000. ... Force:= LOAD * ... ... VAR LOAD, Force : REAL; BEGIN ... L0AD := 1000. Undeclared:L0AD ... Force := LOAD * ... Cross reference: .. . L0AD : 15 LOAD : 20,27,39,44 Normal execution Erroneous value of Force No execution Compilation error

9. Static validation: based on the analysis of the program source code required could guarantee correctness Dynamic validation (testing): based on the execution of the program only the object code is required can only prove the presence of errors, never their absence Static vs. Dynamic Validation

10. Start with small building blocks Finish with the entire system Natural and intuitively attractive Major incompatibilities between building blocks discovered last For dynamic validation: Test programs required for each building block Bottom-up Validation

11. Start with the entire system, ignoring details. Finish with the small building blocks Somewhat counter-intuitive Major incompatibilities between building blocks discovered first For dynamic validation: Dummy substitutes for all building blocks have to be made for high-level tests. Top-down Validation

12. Syntax checking by the compiler Manual code checking Desk checking Walk through Inspection Data flow analysis Correctness proofs Static Validation Techniques

13. Syntax checking by the compiler Manual code checking Desk checking Walk through Inspection Data flow analysis Correctness proofs Static Validation Techniques

14. Syntax checking by the compiler Manual code checking Desk checking Walk through Inspection Data flow analysis Correctness proofs Static Validation Techniques

15. Desk checking Informal code reading preferably by somebody else Walk through Formal meeting Simulated execution of code on blackboard Inspection Formal meeting Contradictory discussion of code and programming style Manual code checking

16. Syntax checking by the compiler Manual code checking Desk checking Walk through Inspection Data flow analysis Correctness proofs Static Validation Techniques

17. Example : Second order equation Read a,b,c d:=b 2 - 4ac No d > 0 ? No d = 0 ? x1 := … x:= … x2 := … Write "no roots" Write x1, x2 Write x

18. Dataflow of a : Read a,b,c d:=b 2 - 4ac No d > 0 ? No d = 0 ? x1 := … x:= … x2 := … Write "no roots" Write x1, x2 Write x

19. Dataflow of x1 : Read a,b,c d:=b 2 - 4ac No d > 0 ? No d = 0 ? x1 := … x:= … x2 := … Write "no roots" Write x1, x2 Write x

20. := X Uninitialized variable ? x := ...

21. x := exp1 Why this statement ??? x := exp2 Write x Useless statement

22. Syntax checking by the compiler Manual code checking Desk checking Walk through Inspection Data flow analysis Correctness proofs Static Validation Techniques

23. Specifications Inputs Outputs EQUIVALENCE Program Correctness Proof

24. Formal proofs can be long and complex Impossible for large systems Very error prone Could be automated Doesn't help with specification errors… Useful for well specified, short, but complex, algorithms. Correctness Proof

25. Testing No guarantee !!! Black box vs. white box testing. Test data Debugging The program state Control flow breakpoints Data flow breakpoints Real time considerations Dynamic Validation Techniques

26. Testing No guarantee !!! Black box vs. white box testing. Test data Debugging The program state Control flow breakpoints Data flow breakpoints Real time considerations Dynamic Validation Techniques

27. "Testing can proof the presence of errors but not their absence" TESTING E.W.Dijkstra

28. Testing No guarantee !!! Black box vs. white box testing. Test data Debugging The program state Control flow breakpoints Data flow breakpoints Real time considerations Dynamic Validation Techniques

29. Testing No guarantee !!! Black box vs. white box testing. Test data Debugging The program state Control flow breakpoints Data flow breakpoints Real time considerations Dynamic Validation Techniques

30. Test data Tests should exercise all paths through a program a b c d 1 3 2 1 1 2 1 0 1 1 1 -3 Read a,b,c d:=b 2 - 4ac No d > 0 ? No d = 0 ? x1 := … x:= … x2 := … Write "no roots" Write x1, x2 Write x

31. Test data Tests should exercise exceptional values a b c d 0 1 1 1 1 0 1 -4 1 1 0 1 Read a,b,c d:=b 2 - 4ac No d > 0 ? No d = 0 ? x1 := … x:= … x2 := … Write "no roots" Write x1, x2 Write x

32. Test data Tests should exercise very large and very small values a b c 1 10000000001 10000000000 Read a,b,c d:=b 2 - 4ac No d > 0 ? No d = 0 ? x1 := … x:= … x2 := … Write "no roots" Write x1, x2 Write x

33. Testing No guarantee !!! Black box vs. white box testing. Test data Debugging The program state Control flow breakpoints Data flow breakpoints Real time considerations Dynamic Validation Techniques

34. Testing No guarantee !!! Black box vs. white box testing. Test data Debugging The program state Control flow breakpoints Data flow breakpoints Real time considerations Dynamic Validation Techniques

35. Testing No guarantee !!! Black box vs. white box testing. Test data Debugging The program state Control flow breakpoints Data flow breakpoints Real time considerations Dynamic Validation Techniques