CS 217 Software Verification and Validation

1. CS 217 Software Verification and Validation Week 3, Summer 2014 Instructor: Dong Si http://www.cs.odu.edu/~dsi

2. REVIEW OF LAST CLASS

3. LOGIC IN COMPUTER SCIENCE Week 2, topic 1

4. Motivation • LOGIC enabled mathematicians to point out WHY a proof is wrong, or WHERE in the proof, the reasoning has been faulty. • Faults (bugs) have been detected inproofs (programs) • Is such a tool that by symbolizing arguments rather than writing them out in some natural language (which is fraught with ambiguity), checking the correctness of a proof becomes a much more viable task.

5. Motivation • Since the latter half of the 20th century, logic has been used in computer science for various purposes ranging from software validation and verification to theorem-proving.

6. Introduction to Logic • CS areas where we use LOGIC • Architecture (logic gates) • Software Engineering (Validation & Verification) • Programming Languages (Semantics & Logic Programming) • AI (Automatic theorem proving) • Algorithms (Complexity) • Databases (SQL)

7. Fundamental of Logic • Declarative statements • Examples of declarative statements • “A is older than B” • “There is ice in the glass” • In CIS, describing the data (variables, functions, etc.)

8. Propositions - a statement that is either true or false. • For every proposition p, either p is T or p is F • For every proposition p, it is not the case that p is both T and F

9. Fundamental of Logic • We are interested in precisedeclarative statements about computer systems and programs. (Verification) • We not only want to specify such statements, but also want to check whether a given program or system fulfills specifications that user needs. (Validation)

10. Propositional Logic: Basics • Propositional logic describes ways to combinesome true statements to produce other true statements. • If it is proposed that `Jack is taller than John' and `John can run faster than Jack' are both T =`Jack is taller than Johnand Johncan run faster than Jack'. • Propositional logic allows us to formalize such statements. • In concise form: A ^B

11. Propositional Logic • Composition of atomic sentences p: I won the lottery yesterday q: I will purchase a lottery ticket today r: I played a football game yesterday • ~ p: Negation. “I did not win the lottery last week” • p v r: Disjunction. The statement is true if at least one of them is true. “I won the lottery or played a football game yesterday.”

12. Propositional Logic • p^ r: Conjunction. “Yesterday I won the lottery and played a football game.” • p q: Implication. “If I won the lottery last week, then I will purchase a lottery ticket today.” p is called the assumption and q is called conclusion. • p implies q • If p then q

13. Natural Deduction • Proof • Set of rules which allow us to draw a conclusionby given a set of preconditions • Constructing a proof is much like a programming! • It is not obvious which rules to apply and in what order to obtain the desired conclusion, be careful to choose proof rules!

14. Rules of Natural Deduction • Fundamental rule 1 (rule of detachment) p p q . . . q • The rule is a valid inference because [p ^ (p q)] q is a tautology!

15. Rules of Natural Deduction • Example: if it is 11:00 o’ clock in Norfolk if it is 11:00 o’ clock in Norfolk, then it is 11:00 o’ clock in DC then by rule of detachment, we must conclude: it is 11:00 o’ clock in DC

16. Rules of Natural Deduction • Fundamental rule 2 (transitive rule) p q q r . . . p r This is a valid rule of inference because the implication (p q) ^ (q r) (p r) is a tautology!

17. Rules of Natural Deduction • FR 3 (De Morgan’s law) ~(p v q) = (~p) ^ (~q) ~(p ^ q) = (~p) v(~q) • FR 4 (Law of contrapositive) p q = (~q ~p) • FR 5 (Double Negation) ~(~p) = p

18. Examples of Arguments • If a baby is hungry, then the baby cries. If the baby is not mad, then he does not cry. If a baby is mad, then he has a red face. Therefore, if a baby is hungry, then he has a red face. • Model this problem!! • h: a baby is hungry c: a baby cries m: a baby is mad r: a baby has a red face h c ~m ~c m r . . . h r h c c m m r . . . h r

19. Logic is the Skeleton • What remains when arguments are symbolized is the bare logical skeleton • It is this form that enables us to analyze the program / code / software. • Software V&V = Logical proof & Logic error detection

20. Answers to Quiz 2 • Q1. Let H = "John is healthy" W = "John is wealthy" S = "John is smart" (1). “John is healthy and wealthy but not smart”: Answer: H Λ W Λ ¬S (2). “John is not wealthy but he is healthy and smart”: Answer: ¬W Λ H Λ S (3). “John is neither healthy nor wealthy nor smart”: Answer: ¬H Λ ¬W Λ ¬S

21. Q2. Let P = “You stay at the hotel” Q = “You watch TV” R = “You go to the museum” S = “You spend some time in the museum” "You can either (stay at the hotel and watch TV ) or (you can go to the museum and spend some time there)” Answer: (P Λ Q) V (R Λ S)

22. Q3. Let P, Q, and R be the following propositions: P = “You get an A on the final exam” Q = “You do every exercise in the book” R = “You get an A in this class” (1). “You get an A in this class, but you do not do every exercise in the book.” Answer: R ∧ ¬Q

23. (2). “To get an A in this class, it is necessary for you to get an A on the final.” Answer: R ⇒ P “If you want an A in this class, you must have an A on the final.” “If you got an A in this class, that means you have gotten an A on the final.” (3). “Getting an A on the final and doing every exercise in the book is sufficient for getting an A in this class.” Answer: P ∧ Q ⇒ R

24. Q4. Problem: “Tom is a math major but not computer science major” M: Tom is a math majorC: Tom is a computer science major • Tasks: Use De Morgan's Lawto write the negation of the above statement as logic expression

25. Answer: • Original: • M Λ ¬ C (Tom is a math major but not computer science major) • Negation: • ¬ (M Λ ¬ C) = ¬ M V ¬ (¬ C) (De Morgan's Laws) = ¬ M V C (Double negation rule)

26. CODE COVERAGE TESTING Week 2, topic 2

27. Definition • Code coverage is a measure used to describe the degree to which the source code of a program is tested by a particular test suite. • A program with high code coverage has been more thoroughly tested and has a lower chance of containing software bugs than a program with low code coverage.

28. Coverage criterias • Function coverage - Has each function (or subroutine) in the program been called? • Statement coverage - Has each statement in the program been executed? √ √ √

29. Coverage criterias • Branch coverage - Has each branch of each control structure (such as in if and case statements) been executed? • For example, given an if statement, have both the T and F branches been executed? • Another way of saying this is, has every edge in the program been executed?

30. Coverage criterias • Condition coverage- Has each Boolean sub-expression evaluated both to true (T) and false (F) ? • In “A and B”, • if sub-expression A is evaluated both to T and F • if sub-expression B is evaluated both to T and F

31. Example • consider the following C++ function: • If during this execution function 'foo' was called at least once, then function coverage for this function is satisfied.

32. Example • consider the following C++ function: • Statement coverage for this function will be satisfied if it was called e.g. as foo(1,1), as in this case, every line in the function is executed including ’z = x;’.

33. Example • consider the following C++ function: • Tests calling foo(1,1) and foo(0,1) will satisfy branch coverage because, in the first case, the 2 if conditions are met and z = x; is executed, while in the second case, the first condition (x>0) is not satisfied, which prevents executing z = x;.

34. Example • consider the following C++ function: • Condition coverage can be satisfied with tests that call foo(1,1), foo(1,0) and foo(0,0). These are necessary because in the first two cases, (x>0) evaluates to true, while in the third, it evaluates false. At the same time, the first case makes (y>0) true, while the second and third make it false. (x>0) && (y>0) T,F T,F

35. Condition / branch coverage? • Condition coverage does not necessarily imply branch coverage. For example: • Condition coverage can be satisfied by two tests: • However, this set of tests does not satisfy branch coverage since neither case will meet the if condition.

36. Condition / branch coverage? IF ( AND ) THEN … ELSE … Y>0 X>0 T , F ? T , F ? T F

37. Answers to Quiz 2 • Q5. Consider the following pseudo code of a program ‘Fun’. It takes x and y as input variables, and outputs the value of z: fun (x, y) { z = 1; IF ((x>z) AND (y>z)) THEN z = 0; Output z; } Fun (0, 0) Fun (2, 0) Fun (0, 2) Fun (2, 2) Fun (8, 9)

38. Consider the following five test cases: 1. Fun (0, 0) 2. Fun (2, 0) 3. Fun (0, 2) 4. Fun (2, 2) 5. Fun (8, 9) Function coverage: all Statement coverage: 4 and 5 Branch coverage: all (4&5 make the branch ’IF’ to T, 1&2&3 make it to F) Condition coverage: all (2&4&5 make the sub-expression ‘x>z’ to T, 1&3 make it F)

39. Bonus Question • What happened if switch AND with OR logic in the program: fun (x, y) { z = 1; IF ((x>z) OR(y>z)) THEN z = 0; Output z; } Fun (0, 0) Fun (2, 0) Fun (0, 2) Fun (2, 2) Fun (8, 9) Function coverage: Statement coverage: Branch coverage: Condition coverage:

40. Input Space Partitioning Week 3

41. Black-box testing • Program is treated as a black box. • Different inputs will be used as tests. • Testing based solely on analysis of requirements (specification, user documentation, etc.). • Black-box techniques apply to all levels of testing (e.g., unit, integration and system).

42. Test Data and Test Cases • Test data: Inputs which have been devised to test the system. • Test cases:Inputs to test the system and the predicted outputs from these inputs if the system operates according to its specification.

43. Input Domains • The input domain to a program contains all the possible inputs to that program • For even small programs, the input domain is so large that it might as well be infinite • Testing is fundamentally about choosing finite setsof values from the input domain

44. Input Domains • Input parameters define the scope of the input domain • Parameters to a program/function • Data read from a file • Domain for each input parameter is partitioned into regions • At least one value is chosen from each region y= Absolute(x) -3 -2 -1 0 1 2 3…… x<0, negative x=0, zero x>0, positive x = -3, x = 0, x = +2

45. Data Testing • If you think of a program as a function, the input of the program has its own domain. • Examples of program data are: • words typed into MS Word • numbers entered into Excel • picture displayed in Photoshop • …

46. Input space partitioning • Also known as equivalence partitioning. • Reducing the huge (or infinite) set of possibletest cases into a smallbut equally effectiveset of test cases. • Dividing input values into valid and invalid partitions and selecting representative values from each partition as test data.

47. Equivalence partitions • Sometimes boundary values need more tests

48. b1 b2 b3 Partitioning Domains • DomainD • Partition schemeq of D • The partition q defines a set of blocks, Bq = b1 , b2 , … bQ • The partition must satisfy two properties : • blocks must be pairwise disjoint (no overlap) • together the blocks cover the domain D (complete)

49. Using Partitions – Assumptions • Choose a value from each partition • Each value is assumed to be equally useful for testing • Application to testing • Find characteristics in the inputs : parameters, semantic descriptions, … • Partition each characteristic • Choose tests by combining values from characteristics • Example Characteristics • Input X is a number (null, negative, zero, positive…) • Input X is a picture (binary, gray scale, …) • Input X is a multimedia disk to a device (DVD, CD, VCD, …)

50. Example 1: compare two numbers • Function ‘compare (x, y)’ • Inputs: Two numbers – x and y • Outputs: A larger number between x and y (x, y) z z = Compare (x, y)