1 / 26

Software Test Coverage

TEST. TCOV. Software Test Coverage. Test Coverage Measures Code Coverage Measurement Levels of Code Coverage Method Coverage Method, Statement, Branch & Path Coverage White Box Test Coverage Data Coverage Measurement Levels of Data Coverage Class Coverage Interface Coverage

brand
Download Presentation

Software Test Coverage

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TEST TCOV Software Test Coverage Test Coverage Measures Code Coverage Measurement Levels of Code Coverage Method Coverage Method, Statement, Branch & Path Coverage White Box Test Coverage Data Coverage Measurement Levels of Data Coverage Class Coverage Interface Coverage State Coverage Data Table Coverage Data Coverage Report Functional Coverage Measurement Levels of Functional Coverage Black Box Test Coverage Functional Test Coverage Predicted Error Coverage Inheriting Test Coverage Non-Transitivity of Test Coverage CRUD Object Access Coverage CRUD Object Access Coverage Matrix Document Coverage Differences in Test Coverage 1 13 2 14 3 15 4 16 5 17 6 18 7 19 8 20 21 9 10 22 11 23 12 24

  2. TEST TCOV-1 Test Coverage Measures Data Coverage Test Coverage Code Coverage Functional Coverage Method Coverage Statement Coverage Branch Coverage Path Coverage Object Coverage Attribute Coverage Value Coverage State Coverage Function Coverage Function Outcome Coverage Function Chain Coverage Function State Coverage

  3. TEST TCOV-2 Code Coverage Measurement Path Branch Test Coverage = 100%could mean: Statement Method Coverage Every Method was invoked Every Statement was executed Coverage Every Branch was traversed Coverage Every Control Path was traversed Coverage

  4. TEST TCOV-3 Levels of Code Coverage Method coverageInterface coverageStatement coverageBranch coveragePath coverage 1 2 3 4 5 Increasing Reliability Increasing Costs

  5. TEST TCOV-4 Method Coverage A_Retcode A_Method (Param_A1, Param_A2, Param_A3){ Invoke Method B_Retcode B_Method (Param_B1, Param_B2, Param_B3){ if (Param_B1 > Attr_B9) return 1;} Statement Coverage C_Retcode C_Method (Param_C1, Param_C2, Param_C3){ if (Param_C1 > Attr_C9) return 1 else continue; return 0;} Branch Coverage 1 Test 1 Test 2 Tests

  6. TEST TCOV-5 Method, Statement, Branch and Path Coverage F1 B1 E1 Method Coverage:(1 test case) Statement Coverage:(2 test cases) Branch Coverage:(3 test cases) Path Coverage:(5 test cases) B2 A2 Test Cases F1, B1, E1 (Method Coverage) F1, B1, E1 (Statement Coverage)F1, B1, B2, A2, B3, A3, A4, E2 F1, B1, E1 (Branch Coverage)F1, B1, B2, A2, B3, A3, A4, E2F1, B1, B2, B3, A4, E2 F1, B1, E1 (Path Coverage)F1, B1, B2, A2, B3, A3, A4, E2F1, B1, B2, A2, B3, A4, E2F1, B1, B2, B3, A3, A4, E2F1, B1, B2, B3, A4, E3 B3 A3 A4 E2

  7. TEST TCOV-6 White-Box Coverage Measures (Class Test) C1 = All Methods are executed at least once. (Method Coverage) C2 = All Statements of all Methods are executed at least once. (Statement Coverage) C3 = All Branches of all Methods are executed at least once. (Branch Coverage) D1 = All Attributes of an Object are altered. (Attribute Coverage) D2 = All Attributes of an Object take on all specified values. (Value Coverage) D3 = All Combinations of all values occur. (State Coverage) S1 = All Messages are sent at least once. (Interface Coverage) S2 = All Parameters of all Messages are altered. (Parameter Coverage) S3 = All Combinations of all Parameters occur. (Interface State Coverage)

  8. TEST TCOV-7 Data Coverage Measurement State Value Test Coverage = 100%could mean: Attribute Object Coverage Every Object was instantiated Every Attribute was altered Coverage Every relevant value was set Coverage Every possible state was generated. Coverage

  9. TEST TCOV-8 Levels of Data Coverage 1 Window/File/DB CoverageObject CoverageAttribute CoverageValue CoverageState Coverage 2 3 4 5 Increasing Reliability Increasing Costs

  10. TEST TCOV-9 Class Coverage Methods Attributes Object States Open_Account AccountNumberAccountOwnerAccountBalanceAccountStatusCreditLimit Account non existant Deposit_Money AccountBalance = 0 Withdraw_Money AccountBalance = plus Transfer_Money AccountBalance = minus, but < CreditLimit Close_Account AccountBalance = minus and > CreditLimit Account closed How many test cases are required to test all Methods with all Object states?

  11. TEST TCOV-10 Interface Coverage  Parameter Coverage = the Interface is generated one time with a repesentative value for each individual Parameter  Value Coverage = the Interface is generated as many times as there are representative values for each Parameter.  State Coverage = the Interface is generated as many times as there are combinations of representative values for all Parameters.

  12. TEST TCOV-11 State Coverage All Paths All Branches All State Transitions All States All Events Subsumption Hierarchy

  13. TEST TCOV-12 Data Table Coverage 1 OBJECT-A 2 ATTR-A1 'A' 'B' 'C' 2 ATTR-A2 100 200 300 400 2 ATTR-A3 'XYZ' 'ABC' 'EFG' 2 ATTR-A4 -1 0 +1 1 PANEL-B 2 FIELD-B1 50 100 150 2 FIELD-B2 100 -50 0 2 FIELD-B3 'XY' 'AB' 'EF' 2 FIELD-B4 'B' 'A' 'C' Value Coverage: Every relevant Value of every Attribute & Field should be set at least one time.(you need as many Objects as the longest array of values)

  14. TEST TCOV-13 Data Coverage Report S O F T E S T | DATA COVERAGE REPORT | | MODUL NAME: SVVL104 DATUM: 98.05.17 |+--------------------------------------------------------------------------------------------+| DATA NAME |RECORD NAME |BASE NAME|PREDIC.|INPUT|OUTPUT|NOT USED|+--------------------------------------------------------------------------------------------+| LISTNUMMER |FILLER |SVVL104 | | * | * | * || K-ENDE-DER-LISTE |ALLGEMEINE-VIS-KONSTANTEN|SVVL104 | | * | | * || CURRENT-DATE-MONAT |ALLGEMEINE-VIS-VARIABLEN |SVVL104 | | | * | || CURRENT-DATE-TAG |ALLGEMEINE-VIS-VARIABLEN |SVVL104 | | | * | || CURRENT-DATE-JAHR |ALLGEMEINE-VIS-VARIABLEN |SVVL104 | | | * | || SH-ENDE-DER-LISTE |ALLGEMEINE-VIS-SCHALTER |SVVL104 | * | | * | || SUMME-AWGR-VERK-MG |LISTSTEUERDATEN |SVVL104 | | | * | || FIRMA |ABSATZWEGGRUPPENSTATISTIK|SVVL104 | | * | * | || TEXT-ENDE-DER-AUSWERTUNG |ABSATZWEGGRUPPENSTATISTIK|SVVL104 | | | * | || FIRMA-GRUPPE |LISTPARAMETER |SVVL104 | | * | * | || LISTENNUMMER |LISTPARAMETER |SVVL104 | * | * | * | || FIRMA-GRUPPE |LISTPARAMETER |SVVL104 | | | * | || LISTENNUMMER |LISTPARAMETER |SVVL104 | | | * | |+--------------------------------------------------------------------------------------------+| || TOTAL NUMBER OF (NOT AUTOMATIC) DATA ITEMS REFERENCED = 13 || NUMBER OF (NOT AUTOMATIC) DATA ITMES USED = 11 || DATA COVERAGE RATION = 85% |

  15. TEST TCOV-14 Functional Coverage Measurement Function State Function Chain Function Outcome Function Coverage Coverage Coverage Coverage % of Application Functions

  16. TEST TCOV-15 Levels of Functional Coverage 1 Every application use case is executed at least once.Every application function is executed at least once.Every application function outcome is produced at least once.Every application function sequence is traversed at least once.Every possible combination of application functions is tested. 2 3 4 5 Increasing Reliability Increasing Costs

  17. TEST TCOV-16 Black-Box Test Coverage (Component Test) F1 = All specified Functions are tested at least once. F2 =All specified Functions are tested with all valid Arguments at least one time. F3 =All specified Functions are tested with all valid Arguments and all expected Results at least once. Results (n) = Function (Arguments (m)) Result (i) = Function (Argument (j)) i = 1 to n j = 1 to m

  18. TEST TCOV-17.1 Functional Test Coverage TRAN:In|OutART_NR:999Amount:999 TRANART_NRAmount TRANSART-NRMenge Use Case Coverage = > 1 Test Case One update transaction Function Coverage = > 2 Test Cases > one Increase> one Decrease Function Outcome Coverage = > 6 Test Cases > one Increase without Storno> one Increase with Storno of an existing order, generated earlier> one Increase with an error message> one Decrease without an order> one Decrease which generates an order> one Decrease with an error message

  19. TRANSART_NRAmount TEST TCOV-17.2 Function State Coverage = > 10 Test Cases > an Increase with invalid Art_Nr.> an increase with invalid amount> an increase to a non existing article> an increase with no storno> an increase with storno of existing order> a decrease with invalid Art_Nr. > a decrease with invalid amount> a decrease to a non existing article> a decrease without an order> a decrease which generates an order> an invalid transaction

  20. TEST TCOV-18 Predicted Error Coverage 500 Specification Errors 400 System Test 300 Design Errors N u m b e r o f E r r o r s 200 CodingErrors 100 Integration Test UnitTest 1 2 3 4 5 6 M o n t h s

  21. TEST TCOV-19 Inheriting Test Coverage Statement,Branch,Method & State Coverage Base Class Is tested first with all possible variations Derived Class Inherits test coverage ofthe base Class.

  22. TEST TCOV-20 Non Transitivity of Test Coverage Base Class Base Class Base Class SubClass X SubClass Y SubClass Z Environment X Environment Y Environment Z The Base Class must be retested in every new target Environment. According to Perry/Kaiser Study of 1990

  23. TEST TCOV-21 CRUD Object Access Coverage For each Object Type Create = 4 Test Cases e.g. Account Retrieve Update Delete For each Object State Create = 16 Test Cases e.g. plus Account Retrieve minus Account Update overdrawn Account Delete locked Account Testing the Creation, Retrieval, Update and Deletion of persistent Objects(relative to one, several or all Object Types and all Object States)

  24. TEST TCOV-22 CRUD Object Access Coverage Matrix Test Object A_Object B_Object C_Object Use Cases C R U D C R U D C R U D A_UseCase    B_UseCase   C_UseCase    D_UseCase    E_UseCase    N_UseCase    According to R. Binder, 1999

  25. TEST TCOV-23 Document Coverage Software Document Software Testing Level Class test Component test System test Requirements Documentation X User Documentation X Use case Documentation X X Application Object Model X X System Object Model X Object Interaction Model X Interface Specification X X Class Specification X Method Specification X

  26. TEST TCOV-24 Differences in Test Coverage Procedural Software Object-orientierted Software Unit = Procedural ModuleUnit-Testing = Module TestWhite-Box Test is in the foregroundData are shared (global)Functional DecompositionFunctional CouplingData and Call CouplingSub Program calls are stubbedEntity/Relationship-DiagramsData Flow DiagramsStructure DiagramsReuse is secondaryDevelopment is TOP-DOWNWaterfall Project Model Unit = Classe or ObjektUnit-Testing = Class TestGrey-Box Test is in the foregroundData are encapsulatedClass Inheritance and AssociationObject CouplingInheritance and Message CouplingForeign Methods are invokedObject Sequence DiagramsObject State Transition Diagrams Object Collaboration DiagramsReuse is primaryDevelopment is BOTTOM-UPIterative Project Model Differences between procedural and object-oriented Software

More Related