1 / 45

An Automated Test Strategy Based on UML Diagrams

PISATEL. An Automated Test Strategy Based on UML Diagrams. F. Basanieri *, A. Bertolino *, E. Marchetti *, A. Ribolini *, G. Lombardi **, G. Nucera ** *PISATEL LAB, IEI-CNR, Pisa, Italy ** ERICSSON LAB ITALY. Agenda. Cow_Suite tool New features Traceability Cow_Suite vs RUP Guidelines

linda-brady
Download Presentation

An Automated Test Strategy Based on UML Diagrams

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PISATEL An Automated Test Strategy Based on UML Diagrams F. Basanieri *, A. Bertolino *, E. Marchetti *, A. Ribolini *, G. Lombardi **, G. Nucera ***PISATEL LAB, IEI-CNR, Pisa, Italy** ERICSSON LAB ITALY

  2. Agenda • Cow_Suite tool • New features • Traceability • Cow_Suite vs RUP • Guidelines • Discussion and future work

  3. PISATEL Objectives • Use for test planning the same UML diagrams developed (e.g., by Rational Rose) for design and analysis • Tool-supported generation and maintenance of an updated and strategic test plan, as the design evolves.

  4. PISATEL • Cow_Suiteconsists of two main components: • UIT (Use Interaction Test) and • CoWTeSt (Cost Weighted Test Strategy) Cow_Suite =Cowtest pluSUITEnvironment Cow_Suite approach • The processing of the Cow_Suite approach starts by analysing the project description*, and proceeds in parallel with the design and analysis refinement steps. • The reference documentation is derived by collecting together and automatically organizing the diagrams developed during the design phase without additional manual effort of formalization [*: the internal description of the project model, given by Rational Rose in a .mdl file]

  5. PISATEL Selection Strategy Test case Test case Cow_Suite Test case Test case Test case Test case Test case Test case SDs selection according to the strategy chosen Test case Test case Interaction with the user for deriving executable test procedures Test case Test case Test case Test case Test case Automatic UIT application for Test cases derivation Test case Obj 1 Obj 2 Obj 3 Method1() U I T Method2() Test specification Method3() Cow_SuiteScheme Test Procedures list

  6. PISATEL Hierarchical Tree Tree Derivation Use Cases (UCs) and Sequence Diagrams (SDs) of the .mdl project design realized with Rational Rose are automatically organized by Cow_Suite tool in a sort of hierarchical (pseudo-)tree The case study is Course Registration System from RUP documentation

  7. PISATEL Hierarchical Tree Tree Derivation The users can get a complete view of the statusof the functionalities specification and a detailed documentation: • UCs and their realization • SDs associated to each specification level • The links between Use Case and Design Model • Reused nodes • Not linked model elements

  8. PISATEL Use Case View vs Logical View The Main Tree can be analyzed from two different points of view which correspond to two different typologies of test. • Using the information of the Use Case View • System test • Subsystem test • Using the information of the Logical View • Subsystem test • Component test

  9. PISATEL Use Case View describes the set of UCs (and scenarios) that represents the significant functionalities and the interaction of the system with the external world. • The Main Tree derived represents a detailed documentation of how features/system requirement are realized in the Use Case View. • The path from a high level Use Case to a leaf (UCs or SDs) describes the refinement of the associated system feature • The Test Cases derived from the SDs of the Use Case View are specifically designed for • System integration Test • Subsystem integration Test

  10. PISATEL Logical View Describes how the system functionalities are provided in term of static structure and dynamic collaborations of the objects. • Cow_Suite tool analyses the Logical View information about: • Use Cases realization • Sequence diagrams, classes and their organization in packages and subsystems • The tool automatically derives: • The UC traceability by analyzing the package Use Case Realizations. • The Design Tree: derived by analyzing the Design Model packages

  11. PISATEL Use Case Diagram “Traceabilities” • The “Use Case Realizations” package links the information of the Use Case View and the Logical View . • The Use Case Diagram “Traceabilities” traces the evolution of the Use Cases in the Design Model • Each TraceabilityUse Case • Represents the evolution of a UC of the Use Case View into the Design Model • Details the sub-flow of a UC with Sequence (Collaboration) Diagrams • Specifies the Classes and the relationship that participate in the realization of the UC (Class Diagrams) • Optionally holds the package Design Link

  12. PISATEL Use Case Realization analysis • The Main Tree derived from the Use Case View is automatically extended with the information of the Use Case Realizations Package • The path from a high level Use Case, through its Use Case Realization, to a leaf documents and traces the implementation of a system feature • Considering the subtree rooted in a Traceability Use Case, its SDs: • contain low level information with respect to those of the Use Case View • describe in a detailed manner the collaboration of the involved objects • The Test Cases derived from these SDs are specifically designed for • Subsystem integration Test • Component integration Test

  13. PISATEL Design Link Package • Design Link package:for every use case realization it collects the package/s that represent the implementation of the relative UC. It is an explicit link between the Main Tree and the Design tree

  14. PISATEL Design Tree Design Tree: is derived directly from the Design Model and represents the organization of the packages in a hierarchy. Analyzing this tree it is possible to: • Observe the dependence relationship between packages • Know which are the reused packages and where the reuse happens • Connect to the main tree the low level SDs not explicitly linked in the .mdl file By selecting the SDs of the Design Tree and applying to them the Cow_Suite test derivation strategies, a list of low level Test Cases can be derived

  15. PISATEL Deduce the Critical Profile • The effort devoted to the test of different functionalities must be planned and scheduled considering their relative “importance” (risk-based testing, reliability guided testing, etc…). • The importance, represented by number (weight) belonging to [0,1] interval, can be assigned in two different manner: • Interacting with Cow_Suite Tool: Following level by level the tree structure of the UCs and SDs, the user annotates the corresponding node in the tree with its weights • Directly in the .mdl file (UCs only):During the development of the Use Case Model architects/designers assign a value, belonging to [1,9] interval, to each Use Case. The value is memorized in the use case documentationtabwith the following structure: WGH = value (For SDs or when no value is expressed for UCs the default value is 5)

  16. PISATEL Weighted tree derivation The tool : • reads the assigned values the .mdl file (if they exist) • sums the values of the node at the same level in the tree • normalizes the weights among the nodes at the same level so that their sum is 1. The user can always modify the assigned weights

  17. PISATEL Integration Stage selection The Cow_Suite user selects the integration stage at which the test is conducted. The nodes that are considered by Cow_Suite tool for the subsequent evaluations are all and only those belonging to the selectedintegration stage plus all the tree leaves existing at higher levels.

  18. PISATEL NT Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Cow_Test Strategy Two different situations are considered in test cases planning: 1: Cowtest_ing with fixed number of tests • A certain budget is available, which is translated in terms of number of tests, NT, to execute. • In this case the tool select the most adequate selection of NT tests from among the (many) various test cases that could be potentially conceived.

  19. PISATEL 10 2 2 Minimum Number of test to execute = 16 1 1 Cow_Test Strategy 2: Cowtest_ing with fixed functional coverage • A certain percentage of functionalities must be covered. • In this case by the tool is possible to define in advance which are the functionalities to be covered and the minimum number of tests to execute. Functional Coverage = 90%

  20. PISATEL Cowtest_ing with fixed functional coverage. The most critical system functionalities are selected for reaching the fixed coverage. Test Procedures are distributed accordingly. Cowtest_ing with fixed number of tests. The selection of the most suitable distribution of test procedures is developed on the basis of leaves weights. Select the Test Strategy

  21. PISATEL SDs selection N. Test Procedures Final weight Sequence Diagrams Selection For each fixed integration stage, a weighted subtree is derived according to the chosen test criterion. Only the relative SDs are considered for Test Case generation.

  22. PISATEL SDs Selection Test Case Derivation The tool uses the UIT method,to derive the Test Cases. The method is mainly based on the SDs (particularly objects, messages and parameters)

  23. PISATEL Sequence Diagram analysis • Test Units:horizontal axis analysis. • Interactions CategoriesandSettings Categories:vertical axis analysis. • MessageSequences: set of Messages (in temporal order), involved in a SD, used by objects to define and elaborate specific functionalities.

  24. PISATEL Number of associated Test Procedures Detailed Test Case Description Test Case Derivation For each traceable Message Sequence a Test Caseis generated. It contains the list of all its Settings and Interactions Categories and their values. The final weight of every SDs is used to automatically derive the number of Test Procedures associated to each Test Case.

  25. PISATEL The user interacts with the tool for inserting Choices values. Test Procedures Specification and derivation Test Specification For each identified category, we consider all its possible values and constraints (“choices”). A Test Procedure is automatically generated from a Test Specification, for each possible combination of choices of every category involved in a Messages Sequence.

  26. PISATEL Cow_Suite vs RUP • Guidelines for using Cow_Suite inside RUP • Best Practices forCow_Suite application • RUP is based on an iterative process characterized by four phases divided in workflows. • A workflow is a set of activities that produces a observable result (artifacts). The involved workflows are: • Project Management • Requirement • Analysis and Design • Test

  27. PISATEL Inception Phase During the inception phase: • The scope of the project and the external entities with which the system will interact (actors) are defined. • The use cases describing the most significant interaction are also identified • Workflow Requirement • Analyse the Problem • Understand Stakeholder Needs • Define the System • Manage the scope of the System • Refine the System Definition • Manage Changing Requirements • Classify Requirements • Workflow Analysis and Design • Perform Architectural Synthesis • Evaluate Use Case • Derive Cow_tree • Workflow Test • Test Plan • Testing Cost Estimation

  28. PISATEL Elaboration Phase • Workflow Project Management • Plan for Next Iteration • Manage Iteration • Monitor ad Control Project • Analyse Cow_tree • Reevaluate Project Scope and Risk • Workflow Requirement • Analyse the Problem • Understand Stakeholder Needs • Define the System • Manage the scope of the System • Refine the System Definition • Manage Changing Requirements • Classify Requirements • Workflow Analysis and Design • Define a Candidate Architecture • Process UCs and SDs • Derive Cow_tree • Analyse Behaviour • Implement Traceability • Evaluate Traceability • Design component/design for Real-Time • Cow design tree • Design database • Workflow Test • Testing Cost Estimation • Plan test • Derive UIT Test • Test Traceability • Design test • Implement test • Execute Test in Integration Test Stage • Execute Test in System Test Stage • Evaluate Test During the inception phase: • The analysis of the problem domain and the basic architecture are established • The project plan is developed and the possible risk elements are evaluated • A prototype is built, and the use case model is almost complete

  29. PISATEL Construction Phase • Workflow Project Management • Plan for Next Iteration • Manage Iteration • Monitor ad Control Project • Analyse Cow_tree • Reevaluate Project Scope and Risk • Workflow Requirement • Manage Changing Requirements • Classify Requirements • Workflow Analysis and Design • Define a Candidate Architecture • Process UCs and SDs • Derive Cow_tree • Analyse Behaviour • Implement Traceability • Evaluate Traceability • Design component/design for Real-Time • Cow design tree • Design database • Workflow Test • Testing Cost Estimation • Plan test • Derive UIT Test • Test Traceability • Design test • Implement test • Execute Test in Integration Test Stage • Execute Test in System Test Stage • Evaluate Test • Includes the development of the system remaining components and features and the test of the entire system.

  30. PISATEL Inception Phase – Workflow Requirement Classify Requirements • The requirements must be classified and grouped depending on: • criticality • system importance • difficulty in the development • Optional: a number in [1,9] interval representing this importance value should be assigned to each requirement/requirement set.

  31. PISATEL Inception Phase – Workflow Analysis and Design Evaluate Use Case • Basing on the requirement classification an importance value ( [1,9] interval) must be assigned to each developed UC. • The importance is an absolute value characterizing the UC • The value is memorized in the use case documentationtabwith the following structure: WGH = value

  32. PISATEL Inception Phase – Workflow Analysis and Design Derive Cow_Tree • Apply the Cow_Suite and derive the main tree. • Possible design weaknesses and incompleteness are displayed. • The user profile is derived • The most important UCs, i.e. system functionalities, are highlighted

  33. PISATEL Inception Phase – Workflow Test Testing Cost Estimation Applying Cow_Suite test strategies it is possible to : • Distribute the Test Cases among the system functionalities • Know how many Test Cases are necessary to cover the developed system functionalities • Derive the completion time or “cost” (in terms of man hours/required budget) of the Test Phase by assigning a value to each typology of Test Case

  34. PISATEL Elaboration Phase – Workflow Project ManagementAnalyse Cow_tree Apply the Cow_Suite and derive the main tree that: • Evidences the degree of detail reached in the requirement development. • Highlights the design weaknesses in the Use Case Model/Design Model developed. • The main tree integrates the information derived from the Use Case View with those of the Logical view and can be used for the completion of the Sw Architecture Document. • Marks the most important functionalities of the developed system • The final weight associated to each node represent the importance of the implemented functionalities for the system. • The correctness of the values assigned must be checked and eventually modified

  35. PISATEL Elaboration Phase – Workflow RequirementClassifying Requirements Software architects and Program managers: • highlight the most critical UCs • select the UCs that will be refined • They assign or modify the importance value associated to the UCs during the Inception Phase.

  36. PISATEL Elaboration Phase – Workflow Analysis and Design Process UCs and SDs • Basing on the requirement classification an importance value ( [1,9] interval) must be assigned to each developed UC. • The value is memorized in the use case documentationtabwith the following structure: WGH = value • Optional: Insert the default values in the SDs. They represent the values of the Interaction Categories of the UIT method. • The values are memorized in the message/class method documentationtabwith the following structure: DEF (value1, value2…)

  37. PISATEL Elaboration Phase – Workflow Analysis and Design Derive Cow_tree • Apply the Cow_Suite and derive the Main Tree. • An extension/modification of the main tree corresponds to every refinement of the Use Case/Design Model. • The evolution of the system requirement (Use case View), into diagrams of the Logical View is highlighted • The missing links between the two views and the diagram not properly connected are shown and evidenced.

  38. PISATEL Elaboration Phase – Workflow Analysis and Design Implement Traceability • Associate to every Traceability Use Case a value representing the criticality in the realization of the associate UC, and annotate in the UC documentation tab this number. • Link to every Traceability Use Case the SDs representing the most important scenarios and the interaction between the involved classes. In this manner every requirement will be automatically associated to the proper set of Test Cases • Optional: create a class diagram, called the Design Link, with the list of the packages in the Design Model that represent the implementation of the UC

  39. PISATEL Elaboration Phase – Workflow Analysis and Design Evaluate Traceability • Apply the Cow_Suite and derive the Main tree and the Design Tree • The Design Link evidences theevolution of the UCs in the Design model: • The Design Tree represents the degree of detail reached in the requirements (UCs) implementation • The objects or packages involved in the requirement implementation are organized in a hierarchical tree using packages dependencies. • The weights associated to the Traceability Use Cases are analyzed and possibly modified

  40. PISATEL Elaboration Phase – Workflow Analysis and Design Cow Design Tree The Design Tree: • shows the hierarchy between the different system components, the subcomponents, the reused components and the associated SDs • Represents a detailed documentation which associates to each requirement the relative system components. • following the path in the main Tree from an high UC, throw Use Case realization, to the Design Link node. Then automatically jump to the proper position in the Design tree and cover it down till the leaves.

  41. PISATEL Elaboration Phase– Workflow Test Testing Cost Estimation • Considering the main tree: • Evaluate the correctness of the assigned weights • Possibly modify the values • Applying Cow_Suite test strategies it is possible to : • Distribute the Test Cases among the system functionalities • Know how many test are necessary to cover the developed system functionalities • Derive the completion time or “cost” (in terms of man hours/required budget) of the Test Phase by assigning a value to each typology of Test Case

  42. PISATEL Elaboration Phase– Workflow Test Derive UIT Test • Apply the UIT method. • The Test Cases derived from the main tree can be used for systemorsubsystem orcomponent test depending on the integration level chosen. • The Test Cases derived from the Design tree can be used for component/unit test. • The Test Procedures, as derived by the tool, are not directly executable ( Test Scripts). The derived Test Procedures have to be executed and logged by a test driver. • The degree of detail of the Test Set depends directly on, the design detail level and granularity of the inserted values (choices of the UIT categories).

  43. PISATEL Elaboration Phase– Workflow Test Test Traceability Considering the main tree and the Design tree: • the SDs on the path from an high level UC to a leaf (of the main tree or Design tree) produces Test Cases specialized for the testing and the verification of the requirement (system functionalities) associated to the UC. • The degree of detail of the derived Test Cases depend on hierarchical position of the SDs in the tree (the higher is the level, the lower is the degree of detail)

  44. PISATEL Discussion and Future Work Systematic and automated support for test planning and test case generation, based on the same UML diagrams developed for design and analysis OK Tool integrated with the design tool (e.g. Rational Rose) OK Test case derivation (UIT) to be validated on ERI case studies To be continued FUTURE OF COW_SUITE ?!?

  45. PISATEL Questions?

More Related