A Test Selection Language for CO-OPN Specifications

1. A Test Selection Language for CO-OPN Specifications Levi Lúcio, Luis Pedro and Didier Buchs University of Geneva

2. Presentation Plan • The CO-OPN Modeling Language • CO-OPN as a prototyping tool • Testing Theory • Stating hypotheses about the SUT for test selection • A Test Selection Language for CO-OPN Specifications • Language Structure • Hypotheses • On Paths • On Values • Conclusion

3. Presentation Plan • The CO-OPN Modeling Language • CO-OPN as a prototyping tool • Testing Theory • Stating hypotheses about the SUT for test selection • A Test Selection Language for CO-OPN Specifications • Language Structure • Hypotheses • On Paths • On Values • Conclusion

4. The CO-OPN Modeling Language • CO-OPN (Concurrent Object-Oriented Petri Nets) is a formal specification language, with clearly stated semantics. Based on: • Algebraic Data Types: data and associated operations • Petri Nets: behavior • It includes object based concepts: • Class • Attributes: places in a Petri Net • Methods: guarded transitions in a Petri Net • Powerful synchronization mechanism • Coordination of the components of a specification • Objects • Contexts (stateless coordination units) • “All or nothing” semantics (sequence / parallel / choice)

5. Structure of a CO-OPN Specification Contexts Coordinate Coordinate (instances of) Classes Coordinate Use Algebraic Data Types

6. Example specification in CO-OPN • A (simple) mobile phone is composed by: • A card containing (among other things) the personal identification number (PIN) to activate the phone; • A phone controller holding (among other things) the state of the phone, which can be: • Off • Standby (waiting for the correct PIN) • Idle (ready to receive or make calls) • In our specification the user can perform two actions: • Turn on the phone; • Insert a PIN number

7. Outer ContextCoordinates the Phone Controller and the Card method (input event) gate(output event)

8. Class PhoneController Algebraic Data Types Typed values (+ operations) inside the Petri Net places…

9. CO-OPN as a Prototyping Tool • Our tool (CoopnBuilder) is able to produce Java code from a CO-OPN specification. We can then execute the specification…

10. SUT (System Under Test) Verification • We can “run” the specification to make sure the prototype will act as expected, but… • What if the prototype is changed / extended, or simply implemented in a language other than Java? • There is no way of keeping a record of previously tested behaviors (for example for regression testing…) • Solution: devise a testing language to include test modules in CO-OPN specifications: • The test module will use the specification to generate syntactically and semantically meaningful tests; • Prototype execution provides an easy way of deciding what should be the result of a given test case when it is applied to the SUT (System Under Test).

11. Presentation Plan • The CO-OPN Modeling Language • CO-OPN as a prototyping tool • Testing Theory • Stating hypotheses about the SUT for test selection • A Test Selection Language for CO-OPN Specifications • Language Structure • Hypotheses • On Paths • On Values • Conclusion

12. Presentation Plan • The CO-OPN Modeling Language • CO-OPN as a prototyping tool • Testing Theory • Stating hypotheses about the SUT for test selection • A Test Selection Language for CO-OPN Specifications • Language Structure • Hypotheses • On Paths • On Values • Conclusion

13. T0 H0 Reduce test set Increase hypothesis strength T1 H1 … … Tk-1 Hk-1 Tk Hk Testing Theory • Problem: How do we select “interesting” tests? • In the general case, the amount of tests for a given system is infinite: • Infinite domains for operation’s domains; • Infinite paths of execution in the SUT’s state space. • It is necessary to restrict the “exhaustive” test set to a practicable one. This is done by providing hypotheses about the functioning of the SUT:

14. Testing Process Does P satisfy SP? P SP Test Selection (Hypotheses H Application) Test Set T Test Procedure Execution of P using T Correction of P Oracle P satisfies, or not, T! no inconclusive yes P satisfies, or not, H no yes P does not satisfy SP Undefined P satisfies SP!

15. Presentation Plan • The CO-OPN Modeling Language • CO-OPN as a prototyping tool • Testing Theory • Stating hypotheses about the SUT for test selection • A Test Selection Language for CO-OPN Specifications • Language Structure • Hypotheses • On Paths • On Values • Conclusion

16. Presentation Plan • The CO-OPN Modeling Language • CO-OPN as a prototyping tool • Testing Theory • Stating hypotheses about the SUT for test selection • A Test Selection Language for CO-OPN Specifications • Language Structure • Hypotheses • On Paths • On Values • Conclusion

17. Test Language Structure Hypotheses constraints over the exhaustive test set HML (Hennessy-Milner Logic) next <_> operator not, and operators CO-OPN event pairs input event (method), output event (gate) (black-box approach)

18. Mobile Phone Example Revisited • Let us create a test module for an extended version of the Mobile phone CO-OPN specification: • A (simple) mobile phone is composed by: • A card containing (among other things) the personal identification number (PIN) to activate the phone. The card gets blocked after 3 wrong attempts at inserting a PIN. • A phone controller holding (among other things) the state of the phone, which can be: • Off • Standby (waiting for the correct PIN) • Idle (ready to receive or make calls) • In our specification the user can perform two actions: • Turn on the phone; • Insert a PIN number, • Reset the card when it becomes blocked

19. Exported constraints Locally defined constraints Imported Algebraic Data Types Variables used in constraint definition Constraints’ Semantics Test Module for the Mobile Phone ConstraintSet NatelBehaviors; Interface Constraints insertPins; block; reachOn; Body Constraints nWrongPins; Use Boolean; Pin; Axiom [] in nWrongPins; f in nWrongPins => f . HML(<insertPin(newPin(1 1 1 1)),pinResult(false)>) in nWrongPins; f in nWrongPins, depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<m,g>) in insertPins; depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<resetCard,null>) in block; depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<insertPin(newPin(1 2 3 4)),phoneReady>) in reachOn; Variables f : HML; m : method; g : gate; End NatelBehaviors;

20. Hypotheses on Paths • The language allows defining the shape of sequences of input/output pairs that will be used to generate tests: [] in nWrongPins; f in nWrongPins => f . HML(<insertPin(newPin(1 1 1 1)),pinResult(false)>) in nWrongPins; f in nWrongPins, depth(f) <= 3 => HML(<turnOn,null>)} . f . HML(<m,g>) in insertPins; • nWrongPins (local constraint) defines an (infinite) sequence of insertPin inputs with a wrong PIN; • insertPins defines a sequence of operations starting by turnOn, followed by a sequence of at most three insertPin inputs with a wrong PIN and finishing by any input/output pair. Examples of tests generated by the insertPin constraint: HML(<turnOn,null>,<insertPin(newPin(1 1 1 1)),pinResult(false)>,<resetCard,null>),false HML(<turnOn,null>,<insertPin(newPin(1 1 1 1)),pinResult(false)>, <insertPin(newPin(1 1 1 1)), pinResult(false)>,<insertPin(newPin(1 1 1 1),pinResult(false)>,<resetCard,null>),true HML(<turnOn,null>,<insertPin(newPin(1 2 3 4)),pinResult(true)>,true ...

21. Hypotheses on Values • By default, the tests generated by the pinTest will correspond to all values of type Pin - large amount • Solution 1: Axioms HML(<turnOn,null>,<insertPin(p), g>) in pinTest; Variables p : Pin; g : gate; uniformity(p) => HML(<turnOn,null>,<insertPin(p), g>) in pinTest • Choose one value from the domain - equivalent to random testing… • Corresponds to the hypotheses: “if insertPin works well for one PIN, it will work well for all PINs”.

22. P is the good PIN => Send pinResult(true) insertPin(p) P is a wrong PIN => Send pinResult(false) Hypotheses on Values (2) • Solution 2: subUniformity(p) => HML(<turnOn,null>,<insertPin(p), g>) in pinTest • Choose one value from each equivalence class of the behavior of the insertPin operation. • In order to decompose the behavior of an operation we need to analyze its choice points… • This is done by analyzing the CO-OPN specification. • Feature under development, using Prolog for symbolic execution and constraint resolution.

23. Managing Non-Determinism in the SUT • We would like to manage behaviors of the SUT which cannot be calculated in test generation time… • Example: an e-banking application proposes a challenge as part of the authentication mechanism. How do we calculate the reply to the challenge in order to login successfully during the test? • Solution: Introduce the possibility to have “external” functions that will perform calculations in testing time. • In the e-banking case there would be a function to calculate the answer to the challenge in function of the proposed value.

24. Conclusion • We have presented a test selection language intended at testing software (and possibly hardware) prototypes specified in CO-OPN; • The test selection is done by the test engineer, assisted by the tool while decomposing the behavior of the operations in a specification; • The tool is currently half implemented. We envisage on the short term a case study to fully validate our ideas, also in terms of scalability.