1 / 38

SATEL

SATEL. A Test Intention Language for Object-Oriented Specifications of Reactive Systems Levi L úcio Thesis’ defense Supervisor: Prof. Didier Buchs. Presentation Structure. Introduction State of the Art Problem Statement SATEL Discussion. Introduction Software Construction.

josette
Download Presentation

SATEL

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SATEL A Test Intention Language for Object-Oriented Specifications of Reactive Systems Levi Lúcio Thesis’ defense Supervisor: Prof. Didier Buchs

  2. Presentation Structure • Introduction • State of the Art • Problem Statement • SATEL • Discussion

  3. IntroductionSoftware Construction • Software Engineering is a discipline under intensive development • Large software projects present important challenges: • Operating systems • Banking systems • CERN detectors • Automotive / Aviation • Games Anatomy of a large project • 1M – 40M € • 20 – 100 engineers • 1 – 5 years • 5 – 10+ languages • Multiple teams • Multiple sites • Complex communication

  4. Understand the requirements Build the Model 1. 2. Build the implementation 3. Verify (e.g. test) 4. IntroductionThe spiral model (Boehm)

  5. IntroductionCatastrophic “bugs” • Numerical data conversion error (insufficient integration testing) • € 500M in hardware + € 7B in development In 1996 the Ariane 5 rocket explodes 37s after takeoff:

  6. IntroductionWhy do “bugs” exist? • The limits of Software Engineering are rapidly reached when building complex systems “The mythical man-month”, Brooks, 1975 • Building software is (still) a subjective activity: • Interpretation of the problem • Conception of the architecture of the system • Choice of the software and hardware tools • Coding • Quality measurement

  7. IntroductionOur Vision • Scientifically developed tools are needed • Strong formal theories induce better tools • Testing is partially automatable • Testing should be formal, too (inspired Gaudel) • The verification problem is undecidable • Heuristics (hypothesis) are needed to test • Qualification / Quantification of heuristics allows measuring quality

  8. Presentation Structure • Introduction • State of the Art • Problem Statement • SATEL • Discussion

  9. State of the ArtModel Based Testing (MBT) Artifacts • Model • SUT (observed) • Implementation relation • Test derivation function • Test cases (Set) • Oracle • Test Purposes

  10. State of the artReal / Formal Worlds Some minimal hypothesis are necessary to connect the real and the formal worlds!

  11. State of the ArtMBT theories

  12. H0 T0 H1 T1 Reduce Test set … Apply Hypothesis … Hk-1 Tk-1 Hk Tk H => (P╞ SP<=> P╞o TSP,H) State of the ArtBGM Exhaustive Test Set Reduction • (=>) soundness – TSPdoes not find bugs in correct programs • (<=) completeness – TSP detects all incorrect programs (P╞ SP) <=> (P╞o TSP) Keeping soundness and completeness depends on the hypothesis on the program

  13. State of the ArtRunning Example • ATM System with the operations • login(password) / logged, wrongPass, blocked • logout • withdraw(amount) / giveMoney(amount), notEnoughMoney • Following the second wrong login no more operations are allowed • The ATM distributes 20 or 100 CHF bills • Initial state • There are 100(CHF) in the account • ‘a’ is the right password, ‘b’ and ‘c’ are wrong passwords

  14. State of the ArtBGM and CO-OPN/2

  15. Presentation Structure • Introduction • State of the Art • Problem Statement • SATEL • Discussion

  16. Problem Statement • Adapt the previous theoretical work on BGM and CO-OPN/2 to test engineering • Build tools that can assist the test engineer in an industrial context

  17. Presentation Structure • Introduction • State of the Art • Problem Statement • SATEL (Semi-Automatic TEsting Language) • Test Intentions • SATEL semantics • CO-OPN/2c++ • Discussion

  18. loginLogout < 4x 1withdraw reachBlocked SATELWhat are Test Intentions? A test intention defines both a subset of the SUT behavior and hypothesis about how to test it

  19. SATELTest Intentions and BGM Inspired from BGM and the work of Péraire, but we produce and reduce an exhaustive test set per test intention

  20. Base case for the recursion (empty test intention) Recursive definition Test intention reuse Regularity over execution path SATEL Recursive Test Intentions and Composition • One test intention is defined by a set of axioms: Variables f : HML T in loginLogout; f in loginLogout => f . HML({login(a) with logged>} {logout} T) in loginLogout; • Test intentions may be composed: f in loginLogout & nbEvents(f) < 4 => f in 4LessLoginLogout HML(T), true HML({login(a) with logged} {logout} T), true HML({login(a) with logged} {logout} {login(a) with logged} {logout} T), true HML({login(a) with logged} {logout} {login(a) with logged} {logout} {login(a) with logged} {logout} T), true

  21. Variables pass : password Axioms Uniformity predicate uniformity(pass) => HML({login(pass) with wrongPass}{login(pass) with blocked} T) in reachBlocked; SATELUniformity HML({login(b) with wrongPass} {login(b) with blocked} T), true HML({login(a) with wrongPass} T), false HML({login(b) with wrongPass} {login(a) with blocked} T), false

  22. Variables obs : primitiveObservation am : natural Axioms subUniformity predicate regularity predicate (am <=140) = true, subUniformity(am) => HML({login(a) with logged} ({withdraw(am) with obs} T) in 1withdraw; SATELRegularity andsubUniformity HML({login(a) with logged} {withdraw(120) with notEnoughMoney T), true HML({login(a) with logged} {withdraw(80) with giveMoney(80)} T), true

  23. SATELTest Intention Definition Mechanisms

  24. Presentation Structure • Introduction • State of the Art • Problem Statement • SATEL (Semi-Automatic TEsting Language) • Test Intentions • SATEL semantics • CO-OPN/2c++ • Discussion

  25. SATEL SemanticsSemantics of a test intention • Test intention unfolding (solve recursion) • Calculate the exhaustive test set • Replace all variables exhaustively • Generate oracles by validating against the model • Positive tests are annotated with true • Negative tests extracted from test intentions but having an impossible last event, annotated with false • Reduce the exhaustive test set by solving all predicates on the variables

  26. SATEL SemanticsAnnotations • Annotations correspond the conditions in the model allowing a state transition • In CO-OPN an annotation is a conjunction of conditions reflecting the hierarchy and dynamicity of the model

  27. Variables obs : primitiveObservation pass : password (AADT) subUniformity(pass) => HML(<login(pass) with obs> in login; SATEL SemanticsEquivalence Class Calculation C1: correct password C2: wrong password HML({login(a) with logged} T), true HML({login(b) with wrongPass} T), true HML({login(a) with logged} T), true HML({login(c) with wrongPass} T), true

  28. Variables path : primitiveHml subUniformity(path), nbEvents(path)<4 => path in allUnder4; SATEL SemanticsEquivalence Class Calculation (cont) C1: correct password C2: wrong password C3: one wrong login C4: true C5: not enough money C6: enough money

  29. SATEL SemanticsMain Results • Complete formal abstract syntax and semantics • Exhaustive test set for a test intention • Formal proof of the capability of expressing a test intention for generating a test set that can establish the correctness of an implementation • Equivalence class calculation by transition system annotation, independent of operationalunfolding • Reduction of both positive and negative test sets • Oracles for test intentions having simultaneous inputs

  30. Presentation Structure • Introduction • State of the Art • Problem Statement • SATEL (Semi-Automatic TEsting Language) • Test Intentions • SATEL semantics • CO-OPN/2c++ • Discussion

  31.  (object sync condition, object id) pairs CO-OPN/2c++ATM Model annotation = context sync conditions

  32. CO-OPN/2c++Main Results • New version CO-OPN/2c++ with abstract formal syntax and semantics • Revision of the abstract syntax regarding context modules • Direct semantics rather than transformational • Primitive treatment of gates used as observations in testing • Primitive treatment of contexts allowing hierarchy and modularity in testing • Clear semantics allowing annotation generation for the transition systems used as oracles

  33. Presentation Structure • Introduction • State of the Art • Problem Statement • SATEL • Discussion

  34. Discussion • Operational experiments with modified Prolog pointed the need for a better formalization • We have developed a theory and a language for test engineering in the context of CO-OPN • Syntactically and semantically well founded • SATEL’s semantics is generic and can be applied to other event-based specification languages

  35. Discussion (cont.) • Developed an IDE for SATEL’s concrete syntax integrated with CoopnBuilder • A case study with an industrial partner (CTI) allowed beginning to identify methodological capabilities of SATEL

  36. DiscussionFuture Work • Formalization of decomposition hypothesis • Quantification of coverage • Implementation of the theoretical model using the DD / SDD approach • Model-checking techniques

  37. DiscussionOutcome • Book Chapters: • Levi Lúcio, Marko Samer: “Technology of Test-Case Generation” Model-Based Testing of Reactive Systems 2004. Published by LNCS • Didier Buchs, Luis Pedro, Levi Lúcio: “Formal Test Generation from UML Models” Research Results of the DICS Program 2006. Published by LNCS • Conferences and Workshops: • Levi Lúcio, Luis Pedro, Didier Buchs: “A Methodology and a Framework for Model-Based Testing” RISE 2004. Published by LNCS • Levi Lúcio, Luis Pedro, Didier Buchs: “A Test Language for CO-OPN Specifications” IEEE International Workshop on Rapid System Prototyping 2005. Proceedings • Levi Lúcio, Luis Pedro, Didier Buchs: “Semi-Automatic Test Case Generation from CO-OPN Specifications” MTOOS 2006. Microsoft Research Technical Report • Talks • Several talks in the context of the past and present projects • Invited keynote talk at the RISE workshop, Geneva, September 2006 • Invited talk at the ETH Zurich, January 2006 • Representation of the SMV group in an expert evaluation of the department, Geneva, October 2006 • Invited talk (speaker Didier Buchs) at the SQS (Software Quality Conferences), Geneva, May 2008

  38. DiscussionOutcome • Past projects: • VALID: Interreg, France/Swiss regional development. With Leirios technologies and CTI (Centre des Technologies de l’Information de l’état de Genève) • VeDISS: DICS initiative, Hasler Foundation • Present and submitted projects • COMEDIA: ManCom initiative, Hasler Foundation • TEST_INDUS: Interreg, France/Swiss regional development. With Leirios technologies and CLIO (industial partner) • Inside and outside the SMV group • Collaboration with Steve Hostettler in the direction of finding efficient operational techniques for SATEL’s semantics • Collaboration with Andrei Berlizev in order to automate testing for fault tolerant system expressed in the CORA language • Perspectives for the application of SATEL for testing in the context of DSLs provided by Ang Chen, Luis Pedro, Matteo Risoldi and Vasco Amaral’s group in Lisbon

More Related