1 / 20

TorX

TorX. Automated Model Based Testing with Formal Methods. TorX : A utomated Model Based Testing with Formal Methods Contents. Model based testing Formal, model based testing with transition systems Transition systems testing and ioco A Tool for transition systems testing TorX

denisc
Download Presentation

TorX

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TorX Automated Model Based Testing with Formal Methods

  2. TorX : Automated Model Based Testingwith Formal MethodsContents • Model based testing • Formal, model based testing with transition systems • Transition systems testing and ioco • A Tool for transition systems testing • TorX • Current and future research • What does it mean for ARTIST2

  3. IUTconfmodel test tool test generation tool IUT passes tests  testcases TTCN TTCN exhaustive sound test execution tool passfail Automated Model Based Testing Automated Model Based Testing model IUT conf model IUT 

  4. Model Based Testing • Testing with respect to a (formal) model / specification • SDL, CSP, Lotos, Promela, UML, state diagrams, Spec#, . . . . • Precise, formal definition of correctness • good and unambiguous basis for testing • Formal validation of tests • Algorithmic derivation of tests • tools for automatic test generation • Maintenance of models / specifications, not of test suites • regression testing

  5. IUTconfmodel i ioco s s LTS model test tool test tool der : LTS (TTS) test generation tool i || der(s) pass IUTpasses tests IUT conf model i ioco s   exhaustive exhaustive sound sound i IOTS IUT t || i test execution tool passfail passfail Model Based Testingwith Transition Systems 

  6. s  LTS TsTTS IUT IMPS iIUTIOTS exec : TESTS  IMPS (OBS) passes : IOTS TTS {pass,fail} pass / fail Formal Testing with Transition Systems Test hypothesis : IUTIMP . iIUT IOTS . tTTS . IUT passes t  iIUT passes t der : LTS(TTS) ioco Proof soundness and exhaustiveness: iIOTS . ( tder(s) . i passes t ) iiocos

  7. pp =  !x LU{} . p!x Straces ( s ) = {   (L{})* | s } pafter = { p’ | p p’ } out ( P) = { !xLU | p!x, pP } { | pp, pP } Implementation Relationioco Correctness expressed by implementation relation ioco: iiocos =defStraces (s) : out (iafter )  out (safter)

  8. 1 end test case 3 observe output pass forbidden outputs allowed outputs ?y ?x  2 supply input fail fail T(S after !x) !a allowed outputs or : !xout(S) forbidden outputs or : !y out(S) T(S after ?a  ) Test Generation Algorithm Algorithm To generate a test case from transition system specification s0compute T(S), with S a set of states, and initially S = s0 after ; For T(S), apply the following recursively, non-deterministically:

  9. Validity of Test Generation For every test t generated with algorithm we have: • Soundness :twill never fail with correct implementationiiocos implies i passest • Exhaustiveness:each incorrect implementation can be detectedwith a generated testtiiocos implies t : ifailst

  10. user: manual automatic next input offer input IUT check output TorX observe output specification pass fail inconclusive A Tool for Transition Systems Testing: TorX • On-the-fly test generation and test execution • Implementation relation: ioco • Mainly applicable to reactive systems / state based systems; • specification languages: LOTOS, Promela, FSP, Automata

  11. explorer primer driver adapter IUT specificationtext statestransitions abstractactions abstractactions concreteactions IUT TorX specification spec. TorX Tool Architecture

  12. TorX

  13. Conference Protocol EasyLink TV-VCR protocol Cell Broadcast Centre component ‘’Rekeningrijden’’ Payment Box protocol V5.1 Access Network protocol Easy Mail Melder FTP Client “Oosterschelde” storm surge barrier-control DO/DG dose control Laser interface TorX Case Studies academic Philips LogicaCMG Interpay Lucent LogicaCMG academic LogicaCMG ASML/Tangram ASML/Tangram

  14. What has been Achieved ……  Sound and precise formal basis for model based testing • iocotest theory • proved test derivation algorithm Test tool TorX • prototype tool for model-basedformal testing • “is at least as good as conventional testing” • supports test generationand test execution • more, longer, and provably correct test cases  Applied successfully to different cases studies

  15. Extensions Status test case with data ?coin1  n: int  ? money ?coin2 ! money ? ?coin3 and action refinement [ n  35 ] -> [ n  50 ] -> ? button1 ? button2 ! button2 Vc := 0 c := 0 c := 0 Vt := 0 dVt/dt = 3 dVc/dt = 2 c < 10 c < 15  ? coffee [Vt= 15 ] -> [ c  5 ] -> [Vc= 10 ] -> ! coffee ! tea ? tea fail fail pass Testing Transition Systems: model and time and hybrid

  16. Current and Future ResearchTwente & Radboud • Testing real-time aspects • multi-channel real-time • Testing complicated data structures • transformational- + transition system based testing • Action refinement • when an abstract action is implemented as sequence of actions • What is a good test suite • test selection and test coverage • Test adapter and test interface • generic test environment • Compositionality and integration testing • differences diminish

  17. Current and Future ResearchTwente & Radboud • Hybrid testing • when continuous variables occur • Compositionality and integration testing • differences diminish • Testing stochastic and probabilistic properties • Multi-disciplinary • system testing • Relations between model checking, testing, static analysis, theorem proving, etc. • differences diminish • . . . . .

  18. Some Dutch Testing Projects • Côte de Resyste(1998 - 2002) - Conformance Testing of Reactive Systems: TorX Philips TU Eindhoven (LogicaCMG) (KPN)Lucent Uni. of Twente (Interpay) • Atomyste - ATOm splitting in eMbedded sYStem TEsting Uni. of Twente Radboud Uni. Nijmegen • Stress - Systematic Testing of Real-time Embedded Systems • Testing real-time properties Uni. of Twente • Testing data-intensive systems Radboud Uni. Nijmegen • Tangram - Model Based Testing and Diagnosis • Testing ASML Wafer Stepper machines - application oriented ASML, ESI, TUD, TUE, UT, RU, S&T, TNO

  19. ARTIST2 ActivitiesQuantitative Testing & Verification • Theory for testing embedded systems • real-time aspects • data aspects • extended conformance testing theories integrating a. and b. • test action refinement • Verification and scheduling • real-time schedulability analysis • optimal control synthesis • Verification of stochastic systems • model checking algorithms for CTMC, MDP • integration of performance analysis into verification • Tool-oriented research • data structures for real-time and stochastic modelling and analysis • test interfaces and test adapters • Application of testing and verification tools in industrial settings • collection of case studies • comparison • identification of links to industrial tools

  20. ARTIST2 ActivitiesQuantitative Testing & Verification • Theory for testing embedded systems • real-time aspects • data aspects • extended conformance testing theories integrating a. and b. • test action refinement • Verification and scheduling • real-time schedulability analysis • optimal control synthesis • Verification of stochastic systems • model checking algorithms for CTMC, MDP • integration of performance analysis into verification • Tool-oriented research • data structures for real-time and stochastic modelling and analysis • test interfaces and test adapters • Application of testing and verification tools in industrial settings • collection of case studies • comparison • identification of links to industrial tools

More Related