Process Modeling Tool Example: Unit Tests

Oscar Slotosch, Validas AG Process Modeling ToolExample: Unit Tests

Goals • Provide an (short overview on PMT) • First steps with the tool • Show how to model an (realistic) example • Requirements • Process • Compliance • Verification & Validation • Tutorial

Contents • Overview PMT • Tool Basics: Installation, Model Creation • Requirements Modeling • Automated Tailoring • Process Modeling • Process Validation • Advanced Modeling Aspects: Parameters, Meta Models,.. • Compliance Modeling • Verification & Validation • Report Generation • Summary

PMT Overview • PMT = Process ModelingTool (In future also Project Management Tool?) • Main goals • Define processes • Argue & show compliance • Use process model for • Product verification • Process documentation • Process consistency & validation • Project Planning • Project Management • Model-based processes • Reuse process modules • Variant handing & tailoring • Graphical visualization • Report Generation • Developed in research project SPEDIT (BMBF) • Successfully used for TÜV certification of Validas qualification processes

Process Modeling Tool PMT Model-Based Process Link Requirementsto Processes and Verification Actions Formalize Tailor Parameters QKit PMT ISO 26262, IEC 61508, .. Formalize ParameterValues Generate Generate Generate VVT ProcessDescription VerificationValidationPlan VerificationValidationReport Project: V&V Compliance Report Safety Case Safety Plan

How to Model Processes?PMT Metamodel Overview Process Requirement References Compliance StakeHolder Artifact Input ProcessModule Output Parameter References Verifies Before / After References Binding Model VerifcationModule Criterion

Graphical Process Notation Elements • Process Module: blue, rounded box: • Verification Module: green, rounded box: • Hierarchical Process Module: blue folders: • Hierarchical Verification Module: green folders: • Artifact: Grey box with note: • Model: Orange box with note: • StakeHolder: transparent box: • Conditions: yellow routes: Relations • Before After: solid arrow: • Read/Write: dashed arrow: • Artifact Containment: dotted arrow: Default • Start & End

Installation & Start • Preparation • Java Runtime Environment (JRE) Version 8 installed • Graphviz 2.38 installed, e.g. from https://www.graphviz.org/download/ • Add dot.exe (from <graphvizhome>/bin) to your system path • Installation: • Unzip PMT Distribution, e.g. PMT-win32.win32.x86_64.zip • Start PMT by double clicking on (ProcessModelingTool.exe)

Creation of Process Model • PMT Tool (started with default views) • File -> New -> PMT Model • Define location and select Process as Model Object • Press Finish

PMT Views Menus for basic functions: New, Open, Save, Preferences Property View Tree Browser: Model Structure Special Views

Property Groups in Properties View • Properties are grouped (in collapsible groups) • Basic: Most important properties • Extended: further properties • Layout: Layout Preferences for graphical views • Tayloring: Properties for tailoring

Meta Model defines Model Structure • PMT has a fixed model structure • The PMT model structure is described using the „meta model“ • Model elements are marked using red color in this training • The model is a tree: and Process is the (type of the) root element • Every model element has „Properties“ that can be edited using PMT • Most model elements in the tree can have children. Children can be created using tree viewer. • There are two (graphical) granularities: • 1) Powerpoint: (model) elements and important relations • 2) User manual (UML): above with multiplicities, association names and attributes Element Process Multiplicity Composition Inheritance Attributes ProcessModule VerificationModule Association Name Association Aritffact

Editing the Model 2) Change values & associations of elements in property view 1) Select element in tree browser Attribute Attribute Association(single Value) Value Composition Composition Link Associations Associations Associations(valuelists) Unlink Create New & Link Composition Boolean Attribute Boolean Attribute

Example: Module Test (ISO 26262) • Example Model for module testing shall cover • ISO 26262 Requirements • Process Definition • Compliance Argumentation • V&V (Safety Plan & Safety Case) • Requirements (from ISO 26262:2018 6-9.4.4) • Process (Validas): • For every module create a test specification, covering all requirements and automatically generate the test cases • Perform tests • Measure required code coverage (100% or arguments)

Creating Process Model • In Tree-View • Unfold file (click on > ) • Select Process • In Properties-View: Edit Properties (Name, Description,..)

Create Requirements • In Tree-Viewer • Select Process (or Requirement) • Start “New Child -> Requirement“ (using right mouse button) • Select new Requirement • In Properties Viewer: Edit Properties (use IDs for traceability)

Safety Level Specification • Can be specified (as in tables) • Use Extended Properties Group • Close Basic Group (Click on it) • Open Extended Group (Click on it) • Specify “Required From” and “Required To” Level (using pull-down menus)

Automated Tailoring • PMT automatically performs tailoring, based on model evaluation • Tailoring requires • Variables and possible values (types) • Conditions (Boolean terms) for elements to exists • Concrete values bound to variables

Automated Tailoring I: Define Variable • Basic Definition • Define a Process Variable “ISO_ASIL” (in Process) • Define an Enumeration Type “ISO_ASILS” (in Process) with values • ASIL_A • ASIL_B • ASIL_C • ASIL_D • Link Type ISO_ASILS to ISO_ASIL variable

Automated Tailoring II: Conditions • A “Variant-Condition” is a Boolean term over Variables and Constants, e.g. (ISO_ASIL==ASIL_A || ISO_ASIL==ASIL_B) • Add Conditions to Requirements(in Tree browser) • Select Requirement • Create “New Child -> OR Term” • Select OR Term • Create “New Child -> == Term” • Select == Term • Create “New Child -> Param Ref” • Create “New Child -> Enum Value Ref” • Select Param Ref • Link to Parameter to ISO_ASIL • Select Enum Value Ref • Link to Enum Value to ASIL_A • Select == Term and Copy it • Select OR Term and Paste == Term • Modify Enum Value Ref (in copy) from ASIL_A to ASIL B • Select OR-Term • Infer and Set Types • Refresh • Result:

Automated Tailoring III: Bindings • Bindings assign variables to parameters • Model (terms) are evaluated automatically • Terms with result false will not be considered (in reports & images) • In Tree-View: • Select Process • Create New Child -> Binding • Set Parameter to ISO_ASIL • Create New Child -> Enum Value Ref • Select Enum Value Ref • Link To Enum Value -> ASIL_D

Automated Tailoring IV: Example • Add Binding • Evaluate Terms(in Tree View)

Process Modeling: Basic Elements Process StakeHolder • Select Process (or ProcessModule) Container • Create new ProcessModule • Create new Artifacts/Models • Create StakeHolder • Specify and link elements Artifact Input ProcessModule Output References Before / After References

Module Test Process: High Level View

Hierarchic Artifacts • Combine many (sub-) artifacts • Simplify process interfaces • Maintain consistency • For example: Safety Case • Contains test specification ( ) • Contains test result • Contains coverage report

Module Test Process: Lower Views Tailoring! other Measurements are tailored away since ISO_ASIL is bound to ASIL_D

Consistency Check: Validate Model • Configuration: Window -> Preferences -> Process Modeler -> Validation • Start (right mouse button) • Results can be found in: • Dialog (Details)Export Markers -> .xls • Problem View (Navigatable)

Example Finding Execute Test creates “Test Report”, but no sub-process creates it Solution: Create a Sub-Process: Run Test

Project Input / Part of Product • Input Output Consistency checks • Every Artifact is created somewhere • Every Artifact is used somewhere • Problems (violations) with • Artifacts that are “Project Input” (i.e. not created in process) • Artifacts that are “Part of Product” (i.e. not read in process) • For Example: • “Subject Under Test (SUT)” • Validation Result: • Solution: Specification with Attributes

Project Parameters • Model that processes are “Parameterized” • Can be repeated for Instances of Parameters (Lists of values) • Important for Verification and Validation • Example Parameter (Module Test): “MODULE:String” • Modeling: • Select Process Module • Create New Child -> Project Parameter • Select Project Parameter • Specify Properties Note: Also sub-modules “Inherit” Parameter <MODULE>

Models (not part of Unit Test Example) • Models have (in addition to Artifacts) a Meta Model • With the Meta model the process can • Specify mandatory elements • Specify optional elements • Specification elements are • Model Elements • Model Attributes • Model Associations • An “ecore importer” allows to import EMF Meta Model • Select Model Element • Start Ecore Importer: right mouse and • Select .ecore file • Use Meta Model, e.g. for TCA Test Models

Compliance Modeling • Compliance Claims: Process claims compliance with requirements • Compliance Arguments: Requirements are satisfied & verified (Plan) • Compliance Evidence: Verification & Validation (Report)

Compliance Claims • Are selected in the Compliance Group of Process Modules • Are displayed in the Tree-Browser using <-> ID

Compliance Argumentation • Add new “Compliance Elements” to Requirements • Select Requirement • Create New Child -> Compliance • Edit Compliance • Description/Argumentation • Links to implementing ProcessModule(s) • Links to Verification Modules Note: Sub-Compliances can be used to comply with hierarchic requirements

Compliance View • Shows argumentation using GSN • Requirements • Compliances • Verification Modules • Criteria Requirement Argument VM

Verification & Validation: Process • Perform all requiredVerificationModules (steps) • Repeat it for all components of the product (values of parameter) • Excel is used for • Description of Parameter Values • Performance of Checks • Verification & Validation Tool (VVT) • Manages V&V • Export/Import of Excel • Generates V&V Report • Initial VVT Model (“Schema”) is exported from PMT

Verification & Validation: Modeling • Necessary to ensure that requirements are satisfied • Verification Modules need to be “operative” • Add “Criterion” (=question to be answered) to it • Assign one verified artefact to them • Use them in compliance argumentations • Artifacts need to have a path (to determine date/version) • Example Criterion: Test Results OK

Export VVT Model • Select ProcessModule • Start PMT2VVT Converter • Save VVT-Model (“Schema”)

Parameter Values • PMT Exports Excel Table • Select ProcessModule • Start • Open Excel Table • Fill in values of Parameter(s)

Plan V&V with VVT • Start VVT (like PMT) • Open Schema.vvt • Select Project • Adapt Properties (assign Tester) • Import Parameters • Save as Plan.vvt

V&V Plan (Model) • Tree of Checks to be performed • Leave-Checks for Criteria • Grouped by Artifacts • Parameterized (like in PMT)

V&V Plan Table (Excel) • Select Project • “Export Excel Check Plan”

Process Modeling Tool Example: Unit Tests