Model validation in model based development
This presentation is the property of its rightful owner.
Sponsored Links
1 / 10

Model-Validation in Model-Based Development PowerPoint PPT Presentation


  • 72 Views
  • Uploaded on
  • Presentation posted in: General

Model-Validation in Model-Based Development. Kurt Woodham L-3 Communications. Ajitha Rajan, Mats Heimdahl University of Minnesota. OSMA SAS ’08September 8-12. Problem: Model Validation. Model-Based Development (MBD) is here to stay Use of MBD is accelerating

Download Presentation

Model-Validation in Model-Based Development

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Model validation in model based development

Model-Validation in Model-Based Development

Kurt Woodham

L-3 Communications

Ajitha Rajan, Mats Heimdahl

University of Minnesota

OSMA SAS ’08September 8-12


Problem model validation

Problem: Model Validation

  • Model-Based Development (MBD) is here to stay

    • Use of MBD is accelerating

      • Estimate 50% of NASA development projects using some form of MBD

      • Many advantages: model-checking, code generation, desktop testing, closed-loop simulation

    • Enhances early detection of requirement, design, or implementation defects

      • “Executable Specifications” enable evaluation of behavior that might otherwise be relegated to Inspections and Testing

  • How do we know the models are “right”?

    • Manually develop black-box tests

  • When have we validated enough?

    • Measure test coverage on an implementation/model

SAS_08_Model_Val_Exec_Heimdahl


Problem current practice

Problem : Current Practice

  • Measure black-box test coverage over the model

    • Indirect measure

      • Defects of omission in model not exposed.

    • Executable artifact is necessary

      • Adequacy can only be determined late in the development process

Incomplete Model

Weak

Black-Box

Test set

SAS_08_Model_Val_Exec_Heimdahl


Goals of project

Goals of Project

  • Define metrics for objective, implementation-independentmeasure of adequacy of a black-box test suite

  • Develop tools to measure validation adequacy based on the defined metrics

  • Provide capability for autogeneration of black-box test suites

SAS_08_Model_Val_Exec_Heimdahl


Testing what does it mean

Assertions

Does it implement?

Specification

Does it implement?

Model

Implementation

Model-Based Testing (MBT) to Verify Code

Model

Does it implement?

Source Code

Testing – What does it mean?

Assertion Based Testing (ABT) to Validate Model

In General

Our contribution is in providing novel ABT capabilities

SAS_08_Model_Val_Exec_Heimdahl


What are assertions

What are Assertions?

Properties/ Formal Assertions

Assertions

Can also be over components, interfaces,...

Defined over

System

in1

out1

ink

outm

SAS_08_Model_Val_Exec_Heimdahl


Contributions abt

Validate

Assess Model and Assertion Completeness

3

Model

Contributions - ABT

2

Auto-generate

Black-Box Tests

Assertions

Measure Adequacy

1

  • We provide the following contributions in the Assertion-Based testing domain (indicated by in the above figure):

  • Objective, implementation-independentmeasure of adequacy of a black-box test suite

  • Auto-generation of black-box validation tests directly from assertions

  • Objective assessment of completeness of model as well as assertions

SAS_08_Model_Val_Exec_Heimdahl


Relevance to nasa

Relevance to NASA

  • MBD is here - estimate one-half of all NASA missions in development or on the books will use model-based subsystem development

    • Extensive use in avionics industry

  • How do we know the models are right?

    • Model validation problem

  • We provide the capability to

    • Objectively measure the “quality” of assertion-based black-box validation tests

    • Objectively assess the completeness of a model

      • Does the model address all assertions?

    • Objectively assess the adequacy of a set of assertions

      • Are there enough assertions to adequately describe the model?

    • Automatically generate truly assertion-based tests

SAS_08_Model_Val_Exec_Heimdahl


Achievements to date

Achievements to Date

  • Formal assertion notation identified

    • Most work with LTL

    • Extended to work with Live Sequence Charts (LSC)

  • Objective validation metrics defined

    • Requirements, Antecedent, Unique First Cause, and Unique Cause

  • Test case generation tool developed

    • Developed tool generating tests from LTL

      • Capable of generating tests to all metrics defined

    • Prototype tool working on LSC developed

  • Developed test-adequacy measurement tool for the defined validation metrics

  • Evaluation of metrics and tool

  • 12 papers and one PhD dissertation (Ajitha Rajan)

Tool Demo in Tech. Session

SAS_08_Model_Val_Exec_Heimdahl


Next steps

Next Steps

  • Investigate alternative requirements notations to LTL

  • Complete empirical evaluation of the effectiveness in model validation

    • Flight Guidance System (FGS) evaluation complete

    • Display Manager (DM) evaluation in work

    • Coordinate evaluation on NASA IV&V project

  • Coordinate technology transfer

SAS_08_Model_Val_Exec_Heimdahl


  • Login