Model validation in model based development
1 / 10

Model-Validation in Model-Based Development - PowerPoint PPT Presentation

  • Uploaded on

Model-Validation in Model-Based Development. Kurt Woodham L-3 Communications. Ajitha Rajan, Mats Heimdahl University of Minnesota. OSMA SAS ’08 September 8-12. Problem: Model Validation. Model-Based Development (MBD) is here to stay Use of MBD is accelerating

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Model-Validation in Model-Based Development' - rafe

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Model validation in model based development

Model-Validation in Model-Based Development

Kurt Woodham

L-3 Communications

Ajitha Rajan, Mats Heimdahl

University of Minnesota

OSMA SAS ’08 September 8-12

Problem model validation
Problem: Model Validation

  • Model-Based Development (MBD) is here to stay

    • Use of MBD is accelerating

      • Estimate 50% of NASA development projects using some form of MBD

      • Many advantages: model-checking, code generation, desktop testing, closed-loop simulation

    • Enhances early detection of requirement, design, or implementation defects

      • “Executable Specifications” enable evaluation of behavior that might otherwise be relegated to Inspections and Testing

  • How do we know the models are “right”?

    • Manually develop black-box tests

  • When have we validated enough?

    • Measure test coverage on an implementation/model


Problem current practice
Problem : Current Practice

  • Measure black-box test coverage over the model

    • Indirect measure

      • Defects of omission in model not exposed.

    • Executable artifact is necessary

      • Adequacy can only be determined late in the development process

Incomplete Model



Test set


Goals of project
Goals of Project

  • Define metrics for objective, implementation-independentmeasure of adequacy of a black-box test suite

  • Develop tools to measure validation adequacy based on the defined metrics

  • Provide capability for autogeneration of black-box test suites


Testing what does it mean


Does it implement?


Does it implement?



Model-Based Testing (MBT) to Verify Code


Does it implement?

Source Code

Testing – What does it mean?

Assertion Based Testing (ABT) to Validate Model

In General

Our contribution is in providing novel ABT capabilities


What are assertions
What are Assertions?

Properties/ Formal Assertions


Can also be over components, interfaces,...

Defined over







Contributions abt


Assess Model and Assertion Completeness



Contributions - ABT



Black-Box Tests


Measure Adequacy


  • We provide the following contributions in the Assertion-Based testing domain (indicated by in the above figure):

  • Objective, implementation-independentmeasure of adequacy of a black-box test suite

  • Auto-generation of black-box validation tests directly from assertions

  • Objective assessment of completeness of model as well as assertions


Relevance to nasa
Relevance to NASA

  • MBD is here - estimate one-half of all NASA missions in development or on the books will use model-based subsystem development

    • Extensive use in avionics industry

  • How do we know the models are right?

    • Model validation problem

  • We provide the capability to

    • Objectively measure the “quality” of assertion-based black-box validation tests

    • Objectively assess the completeness of a model

      • Does the model address all assertions?

    • Objectively assess the adequacy of a set of assertions

      • Are there enough assertions to adequately describe the model?

    • Automatically generate truly assertion-based tests


Achievements to date
Achievements to Date

  • Formal assertion notation identified

    • Most work with LTL

    • Extended to work with Live Sequence Charts (LSC)

  • Objective validation metrics defined

    • Requirements, Antecedent, Unique First Cause, and Unique Cause

  • Test case generation tool developed

    • Developed tool generating tests from LTL

      • Capable of generating tests to all metrics defined

    • Prototype tool working on LSC developed

  • Developed test-adequacy measurement tool for the defined validation metrics

  • Evaluation of metrics and tool

  • 12 papers and one PhD dissertation (Ajitha Rajan)

Tool Demo in Tech. Session


Next steps
Next Steps

  • Investigate alternative requirements notations to LTL

  • Complete empirical evaluation of the effectiveness in model validation

    • Flight Guidance System (FGS) evaluation complete

    • Display Manager (DM) evaluation in work

    • Coordinate evaluation on NASA IV&V project

  • Coordinate technology transfer