model validation in model based development
Skip this Video
Download Presentation
Model-Validation in Model-Based Development

Loading in 2 Seconds...

play fullscreen
1 / 10

Model-Validation in Model-Based Development - PowerPoint PPT Presentation

  • Uploaded on

Model-Validation in Model-Based Development. Kurt Woodham L-3 Communications. Ajitha Rajan, Mats Heimdahl University of Minnesota. OSMA SAS ’08 September 8-12. Problem: Model Validation. Model-Based Development (MBD) is here to stay Use of MBD is accelerating

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Model-Validation in Model-Based Development' - rafe

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
model validation in model based development

Model-Validation in Model-Based Development

Kurt Woodham

L-3 Communications

Ajitha Rajan, Mats Heimdahl

University of Minnesota

OSMA SAS ’08 September 8-12

problem model validation
Problem: Model Validation
  • Model-Based Development (MBD) is here to stay
    • Use of MBD is accelerating
      • Estimate 50% of NASA development projects using some form of MBD
      • Many advantages: model-checking, code generation, desktop testing, closed-loop simulation
    • Enhances early detection of requirement, design, or implementation defects
      • “Executable Specifications” enable evaluation of behavior that might otherwise be relegated to Inspections and Testing
  • How do we know the models are “right”?
    • Manually develop black-box tests
  • When have we validated enough?
    • Measure test coverage on an implementation/model


problem current practice
Problem : Current Practice
  • Measure black-box test coverage over the model
    • Indirect measure
      • Defects of omission in model not exposed.
    • Executable artifact is necessary
      • Adequacy can only be determined late in the development process

Incomplete Model



Test set


goals of project
Goals of Project
  • Define metrics for objective, implementation-independentmeasure of adequacy of a black-box test suite
  • Develop tools to measure validation adequacy based on the defined metrics
  • Provide capability for autogeneration of black-box test suites


testing what does it mean


Does it implement?


Does it implement?



Model-Based Testing (MBT) to Verify Code


Does it implement?

Source Code

Testing – What does it mean?

Assertion Based Testing (ABT) to Validate Model

In General

Our contribution is in providing novel ABT capabilities


what are assertions
What are Assertions?

Properties/ Formal Assertions


Can also be over components, interfaces,...

Defined over







contributions abt


Assess Model and Assertion Completeness



Contributions - ABT



Black-Box Tests


Measure Adequacy


  • We provide the following contributions in the Assertion-Based testing domain (indicated by in the above figure):
  • Objective, implementation-independentmeasure of adequacy of a black-box test suite
  • Auto-generation of black-box validation tests directly from assertions
  • Objective assessment of completeness of model as well as assertions


relevance to nasa
Relevance to NASA
  • MBD is here - estimate one-half of all NASA missions in development or on the books will use model-based subsystem development
    • Extensive use in avionics industry
  • How do we know the models are right?
    • Model validation problem
  • We provide the capability to
    • Objectively measure the “quality” of assertion-based black-box validation tests
    • Objectively assess the completeness of a model
      • Does the model address all assertions?
    • Objectively assess the adequacy of a set of assertions
      • Are there enough assertions to adequately describe the model?
    • Automatically generate truly assertion-based tests


achievements to date
Achievements to Date
  • Formal assertion notation identified
    • Most work with LTL
    • Extended to work with Live Sequence Charts (LSC)
  • Objective validation metrics defined
    • Requirements, Antecedent, Unique First Cause, and Unique Cause
  • Test case generation tool developed
    • Developed tool generating tests from LTL
      • Capable of generating tests to all metrics defined
    • Prototype tool working on LSC developed
  • Developed test-adequacy measurement tool for the defined validation metrics
  • Evaluation of metrics and tool
  • 12 papers and one PhD dissertation (Ajitha Rajan)

Tool Demo in Tech. Session


next steps
Next Steps
  • Investigate alternative requirements notations to LTL
  • Complete empirical evaluation of the effectiveness in model validation
    • Flight Guidance System (FGS) evaluation complete
    • Display Manager (DM) evaluation in work
    • Coordinate evaluation on NASA IV&V project
  • Coordinate technology transfer