model validation in model based development
Download
Skip this Video
Download Presentation
Model-Validation in Model-Based Development

Loading in 2 Seconds...

play fullscreen
1 / 10

Model-Validation in Model-Based Development - PowerPoint PPT Presentation


  • 105 Views
  • Uploaded on

Model-Validation in Model-Based Development. Kurt Woodham L-3 Communications. Ajitha Rajan, Mats Heimdahl University of Minnesota. OSMA SAS ’08 September 8-12. Problem: Model Validation. Model-Based Development (MBD) is here to stay Use of MBD is accelerating

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Model-Validation in Model-Based Development' - rafe


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
model validation in model based development

Model-Validation in Model-Based Development

Kurt Woodham

L-3 Communications

Ajitha Rajan, Mats Heimdahl

University of Minnesota

OSMA SAS ’08 September 8-12

problem model validation
Problem: Model Validation
  • Model-Based Development (MBD) is here to stay
    • Use of MBD is accelerating
      • Estimate 50% of NASA development projects using some form of MBD
      • Many advantages: model-checking, code generation, desktop testing, closed-loop simulation
    • Enhances early detection of requirement, design, or implementation defects
      • “Executable Specifications” enable evaluation of behavior that might otherwise be relegated to Inspections and Testing
  • How do we know the models are “right”?
    • Manually develop black-box tests
  • When have we validated enough?
    • Measure test coverage on an implementation/model

SAS_08_Model_Val_Exec_Heimdahl

problem current practice
Problem : Current Practice
  • Measure black-box test coverage over the model
    • Indirect measure
      • Defects of omission in model not exposed.
    • Executable artifact is necessary
      • Adequacy can only be determined late in the development process

Incomplete Model

Weak

Black-Box

Test set

SAS_08_Model_Val_Exec_Heimdahl

goals of project
Goals of Project
  • Define metrics for objective, implementation-independentmeasure of adequacy of a black-box test suite
  • Develop tools to measure validation adequacy based on the defined metrics
  • Provide capability for autogeneration of black-box test suites

SAS_08_Model_Val_Exec_Heimdahl

testing what does it mean

Assertions

Does it implement?

Specification

Does it implement?

Model

Implementation

Model-Based Testing (MBT) to Verify Code

Model

Does it implement?

Source Code

Testing – What does it mean?

Assertion Based Testing (ABT) to Validate Model

In General

Our contribution is in providing novel ABT capabilities

SAS_08_Model_Val_Exec_Heimdahl

what are assertions
What are Assertions?

Properties/ Formal Assertions

Assertions

Can also be over components, interfaces,...

Defined over

System

in1

out1

ink

outm

SAS_08_Model_Val_Exec_Heimdahl

contributions abt

Validate

Assess Model and Assertion Completeness

3

Model

Contributions - ABT

2

Auto-generate

Black-Box Tests

Assertions

Measure Adequacy

1

  • We provide the following contributions in the Assertion-Based testing domain (indicated by in the above figure):
  • Objective, implementation-independentmeasure of adequacy of a black-box test suite
  • Auto-generation of black-box validation tests directly from assertions
  • Objective assessment of completeness of model as well as assertions

SAS_08_Model_Val_Exec_Heimdahl

relevance to nasa
Relevance to NASA
  • MBD is here - estimate one-half of all NASA missions in development or on the books will use model-based subsystem development
    • Extensive use in avionics industry
  • How do we know the models are right?
    • Model validation problem
  • We provide the capability to
    • Objectively measure the “quality” of assertion-based black-box validation tests
    • Objectively assess the completeness of a model
      • Does the model address all assertions?
    • Objectively assess the adequacy of a set of assertions
      • Are there enough assertions to adequately describe the model?
    • Automatically generate truly assertion-based tests

SAS_08_Model_Val_Exec_Heimdahl

achievements to date
Achievements to Date
  • Formal assertion notation identified
    • Most work with LTL
    • Extended to work with Live Sequence Charts (LSC)
  • Objective validation metrics defined
    • Requirements, Antecedent, Unique First Cause, and Unique Cause
  • Test case generation tool developed
    • Developed tool generating tests from LTL
      • Capable of generating tests to all metrics defined
    • Prototype tool working on LSC developed
  • Developed test-adequacy measurement tool for the defined validation metrics
  • Evaluation of metrics and tool
  • 12 papers and one PhD dissertation (Ajitha Rajan)

Tool Demo in Tech. Session

SAS_08_Model_Val_Exec_Heimdahl

next steps
Next Steps
  • Investigate alternative requirements notations to LTL
  • Complete empirical evaluation of the effectiveness in model validation
    • Flight Guidance System (FGS) evaluation complete
    • Display Manager (DM) evaluation in work
    • Coordinate evaluation on NASA IV&V project
  • Coordinate technology transfer

SAS_08_Model_Val_Exec_Heimdahl

ad