SOUND METHODS and EFFECTIVE TOOLS for ENGINEERING MODELING and ANALYSIS _________________. by David Coppit, College of William and Mary, and Kevin J. Sullivan, University of Virginia Proceedings of the 25th International Conference on Software Engineering Portland, Oregon - May 3-10, 2003.
SOUND METHODS and EFFECTIVE TOOLSfor ENGINEERING MODELING and ANALYSIS_________________
by David Coppit, College of William and Mary,
and Kevin J. Sullivan, University of Virginia
Proceedings of the 25th International Conference on Software Engineering
Portland, Oregon - May 3-10, 2003
Presentation by Bryan E. Bloss - University of Central Florida, Nov. 2003
Designing Software Tools for Formal Design
Engineering modeling & analysis methods
are based on modeling languages for
describing systems, with semantics for
mapping expressions (models) to
estimates of system properties (results).
To be safe and effective, a modeling method requires
a language with a validated semantics; feature-rich,
easy-to-use, dependable tools; and low engineering costs.
Hampered by shortcomings in software engineering & languages, today we lack adequate means to develop such methods.
Two Sub-Problems Addressed in this Paper:
- Finding a cost-effective way to ensure
semantic soundness of a complex method
- Using Package-Oriented Programming (POP) to
produce easy-to-use, functionally-rich tools from
available software packages (such as MS Office)
Results: A package-based tool “Galileo” is evaluated
favorably by NASA engineers, and development of
“Nova”, a similar tool based on a formal semantics,
proves the cost-effectiveness of a combined approach
2. FORMAL SEMANTICS & DEPENDABILITY
Why are they important?
Specification is more fundamental than implementation.
Without a formal specification:
- Validation is difficult
- No basis for a definitive user reference document
- Programmers are left to make uninformed semantic
decisions; unable to thoroughly test correct functioning.
Tools used in the design of safety critical systems should be treated as critical engineering components.
Our inability to develop low-cost, easy-to-use tools can thus be seen as a positive safety mechanism, but far from ideal.
Safety Example: 1996 alert from U.S. Nuclear Regulatory Commission warned of significant errors in several tools which had been adopted for use in nuclear reactor design & analysis
Another Example (not in paper): Crater analysis tool, used inappropriately during flight STS-107 to analyze foam damage
3. APPROACH & BASIC RESULTS
Developing the Galileo Tool for DFT Analysis
Observation: Most applications devote less than 10% of their code to the core function of the system!
90% is devoted to superstructure-- support functions such as text & graphical editing, data validation, etc.
Package-Oriented Programming (POP) is intended to save time in creating superstructure; frees more resources for the critical design activity: applying formal methods to define & validate the syntax and semantics of the modeling language
The Application: Dynamic Fault Trees (DFT)
Graphcal representation of every
conceivable sequence of events
that could cause a system to fail.
Each leaf is a basic event; internal
gates define relationships leading
to system failures at upper levels.
Static trees model how event
combinations lead to failures;
Dynamic trees are order-sensitive.
3./4. CASE STUDY: RELIABILITY ENGINEERING
The Problems With Current DFT Languages
During development of Galileo tool, a non-trivial error was found in the underlying DFT language, DIFtree, where probability of a masked (hidden) failure wasn’t correctly computed
Also, DIFtree’s informal specification had left ambiguities on how to handle special cases; prior software implementations answered these questions inconsistently in different parts of the program.
And formal validation was time-consuming, due to lack of automation in available syntax & theorem-prover tools, and slow run-time perfomance
Worse, the theorem-prover tool required too much user expertise; guidance often needed from the tool’s author
5. The NOVA DFT Tool
Like Galileo, uses POP components from MS Office for fault tree editing:
- Word for text editing
- Visio graphical editor (enhanced for DFT modeling constructs)
- Excel for computational results
Will allow even more emphasis on formalization & validation than in Galileo
6. END-USER EVALUATION OF GALILEO
New version of Galileo tool funded by NASA Langley Research Center, to support new modeling & analysis constructs and be usable in practice
Featured in three workshops:
1st- Managers & engineers from several NASA divisions
2nd- Space Station engineers only
3rd- Space Shuttle engineers only
A short survey (34 questions) and an in-depth survey (77 questions) were offered to engineers, to evaluate user perceptions of usability & features
Feedback indicated that usability was same or better than other tools!
Also confirmed that dependability is crucial; a formal specificaltion of the modeling language was second only to a comprehensive test suite as a means for increasing trust.
____Evaluation of this Article ____
Strengths:A well-informed overview of the authors’ experience with developing a new software toolset for practical engineering, while emphasizing formal validation methods. Many implications for the design of other reliability-critical software applications.
Weaknesses:Little information on costs saved by POP method; little detail on formal proving methods. Also, the more advanced NOVA tool had not been user-tested at time of publication, so final results aren’t known.