Roderick Chapman and Peter Amey Praxis Critical Systems

Practical Experiences of Safety-Critical Ada Technologies Roderick Chapman and Peter Amey Praxis Critical Systems

Programme • Introduction • What is High Integrity Software? • Reliable Programming in Standard Languages • Coffee • Standards Overview • DO178B and the Lockheed C130J • Lunch • Def Stan 00-55 and SHOLIS • ITSEC, Common Criteria and Mondex • Tea • Compiler and Run-time Issues • Conclusions

Safety-critical Software • Safety is a system property • A system is safe if it will: • not endanger life • not cause a major environmental problem • etc. • Where the system relies on the operation of software to achieve safe behaviour the software is safety-critical • Not all software in safety-related systems is safety-critical

Why Have Safety-critical Software? • Advantages of software • Complexity • Disadvantages of software • Complexity!

High-integrity Software • Code where reliability is more important than • Efficiency • Cost • Time to market • Functionality • Clearly safety-critical software should be high-integrity (but reverse is not always true)

Examples of High-integrity (but not S-C) • Security systems • Systems where direct financial loss could be large • Avoidance of “down-time” • Avoiding product recall or loss of customer confidence • Loss of costly one-off mission capability (e.g. Mars lander)

Unique Characteristic of High-integrity Software • The need to be able to show, before there is any service experience, that a system will meet requirements • Qualitatively different problem from “normal” software • Standards required may be very high • 109 hours > 114,000 years • Meeting this challenge is not the same as certification • Not enough just to be “more careful”

Some Definitions • Verification is the process of determining that a system (or component) meets its specification • Validation is the process of determining that a system is appropriate for its purpose • Certification is persuading an external regulatory body that a set of specific requirements have been met or processes followed

A Single-sentence Standard Thefitness for purpose of a software program shall be established by logical reasoning

V&V Techniques • Inspections and reviews • e.g. • Requirements reviews • Code inspections • Test • e.g. • Requirements-based • White box • Analysis • e.g. • Flow analysis • Model checking • Proof

Inspection and Reviews • Advantages • Very flexible, we can review an entity for internal consistency, compare different artefacts for equivalence etc. • No special tools needed • Can be done early • Disadvantages • Informal process; may spot errors but cannot guarantee absence • What is feasible limited by: • Semantic precision of artefact being inspected • Semantic gap between artefacts being compared

Advantages of Dynamic Testing • Spans the entire development process • Potentially able to identify: • Requirements errors • Specification errors • Coding errors • Compiler bugs • Hardware problems

Disadvantages of Dynamic Testing • Theoretical limitations • Exhaustive testing almost invariably impossible • High levels of confidence require infeasible amounts of testing • Practical disadvantages • Can only take place towards the end of development • Can be hard to diagnose unexpected behaviour • Frequently a bottleneck (e.g. Shared use of test rig) • Very expensive • Significant source of project risk

A Note on Theoretical Limitations of Testing • Ultra-high reliability region (<10-7 failures per hour) • Bayesian mathematics clearly limits what we can claim from testing alone • Reliability growth models cannot provide necessary assurance • See: • The infeasibility of quantifying the reliability of life-critical real-time software. Butler & Finelli. NASA Langley Research Center • Validation of Ultrahigh Dependability for Software-based Systems. Littlewood & Strigini. CACM Nov 1993

Advantages and Disadvantages of Analysis • Advantages • Can be used early in the development process • Can establish properties that cannot be demonstrated in any other way. e.g. • Proof of absence of run-time errors • Freedom from timing deadlocks • Disadvantages • Can only compare artefacts (e.g. Code against specification) • What can be achieved is limited by precision of descriptions and notations used

A Balanced Approach • Showing “correctness” is harder than building correct systems • Use a variety of complementary verification techniques • Bug detection and correction is expensive so: • Focus on bug prevention • Use techniques that find bugs early • Regard final testing as demonstration of correct behaviour rather than method of finding bugs

Resources • Littlewood, Bev; and Strigini, Lorenzo: Validation of Ultrahigh Dependability for Software-based Systems. CACM 36(11): 69-80 (1993) • Butler, Ricky W.; And Finelli, George B.: The Infeasibility of Quantifying the Reliability of Life-critical Real-time Software. IEEE Transactions on Software Engineering, vol. 19, no. 1, Jan. 1993, pp 3-12 • Littlewood, B: Limits to Evaluation of Software Dependability. In Software Reliability and Metrics (Procedings of Seventh Annual CSR Conference, Garmisch-Partenkirchen). N. Fenton and B. Littlewood. Eds. Elsevier, London, pp. 81-110

Outline • Language issues • Special purpose languages • Dialects • Subsets • MISRA-C • Ada HRG report • SPARK • Other Ada subsets • Conclusions

Languages Affect the Way We Think • Less likely to think of abstractions in machine code or assembler • Think more in machine terms in C • Unlikely to think in object-terms in FORTRAN or in functional terms in Smalltalk • For larger systems, language support for abstraction and encapsulation is vital

When to Get Formal? • Object (machine) code is formal • The execution machine provides operational semantics for it • Typically we cannot reason about machine code we can only observe its behaviour • Reliance on dynamic testing loads the lifecycle towards the back (expensive) end • Early reasoning saves money and reduces risk but is hampered by lack of formality of programming languages

Causes of Uncertainty • Deficiencies in language definitions • Semantics of C integer division • Pascal named vs. Structural type equivalence • Implementation freedoms • Execution order • Parameter passing mechanisms

Consequences of Uncertainty • Leads to either: • Ambiguity; programs whose behaviour cannot be predicted from the source code; or • Insecurity; violations of language rules that cannot be detected

Example of an Ambiguity y = f(x) + g(x); Suppose functionfmodifiesxas a side-effect of its operation.In this case the meaning of the expression depends on whetherforgisevaluated first. A rule stating “functions are not permitted to have side effects” simply turns theambiguityinto aninsecurity

A Simple C Ambiguity i = v[i++]; Page 46 of the C++ Annotated Reference Manual states that this leads to the value ofibeing undefined. Incidentally, this is a violation of MISRA-C Rule 46

A Tiny Ada Ambiguity procedure Init2(X, Y : out integer) is begin X := 1; Y := 2; end Init2; What is the meaning of: Init2(A, A); Incidentally, this call is not legal SPARK

Resolution • Ambiguities are resolved .... by compiler authors • Insecurities are left for the user to discover • Possible solutions • Invent newlanguages without these problems • Work with dialects associated with compilers • Use logically coherent language subsets to overcome ambiguities and insecurities

Outline • Language issues • Special purpose languages • Dialects • Subsets • MISRA-c • Ada HRG report • SPARK • Other Ada subsets • Conclusions

Special-purpose Languages • Inventing new languages is fun! e.g. • Gypsy, • Newspeak • But very small user base leads to: • Poor or non-existent tools • Staff shortages • Lack of training support

The VIPER Analogy • VIPER was a “formally verified” microprocessor • Commercial failure • Users preferred “defective” processors with wide support • Abandoning the wider user-base is a high-risk strategy

Using Language Dialects • Find out what your compiler does • Document it • Include it in: • Coding standards • Review checklists

Problems With Dialects • What is the compiler’s behaviour? • Is it consistent? • Does it change with new releases? • Is it the same on host and target? • Knowing when it matters—to know the dialect meaning of Init(A, A); we must both: • recognise that implementation dependent behaviour is present; and • know what the compiler’s behaviour is in this case

Dialect Observations • Having a working knowledge of the compiler is always useful, but • You cannot expect to avoid all anomalous behaviour by this method • The approach may be valuable in special cases. e.g. small, specialized processors where one company provides processor, compiler and analysis tools

Language Properties Required Produced by independent specialists on behalf of UK MoD. • Simple • Application oriented • Predictable • Formally verifiable • Sound supporting tools

No currently standardised language could be recommended without reservation for the most critical applications without subsetting” Requirements for Programming Languages, Computer Standards and Interfaces 1992 BAWichmann, National Physical Laboratory

Safe Subsets • Potentially give us the best of both worlds: • Logical soundness and predictability • Access to standard compilers, tools, training, staff But • Level of integrity achievable depends on foundation language chosen • Subsetting alone may not be enough for the highest integrity levels

Constructing a Safe Subset • Selection of base language • Removal of the most troublesome language features • Limitations on the way remaining features may be used • Introduction of annotations to provide extra information

The Subset Spectrum • We can construct subsets that vary on 4 axes: • Precision (security and lack of ambiguity) • Expressive power • Depth of analysis possible • Efficiency of analysis process • Trade-offs quite complex; we are trying to avoid surprise: unexpected behaviour which we don’t find until test • Removing problematic features may reduce this risk • Increased precision may require reduction in expressive power but improves depth of analysis • We may be able to combine expressiveness with depth of analysis but at the cost of efficiency of analysis

The Subset Spectrum (Contd.) • Fundamental trade-off is between discipline we accept to reduce bug insertion and the effort we are prepared to make in bug detection • For example: • Unrestricted C provides little or protection from bug insertion • Ada requires extra discipline (e.g. strong typing) which reduces bug insertion rate • A qualitative shift in what is possible only occurs when precision becomes exact

MISRA-c • C subset defined by UK motor industry research association (MIRA) and associated software reliability association (MISRA) (www.misra.org.uk) • Comprises 127 “rules” presented in the form of a coding standard • The language designers regard it as suitable for “SIL 3” systems; they recommend Ada or Modula-2 if available

Constructing a Safe Subset - MISRA-C • Selection of base language • Removal of the most troublesome language features • Limitations on the way remaining features may be used • Introduction of annotations to provide extra information

Constructing a Safe Subset - MISRA-C • Selection of base language • ISO/IEC 9899:1990 + technical corrigendum 1995 • Removal of the most troublesome language features • e.g. pointer arithmetic • Limitations on the way remaining features may be used • e.g all switch statements should have final default • Introduction of annotations to provide extra information • not provided

Roderick Chapman and Peter Amey Praxis Critical Systems