software engineering issues in system dependability n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Software Engineering Issues in System Dependability PowerPoint Presentation
Download Presentation
Software Engineering Issues in System Dependability

Loading in 2 Seconds...

play fullscreen
1 / 30

Software Engineering Issues in System Dependability - PowerPoint PPT Presentation


  • 133 Views
  • Uploaded on

Software Engineering Issues in System Dependability. Dr Martyn Thomas CBE FREng DIRECTOR, MARTYN THOMAS ASSOCIATES. 60 Years of Computing 40 Years of Software Engineering. The first stored program computer ran its first program in Manchester, UK in June 1948.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Software Engineering Issues in System Dependability' - courtney


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
software engineering issues in system dependability

Software Engineering Issues in System Dependability

Dr Martyn Thomas CBE FREng

DIRECTOR, MARTYN THOMAS ASSOCIATES

60 years of computing 40 years of software engineering
60 Years of Computing 40 Years of Software Engineering
  • The first stored program computer ran its first program in Manchester, UK in June 1948.
  • The first commercial software was the Bakeries Valuations for the Lyons Electronic Office (UK 1951)
  • The first software engineering conference was held in Garmisch, Germany 40 years ago in October

1968

privacy security and software engineering
Privacy, Security and Software Engineering
  • To process and manage confidential information, we need secure, dependable computer systems.
  • Creating secure and dependable systems cost-effectively needs strong software engineering.
  • Commercial software development is usually not good enough to create secure systems with very few errors.
  • A Study by the US National Security Agency found that methods based on mathematics can provide a cost-effective solution.
  • The Challenges of Complex IT Projects
slide4
By Secure SystemsI mean systems that provide the required service and do not permit unauthorised access or interference.
  • By Dependable SystemsI mean systems that can be trusted to operate as required, because there is strong evidence that they do not contain critical errors.
most commercial software contains many errors
Most commercial software contains many errors
  • Researchers have reported fault densities of 1 to 30 defects per KLoC (thousand lines of source program) in commercial software in service.
  • The UK Ministry of Defence found over 20 defects per KLoC in the operational software in the C130J military transport aircraft (including many safety critical faults).
  • Sources: German & Mooney, Air Vehicle Static Code Analysis - Lessons Learnt. 2001. Pfleeger & Hatton,IEEE Computer, pp33-42, February 1997. Fenton & Ohlsson, Quantitative Analysis of Faults and Failures in a Complex Software System, http://www.md.kth.se/RTC/SC3S/papers/FentonOhlsson105280_final.pdf
security errors have caused great damage
Security errors have caused great damage
  • “Several individual viruses and worms have caused events where damage was assessed at over $1billion each” Source: Software for Dependable Systems - Sufficient Evidence? National Academies Press, 2007
  • Many security vulnerabilities arise from simple programming errors, such as “buffer overflows” where the length of input into an array is not checked and this allows a malicious user to overwrite the program.
testing alone can never be enough to show that a program is dependable
Testing alone can never be enough to show that a program is dependable
  • To test all possible paths with all possible data is completely impractical.
  • Finding bugs and fixing them cannot lead to secure, dependable software.
  • Critical software errors may remain hidden for decades.
testing software tells you that the tests work not that the software works
Testing software tells you that the tests work – not that the software works

Continuous behaviour means you can interpolate between test results

Discrete behaviour means that you can’t!

an experiment by the nsa
An Experiment by the NSA
  • A recently published experiment by the US National Security Agency shows that the use of mathematically formal methods can produce dependable software, cost effectively.
tokeneer id station
TOKENEER ID Station

Source: Slides by Randolph Johnson, NSA

the tokeneer experiment
The Tokeneer experiment
  • Praxis developed software for the Tokeneer ID Station using their Correct by Construction (C by C) methods
  • NSA asked an independent team to find errors
  • Praxis then trained a group of NSA staff in C by C
  • The NSA staff modified the TIS to add new functions
  • All the specifications, designs, tests, analyses, proofs and other documentation were collected
  • Productivity was analysed and compared with previous projects
correct by construction software engineering source peter amey praxis his
Correct by Construction Software Engineering source: Peter Amey, Praxis HIS
  • A combination of formal methods and agile development
  • Precise notationsand incremental development
  • Three basic principles:
  • Make it hard to introduce errors (Z, SPARK)
  • Find any errors almost immediately (SPARK Examiner)
  • Generate evidence of dependability throughout development, as a natural by-product of the process
c by c development
C by C Development

Source: Peter Amey, Praxis HIS

c by c guiding principles
C by C Guiding Principles
  • Write right
  • Step, don’t leap
  • Say something once
  • Check here before going there
  • Explain your decisions
  • Use the right tools for the job

Source: Peter Amey, Praxis HIS

write right
Write Right
  • Capture information, requirements, designs and code unambiguously and accurately.
  • Use notations with strong semantics
step don t leap
Step, Don’t Leap
  • Make each development step semantically small, to reduce errors and improve checking.
say something once
Say something once
  • Make each document have a clear purpose and write information in only one place. This aids clarity and avoids inconsistency.
check here before going there
Check here before going there
  • Verify each step before moving on. Check for consistency against something else – usually the previous design step.
explain your decisions
Explain your decisions
  • Document design decisions and explain why they are justified – that will help you be sure they are right, and help you and others to understand them later.
use the right tools
Use the right tools
  • Use the appropriate verification method for each property, whether tool-supported proof, informal peer review, static analysis or testing.
tokeneer was written in spark
Tokeneer was written in SPARK
  • SPARK enforces type security and enables deep static analysis:
  • no buffer overflows
  • complete elimination of several classes of errors
  • complete analysis of program flow and data flow
  • SPARK formal annotations make it possible to prove that the software implements the specification

http://www.praxis-his.com/sparkada/

statistics of tokeneer system
Statistics of Tokeneer System

Source: R Johnson, NSA

statistics of tokeneer system1
Statistics of Tokeneer System

Source: R Johnson, NSA

c by c is a practical approach
C by C is a practical approach
  • The NSA found that the Correct by Construction methods could be successfully taught to postgraduate engineers in a few weeks.
  • Summer intern students successfully added new features to the Tokeneer ID Station using C by C, with high productivity and zero defects
all the tokeneer materials have been published
All the Tokeneer materials have been published
  • http://www.adacore.com/tokeneer
  • The materials allow the entire project to be repeated, including all the static analysis steps and all the proofs.
  • The NSA has licensed the materials under the GPL (GNU General Purpose Licence).
it projects often overrun1
IT projectsoften overrun
  • Standish “Chaos Chronicles” (2004 edition):
    • 18% of projects “failed”; (cancelled before completion)
    • 53% of projects “challenged” (operational, but seriously over budget and/or over time with fewer features or functions than initially specified)
  • Typical Standish figures:
    • Cost overruns on 43% of projects; and
    • Time overruns on 82% of projects.
  • Oxford University/Computer Weekly 2003 study:
    • 10% of UK projects “failed”; and
    • 75% of UK projects “challenged”.
getting the requirements right
Getting the Requirements right
  • In the UK, many Government IT projects have failed.
  • Our software industry is very optimistic; so are our politicians.
  • This can lead to “irrational exuberance” (Alan Greenspan’s phrase)
  • There is no such thing as an IT Project - only Business Change enabled by IT. The IT costs are typically < 50%
  • The Royal Academy published a study in 2004: The Challenges of Complex IT Projects which recommended the use of systems architects
how do you get the right technical solution to a business requirement
How do you get the right technical solution to a business requirement?
  • USE AN ARCHITECT!

A specialist in requirements, high-level designand working with engineers

http://www.raeng.org.uk/news/publications/list/reports/Complex_IT_Projects.pdf

summary
Summary
  • Software that needs to be secure must be dependable
  • Dependability can only be achieved with the use of mathematically formal methods
  • Formal methods are now practical and cost-effective for use in commercial and industrial software development
  • The Tokeneer experiment allows software developers to benchmark their own methods against Correct by Construction and to experiment with developing near zero-defect software, at no increase in cost.
  • Our engineering Academies need to promote the use of scientifically valid methods and tools, to reduce the number of project failures and security disasters.