Software Engineering Chapter 3 Critical systems

1. Software EngineeringChapter 3 Critical systems Ku-Yaw Chang canseco@mail.dyu.edu.tw Assistant ProfessorDepartment of Computer Science and Information Engineering Da-Yeh University

2. Objectives • Understand that in a critical system, system failure can have severe human or economic consequences • Understand four dimensions of system dependability: availability, reliability, safety and security • Understand that to achieve dependability you need to avoid mistakes during the development of a system, to detect and remove errors when the system is in use and to limit the damage caused by operational failures Software Engineering

3. Preamble • System failures • Cause inconvenience but no serious, long-term damage • Result in significant economic losses, physical damage or threats to human life • Critical systems • Three main types • Safety-critical systems • Injury, loss of life, serious environmental damage • e.g. chemical manufacturing plant • Mission-critical systems • Failure of goal-directed activity • e.g. navigational system for a spacecraft • Business-critical systems • Very high costs • e.g. customer accounting system in a bank Software Engineering

4. Preamble • Dependability • Cover related system attributes • Availability • Reliability • The most important emergent property of a critical system • Systems that are unreliable, unsafe or insecure are often rejected by their users • System failures may be enormous • Untrustworthy systems may cause information loss • Safety • Security Software Engineering

5. Preamble • Trusted methods and techniques must be used • Well-tried techniques rather than new techniques • Developers are naturally conservative • Expensive software engineering techniques may sometimes be used • Costs of verification and validation are usually very high – more than 50% of the total cost • Most are socio-technical systems • System operators can • Help recover from problems • Cause problems if they make mistakes Software Engineering

6. Contents 3.1 A simple safety-critical system 3.2 System dependability 3.3 Availability and reliability 3.4 Safety 3.5 Security 3.6 Exercises Software Engineering

7. Diabetes • A common condition where the human pancreas is unable to produce sufficient quantities of a hormone called insulin • Insulin metabolizes glucose in the blood • Low levels of blood glucose (too much insulin) • Temporary brain malfunctioning, unconsciousness and death • High levels of blood glucose (too little insulin) • Eye damage, kidney damage and heart problems • Miniaturized sensors • Automated insulin delivery systems • Monitor blood sugar level • Deliver appropriate dose of insulin when required Software Engineering

8. A software-controlled insulin pump Software Engineering

9. Data-flow model ofthe insulin pump Software Engineering

10. Contents 3.1 A simple safety-critical system 3.2 System dependability 3.3 Availability and reliability 3.4 Safety 3.5 Security 3.6 Exercises Software Engineering

11. System dependability • A property equating to its trustworthiness • The degree of user confidence that the system • Will operate as they expect • Will not ‘fail’ in normal use • Four principal dimensions to dependability • Availability • To deliver services when requested • Reliability • To delver services as specified • Safety • To operate without catastrophic failure • Security • To protect itself against accidental or deliberate intrusion Software Engineering

12. Dimensions of dependability Software Engineering

13. Other dependability properties • Reparability • Reflects the extent to which the system can be repaired in the event of a failure • Maintainability • Reflects the extent to which the system can be adapted to new requirements • Survivability • Reflects the extent to which the system can deliver services while under hostile attack • Error tolerance • Reflects the extent to which user input errors can be avoided and tolerated Software Engineering

14. System dependability • A trade-off between system performance and system dependability • High dependability can only be achieved at the expense of system performance • Dependable software includes extra, often redundant, code • Increasing the dependability can significantly increase development costs • Additional design, implementation and validation costs Software Engineering

15. Cost/dependability curve Cost Software Engineering

16. Contents 3.1 A simple safety-critical system 3.2 System dependability 3.3 Availability and reliability 3.4 Safety 3.5 Security 3.6 Exercises Software Engineering

17. Availability and reliability • Reliability • The probability that the system’s services will be correctly delivered as specified • Availability • The probability that the system will be up and running to deliver these services to users when they request them • Both properties are closely related • Availability is more critical than reliability • A telephone exchange switch • Availability depends on • The system itself • The time needed to repair the faults Software Engineering

18. Practical problems • Different environments • an office environment vs. a university environment • Human perceptions and patterns • Unreliable windscreen wipers in a car may be irrelevant in a dry climate • Severity of failure or consequences of unavailability • A failure of initialization in the engine management software • An engine that cuts out while they are driving Software Engineering

19. Reliability terminology Software Engineering

20. Faults and failures • Failures • Usually a result of system errors • Derived from system faults • Errors • Do not necessarily lead to system failures • Can be corrected by built-in error detection and recovery • Faults • Do not necessarily result in system errors • May be transient and ‘corrected’ before an error arises Software Engineering

21. Reliability achievement • Fault avoidance • Development techniques are used that either minimize the possibility of mistakes or trap mistakes before they result in the introduction of system faults • Fault detection and removal • Verification and validation techniques that increase the probability of detecting and correcting errors before the system goes into service are used • Fault tolerance • Run-time techniques are used to ensure that system faults do not result in system errors and/or that system errors do not lead to system failures Software Engineering

22. Reliability modeling • Model a software system as an input-output mapping • Some inputs will result in erroneous outputs • The reliability of the system • The probability that a particular input will lie in the set of inputs that cause erroneous outputs • Different people will use the system in different ways • The probability is not a static system attribute but depends on the system’s environment Software Engineering

23. Input/output mapping Software Engineering

24. Reliability perception Software Engineering

25. Reliability improvement • Removing X% of the faults in a system will not necessarily improve the reliability by X%. • A study at IBM showed that removing 60% of product defects resulted in a 3% improvement in reliability • Program defects may be in rarely executed sections of the code so may never be encountered by users. • Removing these does not affect the perceived reliability • A program with known faults may therefore still be seen as reliable by its users. • Deliberately avoid using system features that can cause problems Software Engineering

26. Contents 3.1 A simple safety-critical system 3.2 System dependability 3.3 Availability and reliability 3.4 Safety 3.5 Security 3.6 Exercises Software Engineering

27. Safety • A property of a system • The ability to operate, normally or abnormally, without danger of causing human injury or death and without damage to the system’s environment • Increasingly important to consider software safety • More and more devices incorporate software-based control systems • Safety requirements are exclusive requirements • They exclude undesirable situations rather than specify required system services Software Engineering

28. Safety • Safety-critical systems • Systems where it is essential that system operation is always safe • Two classes • Primary safety-critical software • Software embedded as a controller in a system • Malfunctioning cause a hardware malfunction • Result in human injury or environmental damage • Secondary safety-critical software • Indirectly result in injury • e.g., a medical database holding details of drugs administered to patients Software Engineering

29. Safety and reliability • Safety and reliability are related but distinct • In general, reliability and availability are necessary but not sufficient conditions for system safety • Reliability • Bs concerned with conformance to a given specification and delivery of service • Safety • Be concerned with ensuring system cannot cause damage • Irrespective of whether or not it conforms to its specification Software Engineering

30. Unsafe reliable systems • Specification errors • If the system specification is incorrect then the system can behave as specified but still cause an accident • Hardware failures generating spurious inputs • Hard to anticipate in the specification • Context-sensitive commands i.e. issuing the right command at the wrong time • Often the result of operator errors Software Engineering

31. Safety terminology Software Engineering

32. Safety achievement • Hazard avoidance • The system is designed so that some classes of hazard simply cannot arise. • Hazard detection and removal • The system is designed so that hazards are detected and removed before they result in an accident • Damage limitation • The system includes protection features that minimize the damage that may result from an accident Software Engineering

33. Normal accidents • Accidents in complex systems • Rarely have a single cause • A fundamental principle of safe systems design • Be resilient to a single point of failure • Impossible to anticipate all possible combinations of system malfunction • Accidents are an inevitable part of using complex systems Software Engineering

34. Contents 3.1 A simple safety-critical system 3.2 System dependability 3.3 Availability and reliability 3.4 Safety 3.5 Security 3.6 Exercises Software Engineering

35. Security • A system attribute • The ability to protect itself from external attacks that may be accident or deliberate • Becoming increasingly important as more and more systems are connected to the Internet • An essential pre-requisite for availability, reliability and safety • Errors can lead to security loopholes • Not respond to unexpected inputs • Array bounds are not checked • Programs in C Software Engineering

36. Three types of damage • Denial of service • Normal services become unavailable • Corruption of programs or data • Be altered in an unauthorized way • Disclosure of confidential information • Confidential information may be exposed to unauthorized people Software Engineering

37. Security terminology Software Engineering

38. Security assurance • Vulnerability avoidance • The system is designed so that vulnerabilities do not occur • e.g. No external network connection • Attack detection and neutralization • Attacks on vulnerabilities are detected and removed before they result in an exposure • e.g. Find and remove viruses before they infect a system • Exposure limitation • Adverse consequences of a successful attack are minimized • e.g. A backup policy allows damaged information to be restored Software Engineering

39. Security • Vast majority of vulnerabilities • Human failings (rather than technical problems) • Easy-to-guess passwords • Write down passwords in places where they can be found • System administrators make errors • Setting up access control or configuration files • Forget to install or use protection software • Take a socio-technical perspective • Not just about their technical characteristics Software Engineering

40. Contents 3.1 A simple safety-critical system 3.2 System dependability 3.3 Availability and reliability 3.4 Safety 3.5 Security 3.6 Exercises Software Engineering

41. Exercises • 3.1 • 3.8 • 3.10 Software Engineering

42. The End