Chapter 7: Computer Reliability

1. Ethics for the Information AgeForth Edition by Michael J. Quinn Chapter 7: Computer Reliability

2. Chapter Overview • Introduction • Data-entry or data-retrieval errors • Software and billing errors • Notable software system failures • Therac-25 • Computer simulations • Software engineering • Software warranties

3. Introduction • Computer systems are sometimes unreliable • Erroneous information in databases • Misinterpretation of database information (similar names) • Malfunction of embedded systems (fatal errors) • Effects of computer errors • Inconvenience (incorrect bill) • Bad business decisions • Fatalities

4. Data-Entry or Data-Retrieval Errors • A computerized system may fail because wrong data entered into it • A computerized system may fail because people incorrectly interpret data they retrieve

5. Disfranchised Voters • November 2000 general election • Florida disqualified thousands of voters • Reason: People identified as felons • Cause: Incorrect records in voter database • Consequence: May have affected election’s outcome

6. Thu 5-12 False Arrests • Sheila Jackson Stossier mistaken for Shirley Jackson at airport. • Arrested and spent five days in detention • Roberto Hernandez mistaken for another Roberto Hernandez • Arrested twice and spent 12 days in jail • Terry Dean Rogan arrested after someone stole his identity • Arrested five times, three times at gun point

7. Position of Privacy Advocates • Number of records is increasing • More erroneous records  more false arrests • Accuracy of crime records more important than ever

8. Sun 26-4 Software and Billing Errors • Assume data correctly fed into computerized system • System may still fail if there is an error in its programming (a bug)

9. Errors Leading to System Malfunctions • Qwest sends incorrect bills to cell phone customers ($600/minute) • A bill of more than $57,000 for a customer • Spelling and grammar error checkers increased errors (University documents) • BMW on-board computer failure (Thailand finance minister was trapped in his BMW)

10. Tue 27-11 Errors Leading to System Failures • Temporarily out-of-control Boeing 777 Malaysian plane over the Indian ocean-autopilot error (2005) • Japan’s air traffic control system went down for an hour (2003)  some flight were delayed • Emergency calls were put on hold for 30 minutes, ambulances took 3 hours to respond  20 people died because the computerized dispatch system of London has a failure.

11. Analysis: E-Retailer Posts Wrong Price, Refuses to Deliver • In (2005) Amazon.com in its British website offered iPaq (handheld computers – pocket PCs) for £7 instead of £275 • Orders flooded in • Amazon.com shut down site, refused to deliver unless customers paid true price • Was Amazon.com wrong to refuse to fill the orders?

12. Mon 14-7 Notable Software System Failures • Patriot Missile (28 soldiers killed in 1991) – insufficient precision in an floating-point variable (system clock) • Ariane 5 is a satellite launch vehicle self-destructed in 1996 ($500 million not insured) – software error • In 1990, AT&T long-distance network (70 million calls couldn’t be made and 60,000 people lost service because routing switches crashed) • In 2000, direct recording electronic voting machines registered either no or multiple choice. • In 1987, Therac-25 (3 patients killed out of 6 patients) - software error resulted in overdoses.

13. Sun 23-4 Direct Recording Electronic Voting Machines • After problems with 2000 election, punch card voting systems were replaced by direct recording electronic (DRE) voting machines – touch screen • Brazil and India have run national elections using DRE voting machines exclusively • In November 2006 1/3 of U.S. voters used DRE voting machines

15. Issues with DRE Voting Machines • Voting irregularities (due to programming errors) • Failure to record votes • Overcounting votes • Misrecording votes • Lack of a paper audit trail • Vulnerability to tampering • Source code a trade secret, can’t be examined to find out how secure it is. • Possibility of widespread fraud through malicious programming – change programs

16. Moral Responsibility of the Therac-25 Team • Conditions for moral responsibility • Causal condition: actions (or inactions) caused the harm • Mental condition • Actions (or inactions) intended or willed -OR- • Moral agent is careless, reckless, or negligent • Therac-25 team morally responsible • They constructed the device that caused the harm • They were negligent

17. Sun 22-4 Uses of Computer Simulations • Simulations replace physical experiments. Why? • Experiment too expensive or time-consuming • Experiment unethical • Experiment impossible (evolution of the universe) • Examples of uses of simulations: • Model past events (Evolution of the universe) • Understand world around us (Search for Oil) • Predict the future (weather predictions)

18. Thu 11-12 Validating Simulations • Erroneous Simulation because of : • Bugs in SW • Model uses SW is flawed • Solutions: • Validation: Does the model accurately represent the real system? • Validation is to check whether software meets the customer expectations and requirements (Specs) • Verification: Does program correctly implement model? • Verification is to check whether the software conforms to specifications (Implementation) • Validation methods • Make prediction, wait to see if it comes true (Car crash) • Predict the present from old data (Weather forecast) • Test credibility with experts and decision makers

19. Validating Simulations • Validation: Are we building the right system? • Verification: Are we building the system right? • Validation is concerned with checking that the system will meet the customer’s actual needs, while verification is concerned with whether the system is well-engineered, error-free, and so on. Verification will help to determine whether the software is of high quality, but it will not ensure that the system is useful.

20. Wed 16-7 Software Engineering • SE is a four steps process: • Specs: determine the functions to be performed • Development: produce SW that meet specs. • Validation: testing the SW • Evolution: Modify SW to meet change requirements

21. Software Engineering: Specification • Followed SW Crises in 1960s • Specifications of SE: • Determine system requirements • Understand constraints • Determine feasibility (Budget and schedule) • End products • High-level statement of requirements (Summary) • Mock-up of user interface • Low-level requirements statement (Detailed)

22. Software Engineering: Development • Create high-level design • Discover and resolve mistakes, omissions in specification • CASE tools to support design process • Object-oriented systems have advantages • After detailed design, actual programs written • Result: working software system

23. Software Engineering: Validation (Testing) • Ensure software satisfies specification • Ensure software meets user’s needs • Challenges to testing software • Noncontinuous responses to changes in input. No identical data set of inputs • Exhaustive testing impossible. Infinite number of different inputs • Testing reveals bugs, but cannot prove none exist • Test modules, then subsystems, then system

24. 17-7 Software Quality Is Improving • Standish Group tracks IT projects • Situation in 1994 • 1/3 projects cancelled before completion • 1/2 projects had time and/or cost overruns • 1/6 projects completed on time / on budget • Situation in 2006 • 1/6 projects cancelled • 1/2 projects had time and/or cost overruns • 1/3 projects completed on time / on budget

25. Shrinkwrap Warranties • Some say you accept software “as is” • Some offer 90-day replacement or money-back guarantee • None accept liability for harm caused by use of software

26. Moral Responsibility of Software Manufacturers • If vendors were responsible for harmful consequences of defects • Companies would test software more • They would purchase liability insurance • Software would cost more • Start-ups would be affected more than big companies and thus • Less innovation in software industry • Software would be more reliable • Making vendors responsible for harmful consequences of defects may be wrong • Consumers should not have to pay for bug fixes