Toward A Reasonable Programmer Standard

1. Toward A Reasonable Programmer Standard Responsibility and Negligence in Software Design

2. Overview of Presentation • It’s an ethical issue – ask questions; make comments • Why do we need a definition? • New technologies = new legal issues • Software, though not new, is applied in new areas • What to think when software causes harm?

3. Why a Definition? • Objection: you gain nothing by stating a definition • Objection: I know in advance that your definition is wrong • Objection: Standard of negligence doesn’t apply to software

4. The Law and Negligence • “Failure to be sufficiently careful in a matter in which one has a moral responsibility to exercise care...” Online Ethics Center: The Online Ethics Glossary Center. (2003, March 21). • Classic example: leaving a rake on your walkway – kills the mailman • Failing to shovel snow; doctors who leave surgery tools in a patient; etc.

5. When is a person negligent? • The reasonable person standard: “A phrase used to denote a hypothetical person who exercises qualities of attention, knowledge; intelligence, and judgment that society requires of its members for the protection of their own interest and the interests of others.”National Association for Court Management: Glossary of Terms. • Invokes different requirements in different societal roles – the reasonable doctor is very different from the reasonable homeowner • What is a reasonable programmer?

6. Test Case 1: Freeware • Software is distributed freely and with a carefully worded license • What’s the worst that can happen? Computer crashes -> data loss • Holding a programmer responsible for the data is too demanding – not everything is negligent

7. Test Case 2: Free Algorithm • Programmer posts implementation of a sorting algorithm • Algorithm doesn’t work on negative numbers • Company uses algorithm in air traffic control software • Moral intuition: company’s fault, not the algorithm programmer; notion of direct responsibility

8. Test Case 2: Ethical Analysis • Company had contractual and moral obligation to test their software sufficiently • Programmer had no intention of using algorithm in critical environment – no obligation • Conclusion: expectations of performance derive from moral and contractual obligation

9. Test Case 3a: Virus Junkie • A programmer Q gets high off of creating viruses • Q unwittingly unleashes virus, costing U.S. companies millions • Should Q be held responsible?

10. Test Case 3b: Router • A properly functioning antivirus program at company C would have stopped Q’s virus • Is C in any way responsible for the harm caused by the virus?

11. Test Case 4: Therac-25 • Atomic Energy of Canada Limited: faulty software leads to overdose of radiation -> death • AECL probably was ‘sufficiently careful’ – post-accident review showed many hours of testing • Contrasted with bug that fails to save internet books mark: serious consequences important to definition, as is amount of testing proportional to potential harms

12. Test Case 5: Patriot Missile • Programming flaw -> system operates over 20 hours = failure -> military issues patch, deployment is slow • Barracks destroyed, software patch arrives the next day • Raytheon’s fault? Testing revealed the flaw • Army’s fault? Slow deployment of patch, vague memo on proper operation, project extended beyond intended lifetime • New definition: direct causation of flaw, or indirect causation of conditions that lead to flaw

13. Final Definition • Unreasonable, direct failure to be sufficiently careful in software design and testing, or causation of such conditions leading to serious harm, wherein a programmer has a moral obligation to do so from a contractual or otherwise reasonable expectation.

14. Test Case 6: More Viruses • Lots of subtle issues, what can we reasonably expect from each party?: • Obviously, punish those who create virus • OS designer let flaw propagate? • Owners and designers of networks that allow propagation? • User who downloads attachment with subject “Haven’t heard from you in a while!”?