1 / 17

Can We Trust the Computer?

Can We Trust the Computer?. Case Study: The Therac-25 Based on Article in IEEE-Computer, July 1993. Introduction. More computers introduced into safety-critical systems results in more accidents One of the most widely reported accidents involved the Therac-25 radiation therapy machine

jana
Download Presentation

Can We Trust the Computer?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Can We Trust the Computer? Case Study: The Therac-25 Based on Article in IEEE-Computer, July 1993.

  2. Introduction • More computers introduced into safety-critical systems • results in more accidents • One of the most widely reported accidents involved the Therac-25 • radiation therapy machine • June 1985 and January 1987 • Six known accidents - massive overdoses • causing deaths and serious injuries • Worst accidents in 35 year history of medical accelerators

  3. Introduction(2) • Mistakes made not unique to this manufacturer • fairly common in other safety-critical systems • “A significant amt of SW for life-critical systems comes from small firms, especially in the medical industry; firms that fit the profile of those resistant to or uninformed of the principles of either system safety or software engineering.”

  4. Introduction(3) • These problems are not limited to medical industry • Common belief that a good engineer can build SW, regardless of whether they are trained in state-of-the art SW-Engineering procedures • Many companies build safety-critical SW w/o using proper procedures from a SW-Eng and safety-engineering perspective

  5. Genesis of the Therac-25 • Medical linear accelerators accelerate electrons to create high-energy beams that can destroy tumors w/ minimal impact on surrounding healthy tissue • shallow tissue is treated w/ accelerated electrons; deeper tissue requires converting the electron beam into X-ray photons

  6. The Builders • Early 70’s, Atomic Energy of Canada Limited (AECL) and a French company (CGR) collaborated to build linear accelerators • They developed 1) Therac-6 a 6MeV accelerator producing only X rays, and • 2) Therac-20, a 20-MeV dual mode(X Rays or electrons) accelerator • SW functionality was limited in both machines, it added convenience to existing hardware • Industry-standard hardware safety features and interlocks in the hardware were retained

  7. Developing Therac-25(1) • Mid 70’s, AECL developed a new double-pass concept for electron acceleration • needs less space to develop similar energy levels • AECL developed Therac-25, dual-mode linear accelerator • more compact and versatile than Therac-20 • Therac-6,20,and25 controlled by PDP 11 • Therac-25 takes advantage of computer control from outset while Therac-6 and 20 designed around machines already having histories of clinical use w/o computer control • Therac-25 has more responsibility for maintaining safety than SW in previous machines

  8. Safety Issues : New and Old Therac’s • Therac-20 had independent protective circuits to monitor electron-beam scanning • Therac-20 also had mechanical interlocks for policing machine and ensuring safe operation • Therac-25 relies more on SW for these functions • AECL took advantage of computer’s abilities to control and monitor HW • decided not to duplicate all existing HW safety mechanisms and interlocks • This approach is becoming more common • companies choosing to cut cost by avoiding extra HW interlocks and backups • Maybe they are placing more faith in SW

  9. Therac-25 Development • 1st hardwired Therac-25 developed in 1976 • Completely computerized commercial version available in late 1982 • March 1983, AECL performed a safety analysis in form of a fault tree and EXCLUDED SOFTWARE!

  10. The Safety Analysis Report (before release of product) • Programming errors have been reduced by extensive testing on a HW simulator and under field conditions on teletherapy units. Any residual SW errors are not included in the analysis • Program SW does not degrade due to wear, fatigue, or reproduction process • Computer execution errors are caused by faulty HW components and by “soft” (random) errors induced by alpha particles and electromagnetic noise. • The fault tree does include computer failure but only hardware failures • ex) One OR gate leading to the event of getting the wrong energy is labeled with a probability of 1E-11 • ex) the gate leading to Computer selects wrong mode is labeled with a probability of 4E-9 • The report provides NO justification of either number!

  11. Therac-25 Software Development and Design • SW for Therac-25 developed by a single person using PDP11 ASSEMBLY language • Developed over several years • SW “evolved” from Therac-26 (which was started in 1972) • Very little SW documentation produced during development • AECL also had an apparent lack of documentation on SW specifications and a SW test plan

  12. Therac-25 SW Testing • Manufacturer said the HW and SW were “tested and exercised separately or together over many years” • In deposition, QA manager explained, testing was done in two parts • “small amount” of SW testing done on a simulator • most done on system • Reports indicate that unit and SW testing was minimal • Most testing efforts directed to integrated system test • Same QA manager at a Therac-25 users meeting stated the SW was tested for 2,700 hours • Under questioning by users clarified this as “2700 hours of use” • Programmer left AECL in 1986, we know nothing of the programmer • AECL employees could not provide any information about the programmers educational background or experience

  13. How it Operates • SW responsible for monitoring machine status • accepts input about treatment desired, sets machine up for treatment • turns beam on , activated by operator command • turns beam off when treatment is completed, or when operator commands it OR when a malfunction is detected • Unit has an interlock system designed to remove power to unit when there is a HW malfunction • Computer monitors interlock system and provides diagnostic messages • depending on fault the computer either prevents a treatment from starting OR if treatment is in progress, creates a pause or suspension of treatment

  14. Accident History • Eleven Therac-25’s were installed • 5 in US; 6 in Canada • 6 accidents involving massive overdoses to patients occurred between 1985 and 1987 • Machine recalled in 1987for extensive design changes, including HW safeguards against SW errors • Related problems found in Therac-20 SW, not recognized until after Therac-25 accidents • Not detected because of Therac-20 HW safety interlocks (so no injuries occurred)

  15. Kennestone Regional Oncology Center, 1985 • Marietta, Ga • Accident never carefully investigated, no admission that Therac-25 caused injury until much later • This despite claims by patient that she had been injured during treatment, • obvious and severe radiation burns patient suffered and suspicions of radiation physicist involved

  16. Kennestone(2) • After undergoing a lumpectomy to remove a malignant breast tumor, 61 yr. Old woman was receiving follow up radiation to nearby lymph nodes on • The Therac-25 had been operating at Kennestone for about 6 months other Therac 25-’s had been operating w/o incident since 1983. • Jun 3, 1985, patient set up for a 10-MeV electron treatment to clavicle area • When machine turned on, she felt a “tremendous force of heat… this red-hot sensation.” • Technician came in, she said, “you burned me.” • Technician replied that it was not possible • No red marks on patient at the time, but the area was “warm to the touch.”

  17. Kennestone (3) • Patient went home, shortly afterward developed a reddening and swelling in the center of the treatment area • her pain increased to the point that her shoulder “froze” and she experience spasms • She was admitted to West Paces Ferry Hospital in Atlanta, oncologists continued to send her to Kennestone for Therac-25 treatments • 2 weeks later, physicist at Kennestone noticed a matching reddening on her back as though burn had gone through her body • her should was immobile, she experienced great pain, patients breast had to be removed due to radiation burn • obvious that she had a radiation burn but hospital and doctors could not provide a satisfactory explanation • Kennestone physicist estimated she received one or two doses of radiation in 15k-20k range (typical doses are in 200 rad range)

More Related