Complex Systems and System Accidents

1. Complex Systems and System Accidents presented by: Joel Winstead

2. High-risk systems • Many high-risk systems: airplanes, chemical plants, nuclear power, dams • These systems are complex, with many interacting parts • Many industrial accidents • For some systems, we cannot tolerate failures • Concern that risks appear faster than solutions

3. What is a system? • Organizations, and organizations of organizations • Set of interrelated components that act together as a whole to achieve a common goal

4. What is a system? • A system is an abstraction or model • A system has • state • an environment • inputs • outputs • subsystems

5. Methodological Reductionism • Analyze a system by breaking it into parts • This assumes: • Division into parts does not distort the system • Components are the same when examined separately • Principles governing assembly into whole are straightforward

6. Complexity • Organized simplicity • reductionism works • Unorganized complexity • e.g., ideal gas laws • Organized Complexity • systems analysis

7. Hierarchies and Emergence • A complex system has a hierarchy of levels of organization • Each level has its own rules and structure • There are some properties that cannot be reduced to lower levels

8. Communication and Control • Hierarchies separated by interfaces • Control processes operate across interfaces • Control processes impose constraints on lower levels in the hierarchy

9. History of Safety Design • Factories not legally responsible for worker’s injuries • Safety concerns often ignored • A series of accident studies, pressure from labor unions, and legislation changed this • Later, realization that production increases as safety increases

10. Safety Devices • Machinery not initially designed for safety • Accident-investigation-fix approach • Guards attached to machinery to prevent some kinds of accidents • Safety should be built into design • This eventually led to universal safety standards

11. World War II Production • Initially, focus shifted back to functionality over safety • But, industrial accidents hurt war effort • more killed in industrial accidents than battlefield • Increased complexity means a posteriori methods no longer work • People began to think in terms of systems

12. Systems Engineering and Analysis • Large, complex, semi-automatic, unpredictable systems • Must analyze system as a whole • Needs analysis, feasibility studies, trade studies, architecture development, interface analysis

13. System Accidents • Sometimes components fail • Some events in systems are tightly coupled • This leads to interactive complexity • In order to understand the failure, we need to understand the system and not just the first component to fail

14. Normal Accidents • Normal = inherent, not expected or frequent • Multiple failures • Tight coupling • Interdependence of events not visible to operator • Inherent property of systems, not components

15. Perrow’s Day in the Life • The story begins with a coffee pot left on • Many seemingly unrelated things fail, resulting in our hero being unable to get to an important appointment • What was the primary cause of this?

16. Complexity is to blame • There was coupling where it wasn’t expected • Redundant paths don’t help when there are multiple failures or tight coupling • Some components not normally considered individually important had large consequences

17. Aren’t real systems designed? • This “system” consists of many separately designed components stuck together in an ad-hoc way • It is not how this particular system was designed, but the kinds of failures and couplings that occurred in it that are interesting • Jumbo jets have coffee pots too

18. What can we do about this? • Adding new safety systems just adds new systems to the mix • We need to avoid the properties that make these systems complex • We won’t always be able to do this • We need to consider what systems we really need

19. Are Perrow and Leveson talking about the same thing? • Leveson focuses on how systems are built and designed • Perrow focuses on how systems fail • Are they talking about the same “systems”?