85 Views

Download Presentation
##### Complex Systems Modeling and Networks

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Complex Systems Modeling and Networks**Summary of NECSI Course CX202 11 February 2009 John M. Linebarger, PhD Sandia National Laboratories Interactive Systems Simulation & Analysis (06344) Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.**Outline**• Overview • What are Complex Systems? • Complex Networks • Metrics • Types of Networks • Modeling and Simulation of Complex Systems • How to Model • Types of Models • Further Reading • Class Project • Overview of Design for Tractable Analysis (DTA) framework**Overview**• I attended a Complex Systems Modeling and Networks (CX202) course from 23-27 June 2008 • I used all 32 of my FY 2008 Strategic Education Initiative (SEI) hours on the course; the balance was covered by the Systems Analysis LDRD. • The course was held at the New England Complex Systems Institute (NECSI) on the campus of the Sloan School at MIT, in Cambridge, MA • The course prerequisite is CX201, Complex Physical, Biological and Social Systems • I fulfilled the prerequisite by reading Making Things Workby Yaneer Bar-Yam, and Linked by Albert-László Barabási, before the course began • The follow-on course is CX203, Methods for the Study of Complex Systems, which is mathematically-oriented**Overview (cont’d)**• Over 30 attendees from all over the world, predominately PhD students and academics • Only three or four people from “industry” • My project group consisted of a Norwegian PhD student, a South Korean health care researcher (working in the Boston area), an Iranian PhD student (studying in Maryland), and a Bulgarian PhD student (studying in France) • The course was divided into two parts • The first part was complex network analysis, taught by Dan Braha of NECSI and UMass Dartmouth • The second part was modeling and simulation of complex systems, taught by Hiroki Sayama of NECSI and SUNY Binghamton • Project groups met each evening, often until quite late, to work on their course project • Projects (and the associated papers) were presented to the class on Friday, followed by the final exam for the course**What are Complex Systems?**• Complex systems are characterized by emergent, self-organizing, collective behavior • Such emergent collective behavior often arises due to the combination of simple individual behavior patterns • The components of a complex system are interdependent • Complex systems exhibit multiscale variety • The structure of the system differs depending on the level at which it is viewed • A common pattern is a mixture of competition and cooperation at different levels of the system • Complex systems arise due to evolutionary, not deterministic, processes • Engineered systems should be implemented this way too • Examples of complex systems include the health care system, the education system, and recent military conflict From Making Things Work by Yaneer Bar-Yam**Another Characterization of Complex Systems**• Complex systems contain many constituents interacting nonlinearly • The constituents of a complex system are interdependent • A complex system possesses a structure spanning several scales • A complex system is capable of emerging behavior • Complexity involves an interplay between chaos and non-chaos • “Complex systems dance on the edge of chaos” • Complexity involves an interplay between cooperation and competition From “Chaos, Complexity, and Entropy” by Michel Baranger**Wicked Problems**• Every wicked problem is essentially unique • There is no definitive formulation of a wicked problem • Wicked problems are never solved • A wicked problem is not understood until after the formulation of a solution • Solutions to wicked problems change the problem itself • Solutions to wicked problems are not true or false, but better or worse • Stakeholders have radically different world views and different frames for understanding the problem • Conclusion • Wicked problems are a proper subset of complex systems • All wicked problems are complex systems, but not all complex systems are wicked problems**Dancing on the Edge of Chaos**• Two fundamental assumptions of most systems analysis are that the system is continuous, and that if it is decomposed far enough it becomes simple • It can then be analyzed without loss of fidelity • These assumptions may be driven by the predominant analytical tool of 20th century physics, which was calculus • For both derivatives and integrals, if you zoom in far enough a curve becomes a straight line • Thus a calculus problem is transformed into a geometry problem, which has been well understood since Euclid • However, chaotic systems are often discontinuous, and the decomposition of chaotic systems generally yields more complexity • e.g., fractals, Sierpiński triangles • “The arrogance of analysis” • Yet chaotic systems often are simple, not in terms of their decomposition, but in terms of the rules that govern their behavior • e.g., “boids,” where simple rules lead to complex flocking behaviors**Complex Networks**• The interdependencies of complex systems are often modeled by networks • Networks can be analyzed using several important metrics from graph theory • Directed (nodes and arcs) vs. undirected graphs (nodes and edges) • Number of nodes (N): The number of nodes in a network • Degree (k): The number of edges or arcs incident on a node • Average degree is denoted as <k> • Can distinguish between in degree and out degree • Node density is actual connections between nodes divided by potential connections that could exist between nodes • Note that real-world networks are quite sparse, and have an average density of 10-4 • Characteristic path length (L): The average length of the shortest path in the network between two nodes • Clustering coefficient (C): Number of actual links between neighbors of a node divided by the number of possible links between those neighbors • This can be thought of as local density, the probability that two neighbors know each other • The clustering coefficient is a measure of modularity**Random Networks**• For years (since the late 1950’s) the dominant model of networks was the random network, otherwise known as the Erdős-Rényi model, even though it is rarely seen in nature • Random graphs have a fixed number of nodes which are connected randomly by undirected edges. The number of edges depend on a specified probability. • The seminal paper is “On Random Graphs,” by Erdős and Rényi (1959)**Small-World Networks**• Adding a small number of links to a regular ring network, in which only the nearest neighbors are connected, can have dramatic effects • The characteristic path length becomes quite small with only a small number of such cross-neighborhood links • Small-world networks are pervasive in nature: Friend networks, power grids, predator-prey networks, and even neural networks of the brain. • The seminal paper is “Collective Dynamics of ‘Small-World’ Networks,” by Watts and Strogatz (1998)**Scale-Free Networks**• Scale-free networks are so-called because they have no characteristic scale, or average number of links per node • Instead, they are characterized by numerous hubs that hold together highly connected regions of the network • However, direct connections between hubs are rare (for reasons of resilience) • Scale-free networks are also pervasive in nature: Disease propagation, terrorist networks, and the World-Wide Web • The seminal paper is “Emergence of Scaling in Random Networks,” by Barabási and Albert (1999)**Scale-Free Networks (cont’d)**• Scale-free networks seem to follow a power law distribution (which is a variant of a Pareto distribution, or the 80/20 rule) • Earlier models (such as Erdős-Rényi) described networks using a bell curve, with many nodes with the same number of links and no highly connected nodes • The scale free model describes networks as having many nodes with only a few links, and a few nodes with many links • Power law functions appear as straight lines on a log-log plot Power Law Distribution Bell Curve Many nodes with few links Most nodes have the same number of links # of nodes with k links # of nodes with k links A few nodes with many links No highly connected nodes # of links (k) # of links (k)**Scale-Free Networks (cont’d)**Pk ~ e-k Pk ~ k-γ**Scale-Free Networks (cont’d)**• In a more formal sense, a scale-free network is characterized by two properties • Continuous growth (the other network models did not consider growth) • Preferential attachment (e.g., “the rich get richer”) • The number of nodes with exactly k links follows a power law for any value of k • P(k) ~ k-γ • Often 2 < γ < 3 • However, many real-world networks actually exhibit “stretched exponential” behavior, becoming exponential past a threshold • Cost of adding new links • Aging From Faloutsos, M., P., & C., “On Power-Law Relationships of the Internet Topology” (1999)**Topology is Destiny!**• Characterizing a network quantitatively gives insight into the dynamics (performance) and vulnerabilities of the network • Random graphs have a short path length (good), but also a small clustering coefficient (not good) • Regular ring networks have a large clustering coefficient (good) but long path length (not good) • Small-world networks are a balance between those two extremes, but do not have the robustness of scale-free networks • Scale-free networks (which are similar in some ways to small-world networks) are robust against random attacks (good) but not against targeted attacks (not good) • The hubs are vulnerable when directly targeted • The network can become disconnected fairly rapidly in this kind of attack • The moral of the story: When it comes to networks (and to everything else in life), you can’t have it all …**How Complex Network Research is Done**• A network under study is represented and analyzed mathematically • A randomly generated “ensemble” of networks is created with which to compare to the network under study • Ensembles of several types of networks are generated, such as random graphs, small world, regular ring, and scale-free • The network ensembles are analyzed mathematically and compared to the network under study • Both the network under study and the generated ensembles are subjected to the same process (e.g., attack), generally via simulation, and the results are compared**Tools for Complex Network Analysis**• Pajek, an open source tool that generates and visualizes networks • A useful book is Exploratory Social Network Analysis with Pajek, by de Nooy et al. • UCINET, a commercial tool from Analytic Technologies that quantitatively analyzes networks • Python with the NetworkX package • MATLAB, a commercial tool from MathWorks, which can generate, visualize, and quantitatively analyze networks after “a simple matter of programming” • Free network analysis packages are available for MATLAB**Modeling and Simulation of Emergent Behavior**• Fundamentals of Modeling • Modeling and Simulation of Dynamical Systems (complex interdependent systems that can exhibit emergent behavior) • Non-Spatial Modeling • Cellular Automata • Agent-Based Models • Network Models • Other Simulation Paradigms for Dynamical Systems (not covered in the course) • Systems Dynamics • Numerous commercial packages exist, such as Vensim • Can be described in terms of difference or differential equations, so can be addressed by non-spatial modeling • Spatial models with partial differential equations • “Soft computing” (e.g., evolutionary computation, artificial neural networks, or support vector machines)**Fundamentals of Modeling**• Scientific “laws” are really hypotheses, models of observed behavior • There is no way to prove anything about nature, although it is easy to disprove something • A model is a simplified representation of a system • Rule-based modeling allows the system’s possible or future states to be extrapolated, unlike descriptive modeling, which just specifies the system’s actual or past states • A good model is simple, correct, and robust**Modeling Procedure**• In general • Observe the system of interest • Reflect on the possible rules that might cause the system’s characteristics • Derive predictions from those rules and compare them with reality • Repeat to modify the rules until satisfied • For complex systems • Define the key questions and choose the right scale of modeling • Identify the structure (the parts and their connections) of the system • Define the possible states for each part (state space) • Describe how the state of each part changes over time through interactions with other parts • Models must be evaluated and validated, in terms of simplicity, correctness, and robustness**Tools Used for Modeling and Simulation**• Python • NumPy and SciPy (MATLAB-like numerical packages for Python) • matplotlib (MATLAB-like visualization library for Python) • NetworkX (network graphing package) • Agent-based modeling toolkits, such as NetLogo, StarLogo, or Swarm (see the Swarm Wiki for other tools) • And, of course, MATLAB …**Non-Spatial Modeling**• The simplest mathematical model • Called “non-spatial” because the spatial location of the elements of the system is unimportant • A dynamical system is a system whose state is uniquely specified by a finite set of variables and whose behavior is uniquely determined by predetermined rules • Examples are simple population growth, simple pendulum swing, motion of celestial bodies • Two schools of modeling dynamical systems • Discrete-time model (using difference/recurrence equations or iterative maps) • Continuous-time model (using differential equations) • Non-autonomous, higher-order equations can always be converted into autonomous, first order equations, by introducing additional variables • Non-linear systems are generally simulated because they cannot be solved analytically • Exercise: Logistic growth (logisticgrowthmodel.py)**Cellular Automata**• Adds spatio-temporal dynamics, especially for locally connected networks, to the modeling process • Cells in a regular grid interact with their nearest neighbors according to simple interaction rules • Transition from one state to another depends on simple rules based on interactions with neighbors • The resulting system is a linear system • Example: Simple diffusion, panic in a gym • Exercise: Decide whether or not to panic based on the state of a simple majority of your neighbors (majority.py)**Agent-Based Models**• The only way to study the behavior of agent-based models is to simulate, since no global equation exists, only equations at the agent level • Agent characteristics (what makes an agent an agent) • Local state • Spatial localization • Local interaction with other agents based on predefined rules • No central supervisor • May learn autonomously • May produce non-trivial collective behavior**Agent-Based Models (cont’d)**• To develop an agent-based model • Identify the agents and get a theory of agent behavior • Identify the agent relationships and get a theory of agent interaction • Get the requisite agent-related data • Validate the agent behavior models in addition to the model as a whole (thus you must justify your model relative to the real-world, including emergent behavior) • Run the model and analyze the output • Exercise: Garbage collection by ants (ants.py) • Eventually, only one pile of garbage remains • No simulation of the environment is needed, just tracking of the environment; however, if pheremones were emitted and decayed with time, the environment would need to be simulated as well, based on a model of pheremone decay • An excellent two-part tutorial was presented by Macal and North at the Winter Simulation Conference in 2005 and 2006, respectively**Network Models**• Simulation of the types of complex networks we learned about in the first part of the course • Uses the NetworkX package from LANL • Has methods to create random networks, small-world networks, and scale-free networks • Exercise: A disease epidemic on a random network (networkepidemics.py) • Two parameters: Infection probability and recovery probability • Red means infected; yellow means susceptible • Note that the disease is never completely eradicated from a social network; this is called the fixation of a disease • Can experiment with different network topologies**Further Reading**• Complexity and Chaos • Making Things Work, by Yaneer Bar-Yam (2005) • Ubiquity: The Science of History … or Why the World is Simpler than We Think(a.k.a. “Ubiquity: Why Catastrophes Happen”), by Mark Buchanan (2001, 2002) • Chaos: Making a New Science, by James Gleick (1988) • The Dynamics of Complex Systems, by Yaneer Bar-Yam (1997) • Complex networks • Linked, by Albert-László Barabási (2003) • Small Worlds: The Dynamics of Networks between Order and Randomness, by D. J. Watts (2003) • Exploratory Social Network Analysis with Pajek, by de Nooy, et al. (2005) • Modeling and simulation of dynamical systems • Cellular Automata, by Andrew Ilachinski (2001) • Python Scripting for Computational Science, 3rd ed., by Hans Petter Langtangen (2006) • Simulation Modeling and Analysis, 4th ed., by Averill Law (2007)