Ronald Westra, Eric Postma Department of Mathematics Universiteit Maastricht

1 / 121

# Ronald Westra, Eric Postma Department of Mathematics Universiteit Maastricht - PowerPoint PPT Presentation

Introduction Knowledge Engineering. Ronald Westra, Eric Postma Department of Mathematics Universiteit Maastricht. Introduction Knowledge Engineering. Lecture 3 Modelling of Dynamical Systems. Introduction Knowledge Engineering. How to survive this course….

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Ronald Westra, Eric Postma Department of Mathematics Universiteit Maastricht' - harrison-barton

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Introduction Knowledge Engineering

Ronald Westra, Eric Postma

Department of Mathematics

Universiteit Maastricht

Introduction Knowledge Engineering

Lecture 3

Modelling of Dynamical Systems

Introduction Knowledge Engineering

How to find this lecture …http://www.math.unimaas.nl/personal/ronaldw/Education/IKT/IKT_page.htm
Growth and Decay
• Examples of Growth and Decay
• Unlimited growth
• Limited growth
• Modelling growth and decay in nature
• Exponential decay of foraging paths
• Growth of knowledge
Growth and decay
• Growth and decay: two sides of the same coin
• Growth
• At each step: replace each element by n elements
• Decay
• At each step: replace n elements by one element
Mathematical description
• Mathematicians like to make short statements
• At time t seconds the quantity is n times the quantity at t-1 seconds
• They say:
• P(t) = n P(t-1)

P(t)

t

plot for P(t) = nP(t-1)

Logarithms
• The rapid growth makes it hard to draw
• Trick: express quantities in terms of their number of zeros
• A logarithmic plot of P(t) = n P(t-1) makes the curves straight…

Log(P(t))

t

Logarithmic plot for P(t) = nP(t-1)

From growth to decay
• We can perform the same trick with decay
• At time t seconds the quantity is 1/n times the quantity at t-1 seconds
• Mathematicians say:
• P(t) = (1/n) P(t-1)

P(t)

t

plot for P(t) = (1/n)P(t-1)

Log(P(t))

t

Logarithmic plot for P(t) = (1/n)P(t-1)

Growth of a population of bacteria
• Consider a controlled laboratory environment:
• Bacteria are single cell micro organisms which reproduce by cell division.
• They live from nutrition provided in the laboratory setting
• There is plenty of space to multiply
Modelling population growth
• Each cell divides after a constant amount of time.
• Initially cells are of different, unrelated, ages
•  During a period of time Δ, the amount of cells which split is proportional to the size of the population.
Modelling population growth
• The rate of change is proportional to the population size:
• Let N(t) be a function specifying the number of cells at time t, t ≥ 0.
• Then it must hold that
• N’ = β N.
• (where N’ is the first order derivative of N ).

Differential equation:

Which function N satisfies the equation?
• Could N be a polynomial?
• N(t)= ao + a1 t + a2 t2 + …. + an tn.
• N’(t) = a1 + a2 t + a3 t2 + …. + an tn-1.
•  N(t’) ≠ N(t)
• N cannot be a polynomial!
Which function N satisfies the equation?
• Can N be an exponential function?
• N(t) = αeγt
• N’(t) = γαeγt
• N’(t) = γ N(t) as required.
What is the value of α?
• Let no be the initial population size, that is, N(0) = no.
• Then no = N(0) = αeγ0 = α.
Definitions
• N’ = β N is called a linear homogenous first order differential equation, because
• It is a linear function,
• it involves only the first order derivative,
• it only considers the function and its derivative.
Conclusion
• The linear first order homogenous difference equation
• xn+1 = a xn
• has solution xn = an xo.
• This problem can be solved without ‘algorithm’, the analytical solution is a formula.
• Notice that xn converges, reaches an equilibrium if and only if |a| < 1.
Unlimited growth

P(t) = nP(t-1)

• In most cases, there is a limit to the growth
• Although this is obvious, it is often forgotten, e.g.,
• World population growth
• Internet hype
• Success
Bounded growth
• Apparently, growth is generally bounded
• An S-shaped curve is characteristic for bounded growth
• The logistic curve
Bounded growth (Verhulst)

P(t+1) = n P(t) (1-P(t))

Logistic model a.k.a. the Verhulst model

• How do you state this model in a linguistic form?
• Pn is the fraction of the maximum population size 1
• n is a parameter indicating time
Balancing growth and decay
• The Verhulst model balances growth:P(t+1) = n P(t)
• With decayP(t+1) = n (1-P(t))

Problems with the logistic function as a model for population growth :

Verhulst attempted to fit a logistic curve to 3 separate censuses of the population of the United States of America in order to predict future growth. All 3 sets of predictions failed.

In 1924, Professor Ray Pearl and Lowell J. Reed used Verhulst's model to predict an upper limit of 2 billion for the world population. This was passed in 1930.

A later attempt by Pearl and an associate Sophia Gould in 1936 then estimated an upper limit of 2.6 billion. This was passed in 1955.

Bounded growth
• Apparently, growth is generally bounded
• An S-shaped curve is characteristic for bounded growth
• The logistic curve (e.g., the Verhulst equation)
Recall the Logistic Model
• Pn is the fraction of the maximum population size 1
•  is a parameter

Large P slows down P

Logistic model a.k.a. the Verhulst model

Interacting quantities
• The logistic model describes the dynamics (change) of a single quantity interacting with itself
• We now move to models describing two (or more) interacting quantities
Fish statistics
• Vito Volterra (1860-1940): a famous Italian mathematician
• Father of Humberto D'Ancona, a biologist studying the populations of various species of fish in the Adriatic Sea
• The numbers of species sold on the fish markets of three ports: Fiume, Trieste, and Venice.

predator

prey

Volterra’s model
• Two (simplifying) assumptions
• The predator species is totally dependent on the prey species as its only food supply
• The prey species has an unlimited food supply and no threat to its growth other than the specific predator
Two Populations P and Q

xtis the prey-populationytis the predator-populationa,b,c,d are parameters

Behaviour of the Volterra’s model

Oscillatory behaviour

Limit cycle

Effect of changing the parameters (1)

Behaviour is qualitatively the same. Only the amplitude changes.

Effect of changing the parameters (2)

Behaviour is qualitatively different. A fixed point instead of a limit cycle.

Exponential growth

Limited growth

Exponential decay

Oscillation

Why are PP models useful?
• They model the simplest interaction among two systems and describe natural patterns
• Repetitive growth-decay patterns, e.g.,
• World population growth
• Diseases

time

Fibonacci’s rabbits
• Around the year 1200, the italian mathematician Fibonacci asked himself the following question.
• I start with a single newborn rabbit-pair. Mature rabbit pairs create offspring every month. Rabbit pairs are mature from the second month. How many rabbits do I have after t months? (assuming rabbits live forever)
A second order differential equation
• Let Kn be the number of rabbits after n month. Then it must hold that
• Kn = Kn-1 + Kn-2.
• Because all pairs of rabbits that lived in Kn-1are still alive in Kn, and all pairs that were alive in Kn-2 produced a pair of offspring.

Linear homegenous second order difference equation

Solving the 2nd order difference equation
• Kn - Kn-1 - Kn-2 = 0.
• Let’s guess a solution once again…
• Kn = k λn.
• Then it must hold that
• k λn+2 -k λn+1 -k λn = 0.
•  k λn (λ2 -λ- 1)= 0.
Solving the 2nd order difference equation
• There are two solutions:
• λn = 0 for all n = 1,2,….
• (λ2 -λ- 1)= 0.
•  [1 +/- √ (1- 4*1*-1)]/2
• = [½ +/- ½ √5 ].
Solving the 2nd order difference equation
• Thus there are 2 solutions:
• λ1 = ½ + ½ √5 and λ2 = ½ - ½ √5.
• The differential equation
• Kn - Kn-1 - Kn-2 has solution
• Kn = k1 * (½ + ½ √5)n + k2 * (½ - ½ √5)n
Fibonacci’s starting conditions
• Fibonacci started with newborn rabbits, thus K0 = 1, K1=1.
• 1= k1*(½ + ½ √5)0 + k2*(½ - ½ √5)0 (1)
• 1 = k1*(½ + ½ √5)1 + k2*(½ - ½ √5)1 (2)
• (1) can be rewritten to k1= 1 - k2
Fibonacci’s starting conditions
• From (2)
• 1 = k1*(½ + ½ √5)1 + k2*(½ - ½ √5)1
• and the rewritten (1)
• k1=1 - k2
• it follows that
• 1 = (1 - k2)*(½ + ½ √5) + k2*(½ - ½ √5) (3)
Counting rabbits....
• (3) Can be rewritten:
• 1 = (1 - k2)*(½ + ½ √5) + k2*(½ - ½ √5) 
• 1- ½ - ½√5 = - k2 * (½+½√5) + k2 * (½-½√5)
• ½ - ½ √5 = - √5 k2  k2 = ½ - ½ /√5
Still counting rabbits...
• From k2 = ½ - ½ /√5 and k1=1 - k2
• it follows that
• k1=1 – (½ - ½ /√5) = ½ + ½ /√5
• Thus the solution to the difference equation is
• Kn = (½+½ /√5) * (½+½ √5)n + (½-½ /√5) * (½-½ √5)n
Two examples from research
• Modelling foraging
• Decaying step-lengths in foraging
• Modelling semantic network dynamics
• Growth of knowledge
Foraging patterns in nature

Random walk

Levy flight

Universal foraging behaviour
• Foraging behaviour in sparse food environments is characterised by Lévy-flights with  2 is performed by:
• Albatrosses
• foraging bumblebees
• Deer
• Amoebas
• In dense food environments  > 3 (random walk)

lemon

gravitation

pear

Newton

apple

Einstein

orange

Growth of knowledgesemantic networks
• Average separation should be small
• Local clustering should be large
Clustering coefficient and Characteristic Path Length
• Clustering Coefficient (C)
• The fraction of associated neighbors of a concept
• Characteristic Path Length (L)
• The average number of associative links between a pair of concepts

lemon

gravitation

pear

Newton

apple

Einstein

orange

Example

Type of network

k

C

L

Fully-connected

N-1

Large

Small

Random

<<N

Small

Small

Regular

<<N

Large

Large

Small-world

<<N

Large

Small

Network Evaluation

1

C(p)/C(0)

L(p)/L(0)

0

p

0.0

0.00001

0.0001

0.001

0.01

0.1

1.0

Data set: two examples

APPLE

PIE (20)

PEAR (17)

ORANGE (13)

TREE ( 8)

CORE ( 7)

FRUIT ( 4)

NEWTON

APPLE (22)

ISAAC (15)

LAW ( 8)

ABBOT ( 6)

PHYSICS ( 4)

SCIENCE ( 3)

Small-worldlinessWalsh (1999)
• Measure of how well small path length is combined with large clustering
• Small-wordliness = (C/L)/(Crand/Lrand)

5

4.5

4

3.5

3

2.5

2

1.5

1

0.5

0

Semantic

Network

Cerebral

Cortex

Caenorhabditis

Elegans

Some comparisons

Small-Worldliness

What causes the small-worldliness in the semantic net?
• TOP 40 of concepts
• Ranked according to their k-value (number of associations with other concepts)
More complicated interactions
• Clinton established the Giant Sequoia National Monument to protect the forest from culling, logging and clearing.
• But many believe that Clinton’s measures added fuel to the fires.
• Tree-thinning is required to prevent large fires.
• Fires are required to clear land and to promote new growth.
• “Smokey Bear did too good a job,” said Matt Mathes, a Forest Service spokesman. “It was a well-meaning policy with unintentional consequences.”

Fire is dangerous when caused by

surrounding bushes

Fire is needed to clean area and

to open the seeds of the Sequoia

Predator versus Prey?
• Fire acts as “prey” because it is needed for growth
• Fire acts as “predator” because it may set the tree on fire
• Tree acts as “prey” for the predator
• If trees die out, the predator dies out too
Biodiversity

“Human alteration of the global environment has triggered the sixth major extinction event in the history of life and caused widespread changes in the global distribution of organisms. These changes in biodiversity alter ecosystem processes and change the resilience of ecosystems to environmental change. This has profound consequences for services that humans derive from ecosystems. The large ecological and societal consequences of changing biodiversity should be minimized to preserve options for future solutions to global environmental problems.”

F. Stuart Chapin III et al. (2000)

Model predictions
• Predicted relative change in biodiversity in 2100
• T = Tropical forest
• G = Grasslands
• M = Mediterranean
• D = Dessert
• N = Northern forests
• B = Boreal forests
• A = Arctic
Consequences of reduced biodiversity

"...decreasing biodiversity will tend to increase the overall mean interaction strength, on average, and thus increase the probability that ecosystems undergo destabilizing dynamics and collapses."

Kevin Shear McCann (2000)

“equilibrium” states
• Complex systems are assumed to converge towards an equilibrium state.Equilibrium state: two (or more) opposite processes take place at equal rates

VIDEO

unstable

stable

Deterministic versus Stochastic Models
• Deterministic model
• The state at time t+1 is fully determined by the state at time t
• Stochastic model
• The state at time t+1 is partially determined by the state at time t, and partially by noise
What is noise?
• Noise is randomness
• Noise is unpredictable
• except for statistical descriptors such as mean, standard deviation, etc.
• Example
• A die generates random numbers ranging from 1 to 6
• At any time t the number generated by the die is unpredictable
• The probability of a certain number occurring is predictable
What is determinism?
• The Verhulst equation is an example of a deterministic model
• The value at time n+1 is fully determined by the value at time n
Fundamental question
• Given perfect knowledge about the positions and velocities of all particles in the universe, can we predict the future state of the universe?
• YES if the universe is deterministic
• NO if the universe is stochastic
Turbulence
• “Chaotic” behaviour of many-particle systems
• Was poorly understood until chaos theory emerged
• Was known to arise from non-linear interactions
Strange attractors
• The state of chaotic systems does not converge onto a point or limit cycle but on a chaotic attractor
• The same state never reoccurs because that would lead to periodicity
• Very small deviations in starting conditions are amplified
Self-similarity across scales
• Fractals
• Coastlines
• Mountains
• Etc.
Deterministic Chaos
• Implication for deterministic models
• Prediction of future states is limited by the sensitivity to initial conditions
• Hence, despite determinism, future states cannot be predicted due to inevitable measurement errors
• Measurement errors may be very small, but always larger than zero
• Errors are amplified due to the chaotic trajectory
• E.g., Lorentz equations

Fundamental question (reprise)

• Given perfect knowledge about the positions and velocities of all particles in the universe, can we predict the future state of the universe?
• NO if the universe is deterministic
• NO if the universe is stochastic
Predicting War
• Collapse of nations, three factors
• Infant mortality
• Level of democracy
• But is the course of history predictable?
• Small changes can have large effects
Controlling Physiological Chaos
• Nerve tissue: nonlinear coupled system
• Congestive Heart Failure (CHF)
• Chaos is good for you
Chaos and Chaotic Models in Neurosystems

The Freeman-Skarda model of the Olfactory Bulb

Conclusions
• In this lecture, we have been modeling dynamic systems.
• For the studied examples, we have always been able to model them using a linear equation, that we could solve analytically.
• Hence no algorithms were required!
• In general, there are many systems of differential equations for which analytical solutions are not known, and for which more algorithmic approaches are used to solve them.