Modelling Hybrid Control Systems with Behaviour Networks

1. Modelling Hybrid Control Systems with Behaviour Networks Pierangelo Dell’Acqua Anna Lombardi Dept. of Science and Technology - ITN Linköping University, Sweden L. M. Pereira Centro de Inteligência Artificial - CENTRIA Universidade Nova de Lisboa, Portugal • September, 2005 • September ‘05 • ICINCO 05, Barcelona Spain

2. Problem • We present an approach to model adaptive, dynamic hybrid control systems based on behaviour networks. • We extend/modify behaviour networks to make them self-adaptive and dynamic. • The approach is general, reconfigurable, and suitable for environments that are dynamic and too complex to be entirely predictable.

3. Outline • 1. Hybrid control systems • 2. Behaviour networks • 3. Extended/modified behaviour networks • 4. Modelling hybrid control systems • 5. Discussion and future work

4. 1. Hybrid control systems • Closed-loop configuration • Controller: discrete-time system • Interface • Plant: time-continuous system to be • controlled • (possibly traditional controllers) Controller x(tk) r(tk) Generator Actuator x(t) r(t) Plant

5. Plant: described by differential • equations • x(t) - state vector • r(t) - input vector Controller x(tk) r(tk) Generator Actuator x(t) r(t) Plant

6. Controller: described by difference • equations • s(tk+1) =  ( s(tk), x(tk) ) • r(tk) =  ( s(tk) ) • x(tk) – plant events • r(tk) - actions Controller x(tk) r(tk) Generator Actuator x(t) r(t) Plant

7. Interface • Generator: converts continuous-time output of the plant into symbols • x(t) x(tk) • Actuator: receives symbols (actions) from the controller and converts them into an input to the plant • r(tk)  r(t) Controller x(tk) r(tk) Generator Actuator x(t) r(t) Plant

8. 2. Behaviour networks • Introduced by P. Maes in ‘89 for action selection in dynamic and complex environments where the system has limited computational and time resources. • A behaviour network is a network composed of specific competence modules (rules) which activate and inhibit one another along the links of the network. • The activation/inhibition dynamics of the network is guided by global parameters. • Competence modules cooperate in such a way that the network as a whole functions properly. • This architecture is distributed, modular, robust and has an emergent global functionality.

9. 3. Extended/modified behaviour networks • To model hybrid control systems, we extended behaviour networks to allow for: • rules containing variables; • internal actions; • integrity constraints, and • modules (sets of atoms and rules).

10. An extended behaviour network consists of 5 modules: R - a set of rules formalizing the behaviour of the network P - set of global parameters H - internal memory C - set of integrity constraints G - set of goals • We call the state of the network the tuple S=(R,P,H,C,G). • We assume given a module Math formalizing the axioms of elementary mathematics.

11. Language (simplified) • Atom takes the form: • m:X meaning that X belongs to module M whose name is m • m÷X meaning that X does not belong to M • Sequence of atoms: • Atom1 , Atom2 , . . . , Atomn (n  0) • # denotes the empty sequence • Goal is a sequence of atoms. • It expresses a condition to be achieved. • Integrity constraint is a sequence of atoms. • It expresses a condition that must not hold.

12. Rule is a tuple of the form: < prec; del; add; action;  > • prec - preconditions • del, add - represent the internal effect of the rule • action - atom representing the external effect of the rule •  - level of strength of the rule • Variables in rules are universally quantified: • < h÷on, e:temp(x), math:x<20; # ; h:on; heating(on); 0.5 > • E denotes the environment

13. A substitution is a finite set of bindings: variable/expression • A substitution  can be applied to an expression X, written as X, by simultaneously replacing any variable v in X with t, for every binding v/t in . • if X = h:q(x,y,z,c) and  = {x/3, z/6}, then X= h:q(3,y,6,c) • Two expressions X and Y are unifiable (i.e. two-way pattern matched), written as XY, if there exists a substitution  such that X=Y. • Substitutions can be applied to rules: • if r = < h÷on, e:temp(x), math:x<20; # ; h:on; heating(on); 0.5 > and •  = {x/12}, then • r = < h÷on, e:temp(12), math:12<20; # ; h:on; heating(on); 0.5 >

14. Executable rules • At every state, a rule in R has to be selected for execution. • A sequence L of atoms is true at a state S iff: • for every m:X in L, XM and • for every m÷X in L, there exists no  for which XM. • A state S is safe if there exists no  that makes an integrity constraint in C true at S.

15. r • Applying a rule r=<prec; del; add; action; > • S=(M1,M2,M3,M4,M5) r(S)=(M’1,M’2,M’3,M’4,M’5) • every M’i is obtained from Mi : • by removing X for every mi:X in del, and • by adding Y for every mi:Y in add. • A rule r is executable at S iff: • prec is true at S, • r(S) is safe, and • action is an atom with no variables.

16. Rule selection • Rules in a network are linked by three types of links. • Let x and y be rules. • Successor link x y • for every m:X in the add list of x and m:Y in the prec list of y such that XY • Predecessor link x y • for every successor link from y to x • Conflict link x y • for every m:X in the prec list of x and m:Y in the del list of y such that XY

17. Rules use these links to activate/inhibit each other. • Forward propagation • There is input of activation energy coming from S and E towards rules whose preconditions partially match them. • Backward propagation • There is input of activation energy from G towards rules whose add lists partially match G. • Inhibition • There is inhibition by the goals that have been achieved. They remove some activation energy from the rules that would undo them.

18. Rules also inhibit and activate each other along the links in the network. • The mathematical model is based on several global parameters in P that are used to tune the spreading of activation energy through the network: •  - threshold of rules to become active, •  - amount of energy that a true atom injects into the network, •  - amount of energy that a goal injects into the network, •  - amount of energy that achieved goals take away from the network, • ... etc.

19. Let r be a rule in R and  a substitution. • The rule r becomes active when: • it is executable, • its activation level overcomes , • has the greatest activation level among all other executable rules. • When an active rule has been executed, its activation level is set to 0.

20. Math H R C P CU G E Actuator 4. Modelling hybrid control systems • An adaptive, dynamic hybrid controller can be described by an extended behaviour network.

21. The basic engine of CU can be described via the following cycle: • Cycle(n, R, P, H, C, G) • Load the rules of R into CU and calculate their activation level wrt. the global parameters in P. • If one rule becomes active, then execute its internal effect, and send its external effect (its action) to the actuator. • Let R', P', H', C', G' be the modules after the execution of the rule. • Cycle with (n+1, R', P', H', C', G'). • If no rule becomes active, then lower the level of  in P. • Cycle with (n+1, R, P', H, C, G).

22. Example: artificial fish • Consider a virtual marine world inhabited by a variety of fish. • Fish are situated in the environment, and sense and act over it. • The behaviour of a fish is reduced to search and eating food, escaping and sleeping. • Actions of the fish: search, eat, escape and sleep.

23. Modelling an artificial fish • Stimuli of the fish: • hungry: • fear: • tired: • Input vector to the controller State of the controller • hungry(tk) food(tk) • x(tk) = fear(tk) s(tk) = satiated(tk) • tired(tk) safe(tk)

24. C = { } no constraints • G = { h:safe, h:satiated } • Behaviour of the fish: • < e:hungry(x), math:x>0.5, h÷food; # ; h:food; search; 0.5 > • < e:hungry(x), math:x>0.5, h:food; # ; h:satiated; eat; 0.5 > • < e:tired; # ; # ; sleep; 0.5 > • < e:fear(x), math:x>0.5; # ; h:safe; escape; 0.7 > • E - stimuli of the fish • H - internal state of the fish R =

25. 5. Future work • Implementation: XSB Prolog + Java • Extension of L to allow variables in the strength level of rules: • < h÷on, e:temp(x),math:x>20; # ; h:on; heating(on); 0.5*(20-x) > • Engineering network of behaviour networks to model complex environments • Techniques developed in Logic Programming can be employed: • - preference reasoning to enhance action selection • - genetic algorithms to tune the global parameters of the network • - belief revision to resolve cases of conflicting rules if more rules • can become active