Part I Information Flow Modelling: Tools and Approaches

1. Part I Information Flow Modelling: Tools and Approaches

2. INFORMATION MANAGEMENT IN THE NEW MILLENNIUM: SOME CHALLENGES Opportunities and challenges Demand for information Information theory Visualization of information · colour and information, · complexity and clarity of human perception, · use of multi-media, internet, and WWW screens, · animation for scientific visualizations, · design of efficient computer interfaces. Information based integration Information modelling tools Cyberspace, virtual reality and ars electronica

3. Of some interest here may be the fact that one of the best graphical representations of information was developed in 1869 by Charles Minard. It depicts the losses suffered by Napoleon’s army in the Russian campaign of 1812. Six variables were plotted in that graph which tells a rich coherent story with its multivariate data, far more enlightening than just a single number posed again time. Minard’s chart is still regarded as one of the best graphical representation of data ever drawn.

5. INFORMATION FLOW: ANALYTICAL MODELLING PLATFORM Introduction and the modelling approach

6. An autonomous subsystem is usually functioning in external environment which determines the decision-making process. Its knowledge could be described by the following: • characteristics of the internal environment (relationship between the actions of the elements of a system), • characteristics of the external environment (relationship between variables describing the environment and its dynamics), • the range of information about variables describing external environment. • The formal representation of the above knowledge is presented in this Chapter.

7. Energy E E1 C1 Information C The general character of energy/information replacement

8. Energy E E1=Emin Aopt Action A Action/energy dependence for given amount of information

9. VC=f(E) With this modelling tool one can extract knowledge about autonomous agents functioning in various decision situations where external environment is static. A sample of such knowledge, codified as production rules, is presented next.

10. RULE 11 IFan external environment of an autonomous agent is static, AND it is described by random variables, THEN the value of an information structure that represents the flow of information between the agent and its environment depends on interaction between agent members, correlation between random variables, and their variance. RULE 12 IF an external environment of an autonomous agent is static, AND it is described by a random variable, THEN the value of information about this variable realization is proportional to the value of its variance. RULE 13 IF an external environment of an autonomous agent is static, AND it is described by random variables, THEN full information has the value that is always greater or equal to the value of any other information structure. RULE 14 IF an external environment of an autonomous agent is static, AND there is no interaction in the internal environment, THEN it is enough to restrict the information flow only to observation; organizing an information exchange does not improve the value of a resulting information structure.

11. Modelling dynamic environment X(t) = (w1 + w2)X(t-1) - w1w2X(t-2) + m(t)

12. w2 B 1 -1 A 0 1 w1 E C -1 D A independent stochastic process BD first order autoregressive process EC first order autoregressive process shaded area inside the square stable stochastic process area outside the square explosive stochastic process External environments of different character on the (w1,w2) plain.

13. RULE 24 IF an external environment of an autonomous agent is described by stochastic process, AND information is not delayed, THEN the value of full information is, for each decision situation, greater than the values of other possible information structures. RULE 25 IF an external environment of an autonomous agent is described by stochastic process, AND information is not delayed, AND there is no interaction in the internal environment, THEN there is no need for communication inside the agent. RULE 26 IF an external environment of an autonomous agent is described by stochastic process, AND information is not delayed, THEN the more uncertain the realizations in the external environment the bigger the value of the information about these realizations.

14. INFORMATION FLOW: SOFT MODELLING PLATFORM Introduction to qualitative modelling and simulation Qualitative vs. quantitative modelling

15. Real-life system Real-life behaviour mathematical modelling mathematical analysis Quantitative description Analytic solution relaxation qualitative modelling qualitative simulation Qualitative model Qualitative behaviour Qualitative vs. quantitative representation

16. In a broader context, there are following motivations for developing qualitative models of real-life systems: (a) to provide simpler computational mechanisms than those already existing, (b) to provide s description for systems where traditional methods are ineffective, (c) to provide modelling approach that mimics more closely our common sense and intuition, (d) to provide modelling methods for reasoning with partial, uncertain or incomplete information, and for effective explanation facilities.

17. Qualitative modelling in static environment The are the following functional constraints (denoted as M) between the corresponding state variables: (i) M (external_environment; information_structure) (ii) M (external_environment; value_of_information_structure) (iii) M (information_structure; value_of_information_structure) (iv) M (internal_environment; value_of_information_structure)

18. external_environment M M internal_environment information_structure M M value_of_information_structure Simple abstract representation for the evaluation of information flow

19. Behaviour Structure  X2, s2 X1, s2   M+ M+   r q VC +   M+ M+  0 r 1 0 VC q VALUE_OF_INFORMATION_STRUCTURE Structure and behaviour for information structure C2

20. RULE 2 IF an external environment of an autonomous agent is static, AND it is described by random variables, AND there is an interaction between agent elements and correlation between random variables, THEN the value of decentralised information structure (created only by observation) increases with increasing correlation and interaction

21. Soft modelling for informational flow Signed directed graphs Decision tree classifiers Connectionist systems – neural networks Fuzzy logic

22. Signed directed graph (SDG) is one of the simplest models that can be used for qualitative reasoning. Any SDG can be viewed as signed digraph and as such consists of three components: node, directed branch, and sign. A node corresponds to a process variable, branch represents the influence between two nodes, and the sign distinguishes between influences of positive and negative character. The states of variables are represented by qualitative values “+, -, or 0” indicating whether the current value of the variable is “higher than, lower than, or equal to” its nominal value.

23. w + + d L1 + + + + LV + + q s + + + _ a L2 SDG model of informational balance of an agent

24. w + + d LV + _ a SDG model - final simplification

25. (i) solution pattern for a positive change in d (ii) solution pattern for a positive change in w (iii) solution pattern for a positive change in a

26. Decision tree classifiers are used successfully in many diverse areas. Their most important feature is the capability of capturing descriptive decision-making knowledge from the supplied data. They are also attractive for the following reasons: • capability of generating arbitrarily complex decision boundaries from a given set of training samples, • short training time allowing for frequent trainings, • ability of revealing information about the problem they are applied to.

27. internal_environment independent_actions dependent_actions do_not_exchange_information -1 < correlation < 1 true false exchange_information do_not_exchange_information Decision tree classifier for agent decision-making

28. Last Figure: A decision tree that classifies agent decision-making concerning the flow of information in static environment as either exchange_information or do_not_exchange_information The following rules can be delivered from the tree: Rule 1 IF an external environment of an agent is static AND it is described by random variables AND there is no interaction in the internal environment THEN communication (exchange of information) between agent elements is not necessary Rule 2 IF an external environment of an agent is static AND it is described by random variables AND there is interaction in the internal environment AND the relationship between variables describing the external environment is of statistical character THEN exchange of information between agent elements should be organised

29. NEURAL NETWORKS Adaptation, or the ability to learn, is the most important property of neural networks. A neural network can be trained to map a set of input patterns onto a corresponding set of output patterns simply by means of exposure to examples of the mapping. This training is performed by gradually adapting the internal weights of the network, so as to reduce differences between the actual network outputs (for a given set of inputs) and the desired network outputs. Neural networks which learn mappings between sets of patterns are called mapping neural networks. A key property of mapping networks is their ability to produce reasonable output vectors for input patterns outside of the set of training examples. The above is especially important in areas such as discussed in this Chapter, i.e. areas for which it is possible to develop only a very limited number of IF...THEN rules and thus also to make inferences only for a very limited number of decision situations.

35. decisions concerning the flow of information observation exchange of information correlation (R) dynamics (T) interaction (I) delay (D) process (W) characteristics of an agent A three layer 5-10-2 neural network

36. Fundamentals of fuzzy sets and fuzzy logic • A new theory, its applications and modeling power • A new theory extending our capabilities in modeling uncertainty • Fuzzy set theory provides a major newer paradigm in modeling and reasoning with uncertainty. Though there were several forerunners in science and philosophy, in particular in the areas of multivalued logics and vague concepts, Lotfi A. Zadeh, a professor at University of California at Berkeley was the first to propose a theory of fuzzy sets and an associated logic, namely fuzzy logic (Zadeh, 1965). Essentially, a fuzzy set is a set whose members may have degrees of membership between 0 and 1, as opposed to classical sets where each element must have either 0 or 1 as the membership degree—if 0, the element is completely outside the set; if 1, the element is completely in the set. As classical logic is based on classical set theory, fuzzy logic is based on fuzzy set theory. • Zadeh, L.A. (1965): Fuzzy sets, Information and Control 8(3):338–353.

37. Major industrial application areas The first wave: Process control The first industrial application of fuzzy logic was in the area of fuzzy controllers. It was done by two Danish civil engineers, L.P. Holmblad and J.J. Østergaard, who around 1980 at the company F.L. Schmidt developed a fuzzy controller for cement kilns. Their results were published in 1982 (Holmblad & Østergaard, 1982). Their results were not much notice in the West, but they certainly were in Japan. The Japanese caught the idea, and applied it in an automatic-drive fuzzy control system for subway trains in Sendai City. The final product was extremely successful, and was generally praised as superior to other comparable systems based on classical control. This success encouraged a rapid increase in the Japanese’s interest in fuzzy controller during the late 1980s. This led to applications in other areas, like elevator control systems and air conditioning systems. In the early 1990s, the Japanese began to apply fuzzy controller in consumer products, like camcorders, washing machines, vacuum cleaners, and cars. The Japanese success led to increased interest in Europe and the US in fuzzy controller techniques. L.P. Holmblad and J.J. Østergaard (1982): Control of a cement kiln by fuzzy logic. In M.M. Gupta and E. Sanchez, Eds., Fuzzy Information and Decision Processes, North-Holland, New York, pp. 389–399.

38. The second wave: information systems The second wave of fuzzy logic systems started in Europe in the early 1990s, namely in the area of information systems, in particular in databases and information retrieval. The first fuzzy logic based search engine was developed by the author in collaboration with professor R.R. Yager, Machine Intelligence Institute, US. It was aimed for application netbased commerce systems, namely, at that time the only in the world, the French Minitel. It was first demonstrated to the public at the Joint International Conference of Artificial Intelligence in 1992 in Chambery, France. In 1999, the technique was adopted by the Danish search engine Jubii. The several ideas and results applied in the technique were published; see, for instance, (Larsen & Yager, 1993, 1997). Internet and the Web gave new interest to application of fuzzy logic technology. In the net based society we have an enormous amount of information and knowledge electronic accessible for decision makers and human in general. Much of this information is inherently uncertain—lack of precision, vague concepts, more or less reliable information, etc. On the other hand, to be useful, users must be able to utilize it, despite the uncertainties. Larsen, H.L., and Yager, R.R.: The use of fuzzy relational thesauri for classificatory problem solving in information retrieval and expert systems. IEEE J. on System, Man, and Cybernetics 23(1):31–41, 1993. Larsen, H.L., and Yager, R.R.: Query Fuzzification for Internet Information retrieval. In Dubois, D., Prade H., and Yager, R.R., Eds., Fuzzy Set methods in Information Engineering: A Guided Tour of Applications, John Wiley & Sons, pp. 291–310, 1997. Henrik

39. Fuzzy logic and fuzzy sets Classical set theory The set (concept, predicate) A is characterized by a membership function m: X → {0, 1} μ(x)=1 x€A μ(x)=0 x∉A

40. Fuzzy set theory Set A has no sharp border line. It is a fuzzy subset of X, characterized by a membership function μ: X → [0, 1] μ(x) is the element x’s degree of membership in A Example 1: A = tall Example 2: A = about the value a

41. Fuzzy Sets and Membership Functions • Fuzzy sets describe vague concepts • Fuzzy sets admits partial membership • The degree of belonging to a fuzzy set is described by the Membership Function (MF) • MF maps an input (crisp) value into a membership value

42. A B Logical Operations with Fuzzy Sets Fuzzy Sets A, B Fuzzy AND Fuzzy NOT Fuzzy OR

43. Why Fuzzy Logic ? • FL is conceptually easy to understand • The concepts behind the successful implementation of a fuzzy expert are natural and very simple • FL is flexible • Modularity can be easily implemented in a fuzzy logic framework. Rules can be add on top of existing knowledge-base • FL is tolerant of imprecise data • Everything is imprecise if we look close enough. Fuzzy logic builds on the process rather than on the precision • FL can be built on top of experience of experts • Expertise built over years by planetary scientists, geologists, astrobiologists etc. can be easily incorporated into the system • FL is based on natural language • The basis for fuzzy logic is the basis for human communication

44. FUZZY LOGIC IMPACT Updated figures regarding the number of papers in the INSPEC and MATH.SCI.NET Databases in which fuzzy appears in title. This information was compiled by Camille Wanat, Head, Engineering Library, UC Berkeley. INSPEC - "fuzzy" in the title 1970-1979: 569 1980-1989: 2,403 1990-1999: 23,210 2000-present: 21,147 Total: 47,329 MathSciNet - "fuzzy" in the title 1970-1979: 443 1980-1989: 2,465 1990-1999: 5,487 2000-present: 5,504 Total: 13,899

45. IMPACT …….. JOURNALS (“fuzzy” or “soft computing” in title) 1. Fuzzy Sets and Systems 2. IEEE Transactions on Fuzzy Systems 3. Fuzzy Optimization and Decision Making 4. Journal of Intelligent & Fuzzy Systems 5. Fuzzy Economic Review 6. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 7. Journal of Japan Society for Fuzzy Theory and Systems 8. International Journal of Fuzzy Systems 9. Soft Computing 10. International Journal of Approximate Reasoning--Soft Computing in Recognition and Search 11. Intelligent Automation and Soft Computing 12. Journal of Multiple-Valued Logic and Soft Computing 13. Mathware and Soft Computing 14. Biomedical Soft Computing and Human Sciences 15. Applied Soft Computing

46. IMPACT…… APPLICATIONS The range of application-areas of fuzzy logic is too wide for exhaustive listing. Following is a partial list of existing application-areas in which there is a record of substantial activity. 1. Industrial control 2. Quality control 3. Elevator control and scheduling 4. Train control 5. Traffic control 6. Loading crane control 7. Reactor control 8. Automobile transmissions 9. Automobile climate control 10. Automobile body painting control 11. Automobile engine control 12. Paper manufacturing 13. Steel manufacturing 14. Power distribution control 15. Software engineering 16. Expert systems 17. Operation research 18. Decision analysis 19. Financial engineering 20. Assessment of credit- worthiness 21. Fraud detection 22. Mine detection 23. Pattern classification 24. Oil exploration 25. Geology 26. Civil Engineering 27. Chemistry 28. Mathematics 29. Medicine 30. Biomedical instrumentation 31. Health-care products 32. Economics 33. Social Sciences 34. Internet 35. Library and Information Science

47. FUZZY LOGIC IMPACT: SIEMENS EXAMPLE SIEMENS: * washing machines, 2 million units sold * fuzzy guidance for navigation systems (Opel, Porsche) * OCS: Occupant Classification System (to determine, if a place in a car is occupied by a person or something else; to control the airbag as well as the intensity of the airbag). Here FL is used in the product as well as in the design process (optimization of parameters). * fuzzy automobile transmission (Porsche, Peugeot, Hyundai)

48. INFORMATION FLOW AND INTEGRATION PROBLEM Information is seen as one of the main resources that systems analysts try to use in an optimal way. In this Chapter we show how this resource can be used in integration issues. We introduce the problem of information based integration, propose a solution, and illustrate the proposed solution with an example.

49. The essence of information based integration problem can be formulated as below: Given the informational inputs and outputs of autonomous subsystems, find the overall system being designed that meets the desired functions and is integrated through the flow of information.

50. integrated system model base of autonomous subsystems Illustrative example of a three level hierarchical tree of the bottom-up integration process