PAK346 Framework workshop

1. PAK346 Framework workshop Marco Huigen (P8)

2. Content • Introduction (20 min.) • Expert N: needs and requirements • Putting this workshop in PAK-perspective (30 min.) • P8: Overview • The world according to me • Framework thinking: Living between worlds (30 min. - 1 hr) • DANUBIA theoretical (1 hr) • DANUBIA practical (45 min.)

3. The PAK346 P8 project

4. Objectives and responsibilities of P8 Facilitate project communication on integration Integration-on-Demand approach Facilitate technical implementation of the ‘overall land system model’ Danubia framework Facilitate ‘Integrative simulation experiments’ Experimental frame Data Management Simulation input and output Model coupling and data management - Project P8

5. 2 Facilitate technical implementation P8 will support the model integration by using the Danubia framework* „Framelets“ (knowledge manager, model manager, experiment manager) Every project group is responsible for programming its model, its model wrapper and its model interfaces P8 assists in finding and designing the required components (in the IoD approach) Model coupling and data management - Project P8 * Open source, clearance will be requested from developer group

6. 3 Facilitate ‘Integrative simulation experiments’ P8 will set up and maintain the integration environment for running simulations (without dynamic P1). Framework control (e.g. model-sequencer, time-controller) Maintain calculation nodes hardware Data storage capacity Model coupling and data management - Project P8

7. 4 Data Management of simulation input and output P8 will set up and maintain a Data Management System (DMS) for simulation input and output As much as possible and useful simulation data will be stored in universal database format (approachable with SQL) Allows for analyzing scenarios Maintain DMS software (MySQL) and hardware Model coupling and data management - Project P8

8. Framework Thinking

9. DANUBIA framework • If your model-coupling project: • Requires multiple model-coupling configurations • Has long time horizon • Requires re-usability • Requires (scientific and technical) adaptability • Scientific functionality model could (easily) change • Technical functionality might change • … • Then: Use a computational architecture

10. DANUBIA Theoretical

11. DANUBIA: System Structure and Framework Technology Research group “Computer Science” Ludwig-Maximilians-Universität München Rolf Hennicker, Stephan Janisch Andreas Kraus, Matthias Ludwig

12. The DANUBIA system is designed as an integrativeplatform for • coupled simulations of various models of natural-science and socio-economic disciplines • analysis of transdisciplinary effects of mutually dependent processes (including "acting" entities like households, farmers, ...) • integrates 17 simulation models developed by the Danube groups • runs on a computer cluster with more than 32 processors • the simulation models run in parallel and exchange data at run-time

13. System Development Principles • Common graphical modeling language (UML) used by all project groups for the documentation of interfaces, concepts and designs • Framework technology to facilitate the integration of simulation models • Object-oriented approach in all development phases (system analysis, design and programming) • Formal software engineering methods to verify critical parts of the integrated DANUBIA system

14. DANUBIA – Architecture

15. The Framework Idea • Extract common properties and rules which hold for all DANUBIA models and implement them in a general abstract pattern (template). • The model developer must only implement the open pieces of the template by model specific realizations.

16. The DANUBIA Developer Framework Common properties of all DANUBIA models <<extends>> <<extends>> <<extends>> <<extends>>

17. Common Behaviour of all DANUBIA Models getData compute commit run() getData() compute() commit()

18. DeepActor Models • Deep actor models integrate into their simulations decision-making entities, called actors (e.g. households, water suppliers, farmers). • Any actor has a repository of potential plans that the actor can execute (initial plans) . • In each computation step an actor decides which of the initial plans should actually be executed (active plans). • To support decisions each actor has • sensors through which he can observe the "environment" and • a history to remember previous decisions

20. run() getData() compute() commit() getData() compute() commit() modelGetData() … modelGetData() … query modelGetData options modelPreCompute rating modelPostCompute execute filter

21. A model is blocked (waitForGetData) if the coordination condition for getData is not satisfied (similarly, waitForCommit). • The correctness of the coordination can be formally verified with model checking techniques.

22. Conclusion: DANUBIA framework • Common modelling language (UML) has been proved to provide a valuable tool for integrative system modelling. • Framework technology • supports model developers to integrate (conceptually and technically) their simulation models into the overall system structure of DANUBIA • provides general rules (templates) which support the trustability and quality of the system • Additional provisions for software quality:Formal verification of the correct coordination of concurrently running simulation models according to their local time scales.

23. Conclusion: DANUBIA framework • Uses Object Oriented technology • Gives and keeps a very clear, transparent overview of complex models/components • Completely written in JAVA • Each model/component is packed into a compiled jar-file • Component based • Each component can be plugged on and off in a simulation • Uses JAVA R(emote)M(ethod)I(invocation) technology • Individual model components may run on different computers and platforms • Models and Components (=client) log on to the framework server. The server controls the data exchanges and does the time-management

24. Conclusion: DANUBIA framework • Badly chosen name • Framework is generically applicable, not only for Danube area • Framework is spatially explicit • By using configuration files users may set up any ‘simulation environment’ based upon ASCII grid-files. • The extent and resolution (= proxel size) are user-defined in the configuration. • Framework allows for partial area testing. • Framework is temporally explicit • In each model configuration file users indicate the model time step. • The time scale is flexible (can be expressed in seconds, min, hours, days, months, years). • Framework does not support itself dynamic coupling within a time step, but due to time scale flexibility this can be overcome. • Framework allows for distributed and parallel processing (task based) • Uses JAVA RMI (OS independent)

25. Conclusion: DANUBIA framework • Framework has a high learning curve • Due to its many functionalities and advanced level of programming, users will find it difficult to understand the ‘inner-working’. • Framework is transparent • The architecture, although evolved drastically over the years, has a very logical built-up • Framework requires models to have ‘plug-points’ • In a simulation run the framework controls all flows and needs therefore control over the components. • Framework supports model development and ‘testing’ very well. • Has extensive logging capacities and j-unit support • Framework is open source • Although open source, it is unlikely that our project will be able to structurally adapt the code due to lack of required skills

26. DANUBIA Practical

27. Our DeepFarming model consists of the following components: • 1. ACRE: An agricultural optimization model • Programmed in GAMS, Calculates on district level, Calculation annual interval, Calculation provides input for agents to create a new land use plan • 2. ACRE wrapper: a java program controlling the GAMS model • Responsible for file maintenance; zips, deletes and moves files, • Responsible for data maintenance; reads and writes GAMS from/to MySQL. • 3. DeepFarming Core model: a deepactors model implementation • Responsible for data maintenance; reads and writes our input and output from and to a MySQL database. • Responsible for all data interactions with other models in the simulation environment. • 4. DeepFarming District models: a deepactors model implementation • Responsible for actual calculations of our agents. • Every model represents a set of districts and has around 6000 farm actors. • Because of this sub-model architecture, a higher level of distributed calculation is achieved. • 5. MySQL database: a Relational Data Management System • 6. mydfm connector: a controller for unified access to the mysql RDMS • 7. Several tools for data modification and analysis

28. Deep Farming • In the Danubia framework our DeepFarming model directly interacts with the models • Biological (crop growth) • Soil (nutrient flows) • WaterSupply (animal drinking water, irrigation) • Meteorology (weather) • DeepFarming actors: • Total of more than 50.000 actors ‘in sim’ • Each actor ‘lives’ on a proxel • 28 farm system types (e.g. CashSugMaize, MeatBreedSum) • Each actor has statistically based size of land (consisting of arable and/or grassland) • Each farm system type has unique crop rotation system (17 land use types are represented) • Each farm system type has unique combination of husbandry (9 animal types are represented)

29. Daily decisions: • Planting (weather forecast, vegetation time demands) • Fertilizing (Crop stage, weather conditions) • Harvesting (Crop stage, weather conditions) • Cutting (biomass increase, weather conditions) • Irrigate Yr1 Yr2 • Run ACRE • Make district plan per year (after crop silage is planted) • Disaggregate plan of ACRE to Agents Deep Farming • Per simulated day our actors may perform various actions. Whether these actions take place depend on: • Actors’ plan • Environmental sensors (weather conditions) • Crop conditions

30. Deep Farming The farm actors derive their crop rotation plan based on their interpretation of the ACRE output

31. Deep Farming • Per crop advanced farm management decision rules have been programmed, taking into account weather conditions, environmental conditions, crop requirements and conditions. • e.g. • Planting depends on: • vegetation period in days, • vegetative rest, • temperature sum, • minimum germination temperature, • „Temperaturgrenzwert“ for germination and • “Befahrbarkeit” of land

32. heading germination temperature Vernalisation Deep Farming An actor dynamically adjusts its plan based on experiences regarding weather in combination with crop requirements. • Requirements of winter wheat: • Germination temperature • Vernalisation • Enough rainfall during heading Weather forecast Is it time to drill winter wheat?