Learning Process-Based Models of Dynamic Systems

1. Learning Process-Based Models of Dynamic Systems Hipeac 2014Ljubljana Nikola Simidjievski Jozef Stefan Institute, Slovenia

2. Introduction • Equation Discovery (ED) is a subfield of machine learning, dealing with the task of inducing scientific laws and models in form of equations from observations. • In the context of modeling system dynamics, the observations are time-series and the models take form of ordinary differential equations (ODEs) • Process-Based Modeling (PBM) is an ED approach, which integrates domain-specific modeling knowledge and data into explanatory models of the observed systems. • Using modeling knowledge formulated in a library, and observed data from the system at hand, this approach induces process-based models - an accurate, understandable and modular representation of the observed system dynamics.

3. Process-based models • Conceptual (high-level) representation of system dynamics. • Process-based models are comprised of entities and processes. • Entities and processes represent specific components and interactions observed in the system. • Entities represent the state/variable of the system. • Processes the represent the interactions between the entities. • Knowledge is represented as library of entity and process templates.

4. The task of learning process-based models • Determining the structure of the model (ODE) • Heuristic/exhaustive search over the space of suitable candidate models • Parameter estimation • finding values which minimize the difference (error) between simulated and measured (real) data • 3 Inputs : • Library(Domain specific) • Conceptual model(Problem specific) • Data(Task specific)

5. Library of domain-specific modeling knowledge Process-Based Library NatašaAtanasovaet al. , Constructing a library of domain knowledge for automated modelling of aquatic ecosystems, Ecological Modelling, 2006

6. Conceptual Model

7. ProBMoT(Čerepnalkoski, Simidjievski, Tanevskiet al.) • ProBMoT1 (Process Based Modeling Tool) Tool for complete modeling, parameter estimation and simulation of process-based models DarkoČerepnalkoskiet al., The influence of parameter fitting methods on model structure selection in automated modeling of aquatic ecosystems, EM 2012

8. The Process Library Conceptual Model Model generator

9. The Process Library ~ 30 000 Specific models structures Conceptual Model Model generator

10. The Process Library Specific models structures Conceptual Model Model generator Parameter Estimation Measurements

11. The Process Library Specific models structures Conceptual Model Model generator Parameter Estimation Parameter Estimation Measurements ~50K-100K Simulations per Structure

12. The Process Library Best Model Conceptual Model Model generator Parameter Estimation Parameter Estimation Measurements Validation Error values

13. The Process Job Library Best Model Conceptual Model Model generator Parameter Estimation Parameter Estimation Measurements Validation

14. Lets complicate the story a little bit…

15. Ensembles Ensemble T1 Learning algorithm Model M1 Training set T T2 Learning algorithm Model M2 TN Learning algorithm Model MN Improved predictive performance Better address the complexity of real systems: combination of base models for better description of observed behavior

16. Ensembles of Process-based models • Base models are homogeneous • Training data is represented as a time series. • Each base model is trained on different samples of the data (Bagging) • 2-D space of candidate models A list of base models is generated by every ensemble iteration / replica

17. Conceptual Model Model generator Parameter Estimation Library Sample of Measurements

18. Conceptual Model Model generator Parameter Estimation Library Sample of Measurements . . . . . . . . . . . . . . . . . . 100 Parameter Estimation Sample of Measurements

19. Best Model 2 Best Model 100 Best Model 1 Conceptual Model Model generator Parameter Estimation Library Sample of Measurements . . . . . . . . . . . . . . . . . . Let say 100 Validation Parameter Estimation Sample of Measurements

20. Ensembles of Dynamic Systems Best Model 1 ENSEMBLE MODEL . . . Best Model 2 . . . Best Model 100

21. Conceptual Model Model generator Parameter Estimation Library Sample of Measurements Job . . . . . . . . . . . . . . . . . . 100 Parameter Estimation Sample of Measurements

22. The “fun” stuff… .xRSL • & • (executable ="run_[JOB]##.sh") • (jobname = "Awsome_[JOB]##") • (stdout = "[JOB]##.out") • (stderr = "[JOB]##.err") • (inputfiles = ("[JOB]##.tar") • (outputfiles = ("[JOB]##.tgz") • (cputime = "30 days") • (memory = "2560") run_[JOB]##.sh • #!/bin/bash • tar -xf [JOB]##.tar • cd JOB_workingDir • java -Xms256m • -Xmx2048m • -jar PROBMOT.jar task/PROBMO_TaskSpec.xml • cd .. • tar czf [JOB]##/out/ [JOB]##/*.log

23. Case Study : Population Dynamics in Lake Ecosystem Modeling phytoplankton dynamics in lake ecosystems (1 ODE) Lake Bled (Slovenia) Lake Kasumigaura (Japan) Lake Zurich (Switzerland) Lake Walensee (Switzerland)

24. Experiments • Experimental Size • 100 Ensemble Iterations • 4 Different Lake Domains (Lake Bled, Lake Kasumigaura, Lake Zurich, Lake Walensee) • Total of 40 different modeling scenarios • ~200 Model Structures per scenario • 50000 Iterations in the Parameter Estimation Phase per structure • 1 Model (Structure Identification + Parameter Estimation) = 2-3 minutes => Total Time of the whole experiment ~ 1,000,000min or ~ 2 years We did it in ….. 3 Days Nikola Simidjievski et al., Learning ensembles of population dynamics models and their application to modelling aquatic ecosystems., EM 2014

25. Thank You Questions www.probmot.ijs.si probmot@ijs.si