1 / 26

ISIP: status & plans B.Guillerminet, G.Manduchi

ITM general meeting 2007. ISIP: status & plans B.Guillerminet, G.Manduchi. Infrastructure & Software Integration Project. ISIP belongs to the European Integrated Tokamak Modelling project 15 members, 6 laboratories:

bcostillo
Download Presentation

ISIP: status & plans B.Guillerminet, G.Manduchi

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ITM general meeting 2007 ISIP: status & plansB.Guillerminet, G.Manduchi ITM General Meeting Garching 17-21 September 2007

  2. Infrastructure & Software Integration Project • ISIP belongs to the European Integrated Tokamak Modelling project • 15 members, 6 laboratories: • M. Airaj, J-F Artaud, Y. Buravand, A. Duarte, B. Guillerminet, P. Huynh, F. Iannone, F. Imbeaux, B. Knaepen, G. Manduchi, A. Neto, V. Pais, D. Robson, J. Santos, J. Storrs • G.Bracco(?) • In charge of the infrastructure (soft + hard) for the ITM: • Code platform: framework, editor, … • Data handling: data structure, data storage & access, … • Tools: version handling, portal, … • Hardware: gateway, GRID/HPC, … • Many thanks to all the contributors • A lot of works has been done since 2.5 years • ITER IM workshop: confrontation with others simulation approaches • Major issue: • 1st ITM framework & gateway will be released in 3 months • Question: Are we ready??? => ISIP status & plans ITM General Meeting Garching 17-21 September 2007

  3. Outline • What will be provided? Current status & short term plan (=> Jan08) • Computing infrastructure • Applications & tools • Codes • Data structure, access & storage • How does all that work? • Summary: are we ready? • Plans • Mid-term (2008) • Long term (2009-2010) ITM General Meeting Garching 17-21 September 2007

  4. Lab. Lab. user login user login Lab. user login Architecture resources applications Gateway Simulation editor Data servers: MDS+, Pub-Sub, HDF5 Workflow editor portal Clusters: JET, IPP, … Workflow engine GRID EGEE Codes: Transport, heating,… HPC UAL: C/C++, F95, Java, … Data mining tools Code repositories: Mercurial, … Post-processing: Scilab,Visit,Root,… Private data: XML files, … Catalogues • Currently, no portal, no gateway, no GRID, … • Use itm.partenaires.cea.fr, one login per user ITM General Meeting Garching 17-21 September 2007

  5. What will be provided? • Short term issue (January 2008) • Computing infrastructure • Gateway: cluster 32-64 CPUs (16GB), 10-100TB, 10Gb/s • Currently, use “itm.partenaires.cea.fr” => then, will used for developments & tests • Logins, private & shared storage area • Tutorial, documentations • No catalogues • No Web Portal, no SSO • No GRID computing (EGEE or DEISA) • Applications • Data structure: v 4.05 • XML tools • Include/import files for F90/95, C/C++ • UAL library for F90/95, C/C++, Java, Matlab (?), Scilab (?) ITM General Meeting Garching 17-21 September 2007

  6. What will be provided ? (2) • Applications: • Workflow engine (KEPLER): • running • Simulation editor/director: • Parameters editor (Nov 07) • Simulation run/control (Jan 08) • SCILAB: post-processing tool • UAL access (?) • Control & monitoring tools: • Cluster tools • On-line data visualization: • 2D/3D actors in KEPLER (VTK based) • Scilab ITM General Meeting Garching 17-21 September 2007

  7. What will be provided? (3) • Codes: • IMP#1: • Helena • Cedres • Mishka • Efit • Soloviev • Database: • MDS+ storage • Machines: ITER, JET, TS • Shots • Use Soloviev to populate it • In-memory ITM General Meeting Garching 17-21 September 2007

  8. How does all that work? • XML schemas • Generate automatically the include files for C, Fortran, Java, Matlab, … • Use it in your code: #include …, import … • Fill the structure through the provided API (UAL): ITM General Meeting Garching 17-21 September 2007

  9. Code adaptation • Data access through the UAL: • Independent of the storage • The framework does not know our data • Principles: • data transfer through a data server (in-memory for the simulations) • Advantages: • Solve the problem of languages mixing • Allow // computations • Small changes: • I/O => UAL • Read  euitmget() • Write  euitmput() • Structure: • Use/import ITM structure • Call euitmget(“CPO name”, CPOin) • Call compute(CPOin,CPOout) • Data access is separated from the computational part • Call euitmput(“CPO name”, CPOout) codes EFIT, … codes HELENA, … codes MISHKA, … UAL In-memory ITM General Meeting Garching 17-21 September 2007

  10. Code integration => actor • =>one-click integration of C/C++, Fortran local codes • GUI tool: define the arguments • JNI based • =>one-click integration of C/C++, Fortran remote codes • GUI tool: define the arguments • Automatic generation of Web Services based on gSOAP • =>one-click catalogue insertion • GUI tool • XML based ITM General Meeting Garching 17-21 September 2007

  11. Building a workflow • Based on KEPLER • =>design: drag & drop actors • =>many actors: database, math, display, web service, grid service, R-expression, … • Advanced models of computation • =>directors: DE, CT, SDF, PN and more in Ptolemy II (FSM, Giotto, DDE) • =>hierarchical design Run control buttons Scheduler: time behavior Catalogues ITM General Meeting Garching 17-21 September 2007

  12. Directors • SDF (synchronous data flow) • Dataflow oriented model • Modules are triggered by the data arrival • DE (discrete event) • Time-oriented models • One global time • Events can be posted at some specific time • PN (process networks) • Message oriented model (network of processes) • Communication by passing messages through FIFO • CT (Continuous Time) • Use to solve differential equations (integrator with feedback). • Implemented solvers: • Forward Euler • Backward Euler • 2(3)-order Runge-Kutta • Trapezoidal rule DE Transformation component: Waveform generator Transformation component: Event generator CT ITM General Meeting Garching 17-21 September 2007

  13. Plot 2D Plot 3D in-memory Data server SOLOVIEV HELENA MISHKA ITM General Meeting Garching 17-21 September 2007

  14. Data visualization • In the workflow (on-line): • =>2D, 3D data plots • VTK based • Scilab • Data & synchronisation through Kepler • In the GUI for the simulations (ISE see later) : • => Y=f(t) • Java graphs • polling • Interactive tools (UAL access is mandatory): • Scilab (using the Kepler data) • VISIT (planned) ITM General Meeting Garching 17-21 September 2007

  15. Data visualization (cont’d) Examples (Scilab) ITM General Meeting Garching 17-21 September 2007

  16. Post-processing tools • =>Scilab • Signal processing, non-linear control, … • Data analysis • Visualization • =>Implementation • Scilab expression • Numerical actors ITM General Meeting Garching 17-21 September 2007

  17. Define a simulation: ISE Simulation Editor: GUI • Objectives: • define friendly all the parameters for a simulation • run & control a simulation ITM General Meeting Garching 17-21 September 2007

  18. Edition mode Simulation mode Monitoring window 2D plot (Kepler actor) Overview window ITM General Meeting Garching 17-21 September 2007

  19. ISE windows Menu toolbar: Overview: Tabs: • General: all the parameters (machines, fields, equilibrium, …) • Favorites: parameters changed often by the user • Overview: global view of the simulation on a timeline • Monitoring: plotted data during the simulation and waveform • Workflow: parameters for the codes relevant to the current workflow • Simulation: special parameters for the simulation ITM General Meeting Garching 17-21 September 2007

  20. Are we ready? • Status • ok • ok • ok with the Kepler data, × with ITM data • ok • version 4.05 • ok for Fortran 90, C/C++ • × for Java, Matlab & Scilab • ok • Last modifications not yet reported • tests in progress • × in development • × not yet scheduled • × not yet scheduled • ok • expected October • Kepler • Local code integration JNI GUI • Remote code integration WS GUI • Post-processing Scilab • Visualization VTK • Data structure • XML schemas • XML transformers: Fortran 90, C/C++, Java, Matlab, Scilab • Data access: UAL • API: • C/C++ • Fortran 90 • Java • Matlab • Scilab • IDL • Storage: • MDS+ • In-memory ITM General Meeting Garching 17-21 September 2007

  21. Are we ready? (2) • Status • ok 1st draft, improvements expected • ok 1st release mid-November • × release January 2008 • ? • ok software & procedures (64 bits SL4) • × must be investigated • × provisional (catalogue in Kepler) • × provisional version must be provided • × provisional (catalogue in Kepler) • ok doc + this meeting • ? This meeting • US-EU collaboration • ISE: Integrated Simulation editor: • Requirements • Parameters setting • Simulation control & monitoring • Gateway • Purchase • Installation • System • applications • Catalogues • Requirements • Codes • Specific parameters • Workflow • Tutorial, doc • What is missing for the 1st release? • Workflow consistency (design) ITM General Meeting Garching 17-21 September 2007

  22. Lab. Lab. user login user login Lab. user login Mid-term plan: 2008 resources applications Gateway Simulation editor Data servers: MDS+, Pub-Sub, HDF5 Workflow editor portal Clusters: JET, IPP, … Workflow engine GRID EGEE Codes: Transport, heating,… HPC UAL: C/C++, F95, Java, … Data mining tools Code repositories: Mercurial, … Post-processing: Scilab,Visit,Root,… Private data: XML files, … Catalogues • 2008 work plan: one objective of this meeting ITM General Meeting Garching 17-21 September 2007

  23. Mid-term plan: 2008 • Catalogues • Database for the specific parameter, codes and workflows • Tools (editor), Java API • Expected: 1st semester 2008 • Portal • SSO, Web portal, applications integration • Based on GridCore & Migrating Desktop • Realized by Euforia • Expected: 4Q08 • Grid-HPC: • Java API for job launching & notification on GRID EGEE: • Based on Glite, integration to Kepler • Expected: 3Q08 • Similar java API for HPC • Euforia • Expected: 3Q08 • Data access for GRID EGEE • How? Objectives: UAL integration • Expected: 4Q08 • Data access for GRID HPC • Euforia • Expected: 4Q08 ITM General Meeting Garching 17-21 September 2007

  24. Mid-term plan: 2008 • Database & data structure • Requirements • Parallel & distributed storage • Data mining tools • Expected: 4Q08 • Visualization • VISIT & 4D/5D visualization • Euforia • Expected: 4Q08 • UAL: • HDF5 • Expected: 1-2Q08 • Handling large memory simulation demand • Expected: ? • Scilab access • Expected: 1Q08 • IDL access • Expected: ? ITM General Meeting Garching 17-21 September 2007

  25. Mid-term plan: 2008 • ISE • Improvements • Parameter distribution, not single value => error bar • Consistency checking • Kepler • Integration of various IMP# codes • Checkpoints: need special consideration? collaboration: CPES? • … • Meshing: • Defining the requirements • Studies must start early • Collaboration: OpenCascade, Salome….? ITM General Meeting Garching 17-21 September 2007

  26. Long-term plan: 2009-2010 • GRID-HPC • Extension of the workflow on EGEE & HPC: • Kepler, MetaGrid … • Euforia project • Expected: 2Q09 • Whole device simulation • Sophisticated meshing tools • Experiments plan (error bar, parameter space …) • Material database ITM General Meeting Garching 17-21 September 2007

More Related