Development of new tools and interoperability at CDPP.

1. Development of new tools and interoperability at CDPP. Christian Jacquey The CDPP (Centre de Données de Physique des Plasmas) is the french national centre of natural plasma data. Graz, IWF, Nov. 23, 2007

2. Origin and missions of the CDPP • First phase of CDPP (1998-2005) • Present phase (2005-…) • Context • Scientific targets • AMDA (Automated Mutli-Dataset Analysis) • Scientific animation • Potential collaborations

3. Origine and missions of the CDPP The CDPP was created in 1998 by both CNES and CNRS. The CDPP is hosted at CESR, Toulouse 1st mission: long term archive of plasma data obtained with experiment realised with french participation, making them available to the community 2nd mission: providing added value on the archived data  services, tools 3rd mission: participation to Virtual Observatory projects 4th mission: scientifical animation (workshop, …)

4. First phase: 1998 - 2005 • Archiving more than 250 datasets, sometimes saving almost lost ones (GEOS 1,2, ISEE 1, 2, 3, ARCAD, WIND, ULYSSES, INTERBALL, CLUSTER,…) But: • Data available only in the native format • « Isolated » data sets • Very difficult to develop tools and services • Small number of users • Implication in the interoperability development, founder member of the SPASE consortium

5. Second phase: 2005 - … • Technological context: • The volume, the diversity and the complexity of the data are increasing more and more • The capacities of the machines and the networks are increasing • The interoperability is developing • Scientific context: • THEMIS launched in February 2007, simultaneously flying with CLUSTER, DSTAR, GEOTAIL, ACE, WIND, and others • STEREO launched in October 2006, simultaneously flying with ULYSSES, ACE, WIND • Planetary missions are flying: VEX, MEX, MESSENGER, CASSINI So many data  fantastic scientific opportunities  but need of infrastructures for helping researchers to take advantage of them

6. Scientific targets • Heliophysics, Sun-Earth relationships • Magnetospheric physics • Planetology

7. Heliophysics, Sun-Earth relationships • Origin, propagation, characterisation, classification of the solar/interplanetary disturbances (CME, CIR, interplanetary shocks, clouds, filaments, …) • Identification of the mechanism and sites of the acceleration of the solar energetic particles and their propagation • Coupling to the Earth, geoeffectivness, level of disturbances in the ionosphere and the internal magnetosphere • Modelling the 4D-heliosphere • Predicting the heliospheric conditions: space weather • Predicting their impact on the Earth environment

8. Large scale constellation August 15, 2007 Messenger MEX ejecta VEX STEREO A and B Compressed solar wind shock Energetic particles Ulysses Around the Earth: ACE, WIND THEMIS, CLUSTER, GEOTAIL, …

9. The multi-scale heliospheric constellation Continuous solar observations: SOHO, STEREO, HINODE, RHESSI, Ground observatories Constellation of probes distributed at large scale : Heliospheric probes: STEREO-A/B, ULYSSES, VOYAGER Planetary probes: MESSENGER, VEX, MEX, MGS, CASSINI Constellation of probes distributed at medium scale around the Earth orbit: ACE, WIND, THEMIS, GOES, GEOTAIL, CLUSTER, LANL Two sub-constellations in small scale cluster configuration THEMIS, CLUSTER, + detailed earth-ionosphere data + astronomical observations (aurora)

10. General functions X-ray GOES Links to simulation FALSE DATA SOL/tools PLAS tools Time tools SOL/Workspace/Composer PLAS/Workspace/Composer

11. Magnetospheric physics • Large-scale process at the origin of substorm • Instabilities • Multi-scale dynamics of the magnetopause versus solar wind or magnetosheath conditions • Particle acceleration (shock, magnetotail, cusp, …) • Turbulence • … (so many interesting topics)

12. The magnetospheric constellation

13. Planetology • Comparative study • Dynamics of the planetary ionised environment versus solar wind conditions • Plasma interaction with moons, rings, dust, neutrals … • Magnetosphere without ionosphere (Mercury) • Multi-species plasmas • Plasmas in small structures (comets, L < RL)

14. Example of comparative study H+ O+ MARS VENUS Ferrier et al.

15. Basic requirements • Easy access to many datasets from various origins • Standardisation of the data for making them usable together • Data in physical values • Tools for exploring the content of data • … Taking into account the present context, the development data centres are one of the keys for getting a fruitful return on the experimental investment

16. Toward a new CDPP • Rebuilding the system • Completing the data collections • Making « hot » data available • Standardisation of the data to make them usable by generic tools • Developing tools (AMDA) • Developing concrete application of interoperability, in collaboration with other data centres or data/service providers

17. “Hot” data at CDPP • CLUSTER Prime Parameters • High-res data for CLUSTER/WHISPER, STAFF, CIS • The CDPP hosts a full mirror THEMIS database (space and ground data) • DEMETER • STEREO/SWAVES, IMPACT • VEX, MEX ASPERA But also older data, useful for very large statistical analysis And remote access to the whole CDAWeb database, and soon, the CASSINI/MAPSKP and the VEX/MAG data

18. AMDA Automated Mutli-Dataset Analysis

19. Example: event search Old fashion: “paper” search ISEE MAG IMP8 PLA ISEE- electron AE, AL IMP8 MAG IMP-8 AE, AL ISEE-data Current fashion: web CDAWeb SPDF UCLA CAA CDPP And others Event search takes time and energy

20. AMDA: the goals Automated Mutiple Dataset Analysis Multiple Dataset: the study of the (multi-scale) dynamics of plasma objects requires to perform the integrated analysis of multi-point and multi-instrument data • Automated (or semi-automated) Analysis: help for • Search, characterisation, classification of events • Extensive exploitation of the vast database (statistics) • “historical” studies (example: through solar cycles) • Building catalogues • Building virtual constellations • Time-Table (EventList) production

21. AMDA, fact sheet • Built in IDL, C, FORTRAN, Javascript • Works on standardised and “simple” data, NetCDF •Includes models (Tsyganenko, Shock, MP, NS, …) •Uses external data (now from CDAWeb and soon from MAPSKP, IWF, …) • Forum for user feedback •Twiki for information (development, bugs, …) exchange • Already in use for scientific studies AMDA DOES NOT WORK WITH InternetExplorer and Safari AMDA is public

22. Functionalities Production of Time-Tables • Automated access to data • Access to and on-the-flight ingestion of external data (test version only) • Data merging • Data download • Data visualisation • User edited parameter computation on the data • Automated conditionnal search • Visual search

23. AMDA, DEMO

25. Conditionnal search B B • STEP 1, on AMDA: search • Test: • X1< -10 Re • • BX1 * BX2 * BX3 * BX4 > 0 • • min([BX1 BX2 BX3 BX4]) < 0 • • max([BX1 BX2 BX3 BX4]) > 0 • Time-Table

26. AMDA/”External Data” MAPSKP AMDA CDPP System Local Database CDAWeb others

27. Scientific animation • Organisation of workshops • Hosting exchange pages (twiki) • Hosting researchers for extended use of AMDA

28. Collaborations Targeting data or service exchanges, interoperability, Virtual Observatory construction  Developing more and more data access/processing/analysis facilities for serving the community Virtual Observatory? Example: a VO for heliophysics CDPP participates to the FP7 proposal HELIO

29. Necessity to access to many data and tools Astronomical data Simulation data/runs Solar data Heliospheric data CDPP MEDOC BASS2000 NSSDC Planetary data PDS NSSDC PSA CDPP USER NSSDC CDPP CAA Iono/Magnetospheric data

30. A virtual observatory Astronomical data Simulation data/runs Solar data Heliospheric data CDPP MEDOC BASS2000 NSSDC Planetary data PDS NSSDC PSA VO CDPP USER NSSDC CDPP CAA Iono/Magnetospheric data

31. “Baby” magnetospheric VObased on Time-Table exchange Collaboration discussed with CESR, CAA, GAIA and CDAWeb. Already working between AMDA and CL • Definition of a standard format for time-table (likely VO-Table) • Implementation of interface for Time-Table exchange

32. Time-Table: a simplified use case B B Science goal: study based on CLUSTER of the cross-tail current in the neutral sheet in relation with interplanetary and geomagnetic activity conditions • STEP 1, on AMDA: search • Test: • X1< -10 Re • • BX1 * BX2 * BX3 * BX4 > 0 • • min([BX1 BX2 BX3 BX4]) < 0 • • max([BX1 BX2 BX3 BX4]) > 0 • Time-Table #1 STEP 2: Edition of Time-Table#1 Adding 2 hours before and after the time intervals of Time-Table #1 Time-Table #2 STEP 3, on CAA: Sending the Time-Table#2 and extracting a sub-database STEP 4, on GAIA: Sending the Time-Table#2 and extracting a sub-database STEP 5, on CDAWeb: Sending the Time-Table#2 and extracting a sub-database … STEP 7, on CL: Sending Time-Table#1 and looking at the distribution function Etc…

33. Collaboration CDPP/IWF on planetology • Final goal: construction of a VO for planetology • VERY CHALLENGING: strong heterogeneity of disciplines, cultures, types of data • Short term: • Demonstrator based on AMDA using local ASPERA data and distant data (VEX-MAG at IWF, MAPSKP at CESR)

34. Conclusion The CDPP is in strong evolution The CDPP develops tools for serving the community. These tools are free and public. The CDPP is opened for hosting workshops or visiting researchers The CDPP is volunteer to participate or stimulate collaboration for developing new services, facilities or infrastructures helping the progresses of the community.