Recent advances in GRID use developed at CNES: The Virtual Private Facilities (WGISS-20, Kiev )

1. Recent advances in GRID use developed at CNES: The Virtual Private Facilities (WGISS-20, Kiev ) • Earth Observation as a tool for Services • Virtual Private facilities & GRIDs • The Emulation way (EPs versus IPs) • Practical consequences Jean Pierre Antikidis CNES Programme & Strategy Directorate-ARP "Space Information Systems” jean-pierre.antikidis@cnes.fr JP Antikidis, WGISS20-Kiev, 12 sept 2005

2. To day EO large scale objectives World-wide follow-on ozone World-wide follow-on of sea state (MERCATOR) World-wide follow-on of vegetation index World-wide follow-on of crop and food resources GMES programme GEO endeavour .... Can we afford on the long run to pile-up investments as we do today ? JP Antikidis, WGISS20-Kiev, 12 sept 2005

3. What is a Software based service ? It is a set of Hardware , Software and user interfaces connected in such a way a given end-user service is provided. To make it available we need practically: # To decide on the needed HW arrangement # To decide on a suitable SW running on this HW # To decide on machines interconnections and interfaces with final users This can be made on a real fashion (Private Facilities) or rely on an exact virtual equivalent (Virtual Private facilities). => One possible way to implement a VPF is to “Emulate” the needed HW/SW/Files and Connections topology JP Antikidis, WGISS20-Kiev, 12 sept 2005

4. Some upcoming concepts Service is the Goal and is achieved by creating Hw/Sw facilities: The Network story PN: Private Network VPN: Virtual PrivateNetwork=VP+distr.Networks X. Jiang, D. Xu, “VIOLIN: Virtual Internetworking on OverLay INfrastructure”, Perdue Un. Combined Services (Network+computers+data) PF: Private Facilities VPF: Virtual Private facilities=PF+Grids Emulation is one way to create a VPF =>EVPF JP Antikidis, WGISS20-Kiev, 12 sept 2005

5. Exemple of to-day computer systems definition for a given Service JP Antikidis, WGISS20-Kiev, 12 sept 2005

6. Real File 1 A Software service created as a VPF Virtual File Machine 5 Sw3 Machine 1 Sw1 Machine 2 Sw2 Sw5 Machine 6 Machine 3 Sw3 Sw6 Machine 4 Sw4 VPF=MACHINES+PROCESSES+FILES+CONNECTIONS Machine 7 Sw7 Real File 2 Real access to Web service Real File 3 JP Antikidis, WGISS20-Kiev, 12 sept 2005

7. Experiment Setup(ref:violin project) Two mutually isolated virtual clusters VM VS VS Physical Cluster (ITaP) Physical Switch JP Antikidis, WGISS20-Kiev, 12 sept 2005

8. JP Antikidis, WGISS20-Kiev, 12 sept 2005

9. JP Antikidis, WGISS20-Kiev, 12 sept 2005

10. Instead of creating complex processing system, perhaps better describe them in a powerfull and fully flexible computing topology and make the resulting emulation running as a replacement for the real system. The HEAVEN paradigm "The result produced by running an "emulator" is equivalent to the result produced by the real system." HEAVEN can "mimic" any required system JP Antikidis, WGISS20-Kiev, 12 sept 2005

11. Simple emulation SW1 Game machine Emulation on a PC SW1 SW2 SW2 PC emulation in a PC (looks stupid but …) JP Antikidis, WGISS20-Kiev, 12 sept 2005

12. Somewhat more interesting… SW2 SW2 SW1 SW1 SW3 SW3 Emulation of a complex PC set of machines with data storage & multi-Hw/Sw layout ("Still stupid but not that much in a GRID based system") Key issue: Should power compute resources becomes cheap and flexible enough, perhaps better exchange flexibility and full Sw compatibility with raw CPUs power =>> GRID is bringing the needed characteristics for such an evolution JP Antikidis, WGISS20-Kiev, 12 sept 2005

13. Real File 1 Virtual File A Software service created as a EVPF Machine 5 Sw3 Machine 1 Sw1 Machine 2 Sw2 Sw5 Machine 6 Machine 3 Sw3 Sw6 Machine 4 Sw4 EVPF=Emulation of (MACHINES+PROCESSES+FILES+CONNECTIONS) Exchange Virtual File Machine 7 Sw7 Real File 2 Real access to Web service Real File 3 JP Antikidis, WGISS20-Kiev, 12 sept 2005

14. Building elements for a given machine Machine “n” Software “n” Emulated machine Standard SW Exchange Files File services Real Virtual JP Antikidis, WGISS20-Kiev, 12 sept 2005

15. Files & data Multitasking environnement Machine “n” Software “n” Machine “n” Software “n” Machine “n” Software “n” Reproduces the full topology needed for a given VPF JP Antikidis, WGISS20-Kiev, 12 sept 2005

16. JP Antikidis, WGISS20-Kiev, 12 sept 2005

17. JP Antikidis, WGISS20-Kiev, 12 sept 2005

18. JP Antikidis, WGISS20-Kiev, 12 sept 2005

19. Multitasking environement implementation 1) Create as many machine Emulators as needed machines by either: - Creating a machine HW emulator and plug a genuine O/S on it - Emulate a full compound of machine+O/S emulator 2) Plug without change the applications softwares associated to any of the needed machines 3) Describe inter-machine data exchanges with respect to their content (can correspond to real or virtual files) The result behave in an asynchronous fashion as the full equivalent to the real implementation JP Antikidis, WGISS20-Kiev, 12 sept 2005

20. The Emulation Packet concept (VMWare based example) “With VMWare, you have the option to run your choice of 'primary' OS at all times. Then, when you need access to the other OS all you need to do is start up a vmware session. VMware includes file system shares to facilitate getting files betweeen one environment and the other, and it automatically supports a seamless NAT translation that allows your virtual machine to have full access to the Internet” Emulation Packet (1 EP per Machine to reproduce) EP JP Antikidis, WGISS20-Kiev, 12 sept 2005

21. Autonomous processing packet containing:The HW emulator (To be developed once for a given machine brand) The genuine O/S (in theory the one already available) The native user application SW (no change) The Files description associated to the given machine (as created by the GUI) FILES DEFINITIONS (METADATA) APPLICATION SW O/S IMPLEMENTATION HW EMULATOR “Emulation Packet” EP STANDALONE “Emulation Packet” (1 EP per Machine) JP Antikidis, WGISS20-Kiev, 12 sept 2005

22. EPs versus IPs (Grid versus Internet) The efficiency of Internet principle rely on the IP concept by which a data packet once submitted to the system becomes autonomous (no time nor routes relations in between the packets) This allow Internet to handle on an unvisible fashion Billions of packets at a time without need for strong centralised means. Same stand with “EP” concept (these are much bigger packets of course !) in which Emulation packets once generated becomes fully autonomous with respect to their location or timing. This allows the full compatibility of millions of EPs at a time corresponding to perhaps several thousand of “Private facilities” that look from the user viewpoint as a pseudo HW/SW service implementation under his full control. JP Antikidis, WGISS20-Kiev, 12 sept 2005

23. Emulation Packet prep. XML descr. UpperWare EP EP EP EP EP EP EP Graphical Interface for Service developer Power Computing Resources EP Virtual Private Facilities created by Emulation UpperWare Information provider Real interfaces End user JP Antikidis, WGISS20-Kiev, 12 sept 2005

24. GRID environement implementation 1) Create as many Emulation Packets (“EP”s) machine Emulators as needed machines by either: - Creating a machine HW emulator and plug a genuine O/S on it - Emulate a full compound of machine+O/S emulator 2) Plug without change the applications softwares associated to any of the needed machines 3) Describe inter-machine data exchanges with respect to their file content (can correspond to real or virtual files) The result behave in an asynchronous fashion as the full equivalent to the real implementation JP Antikidis, WGISS20-Kiev, 12 sept 2005

25. GUI UPPERWARE HEAVEN Systems and services deployments onto a GRID based on autonomous “Emulations Packets” EP EP EP EP EP EP Interfaceto GRID EP EP EP EP EP EP EP EP EP EP Urls ? GRID Files Web Service Ftp? JP Antikidis, WGISS20-Kiev, 12 sept 2005

26. CONCLUSION, Quote: "Overview of New technologies for using Grids by :Rhys Newman Manager of Interdisciplinary Grid Development, Oxford University • A grid can generate significant value from idle resources but: • Users must be given absolute control of their environment. • Owners must retain absolute control of their machines. • Virtualisation/Emulation enables this to happen: • Administration can be simple enough to become a fixed cost. • Security concerns can be minimised almost to the “anonymous” level. • We still need new technology to make this happen, but we may be very close…… Quote CNES=>>> VFC, EVFC,WAG are such a Tool JP Antikidis, WGISS20-Kiev, 12 sept 2005

27. Virtualisation is the Key • To provide these properties, reliably and uniformly on heterogeneous resources, the user’s environment must be “Virtual”. • Simplifies the administration: • The “service end” of eScience can focus on the provision of “virtual hardware” • The “user end” of eScience can focus on developing applications in a standardised environment. • Imagine M users and N owners, each wanting their concerns addressed: • Without Virtualisation: MxN negotiations. • With Virtualisation: M+N negotiations. JP Antikidis, WGISS20-Kiev, 12 sept 2005