1 / 25

ESIP - Grid based Satellite Data Processing Platform

ESIP - Grid based Satellite Data Processing Platform. Dorian Gorgan Computer Science Department Technical University of Cluj-Napoca SEE-GRID-SCI Project (FP7). Outline. Objectives ESIP platform description gProcess – basic platform ESIP based EO application development

maeve
Download Presentation

ESIP - Grid based Satellite Data Processing Platform

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ESIP - Grid based Satellite Data Processing Platform Dorian Gorgan Computer Science Department Technical University of Cluj-Napoca SEE-GRID-SCI Project (FP7)

  2. Outline • Objectives • ESIP platform description • gProcess – basic platform • ESIP based EO application development • Application development methodology • User interface prototyping • Conclusions ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  3. Objectives • Explore huge spatial data (i.e. satellite images) to supply information on the earth surface, weather, clime, geographic areas, pollution, and natural phenomena • Flexible combinations of multispectral bands of the satellite data • Support many variables based processing - satellite image type (e.g. MODIS, Landsat), geographic area, soil composition, vegetation cover, season, and context (e.g. clouds) • Flexible description, instantiation, scheduling and execution of the Grid processing • Support processing optimization in terms of code, execution time, and data communication costs over the Grid • Tools and components to support the development of EO (Earth Observation) applications • Scalability – data, processing, Grid resources • Compatibility with other EO projects: GiSHEO, GENESI-DR, EGEE, EnviroGRIDS ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  4. ESIP Platform • ESIP - Environment Oriented Satellite Image Processing Platform • The Environment oriented Satellite Data Processing Platform (ESIP) is supported through the SEE-GRID-SCI (SEE-GRID eInfrastructure for regional eScience) FP7 project (2008-2010), funded by the European Commission, http://www.see-grid-sci.eu/ • ESIP provides a Grid based software platform for satellite image processing and development of environmental applications. The services and interactive tools are available by Web applications. • ESIP will be extended to a larger Grid infrastructure by processing real data for SEE regions. ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  5. Partners and Beneficiaries • Partners • ESIP Platform development: 1. Technical University of Cluj-Napoca (UTCN), ESIP coordinator 2. National Center for Information Technology (NCIT) Bucharest 3. West University of Timisoara (UVT) 4. ICI Bucharest, national coordinator and SEE-GRID-SCI partner • Beneficiaries • EO application developers • EO tool and service developers • Government Organizations, Environment Agencies, Hydrological Institutes, and Research Groups involved in environment supervision and behavior prediction of natural phenomena, especially in vegetation related studies ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  6. ESIP Features • The ESIP platform is a collection of Grid services and tools providing the following basic functionality: • Visual manipulation based interactive description of the Grid based satellite image processing by pattern workflow (i.e. DAG) • Development of hypergraphs as a composition of basic operators, services, and subgraphs • Pattern workflow instantiation for particular satellite image • Satellite data management, access and visualization • Workflow based Grid execution • Process execution control and visualization • Optimal execution for appropriate mapping of the processing over the Grid resources ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  7. ESIP and EO Projects Project: EnviroGRIDS Product: BSCOS Platform, Appl’s Project: EGEE Product: Appl’s, Services, Tools Project: SEE-GRID-SCI Product: ESIP Project: GiSHEO Product: eGLE Project: MedioGrid Product: gProcess Project: GENESI-DR Product: EO Services, Data ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  8. eGLE Appl. Other EO Appl’s . . . Application Oriented Level ESIP gProcess Platform Grid Infrastructure Process Description EO Data Image and Spatial Data Functional Levels • Data: satellite images (e.g. MODIS, Landsat, Ikonos and QuickBird); environmental data; diagrammatic process description. • Grid Infrastructure: middleware (Globus Toolkit 4, gLite). • gProcess Platform: service oriented architecture; collection of services providing creation, execution, and scheduling of the jobs; job resource management; access to image database; file transfer; data replication; workflow based execution; image processing. • ESIP: adds satellite image oriented operators, services, and particular algorithms. ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  9. eGLE Appl. Other EO Appl’s . . . Application Oriented Level ESIP gProcess Platform Grid Infrastructure Process Description EO Data Image and Spatial Data Functional Levels • Application Oriented Level: set of tools and components that support the development and the execution of the workflow based distributed processing. • EO Oriented Applications: pilot applications through which the user may access data, and particular processing and visualization. ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  10. Applications User oriented application level Client EditorIC ManagerIC ViewerIC Client Java API EditorWS ManagerWS ExecutorWS ViewerWS Server GRID Infrastructure gProcess Platform Architecture ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  11. Bands: NIR, RED Bands: NIR, SWIR Gond’s Water Detection Algorithm • Gond’s water detection algorithm Gond V., Bartholom E., Ouattara F., Nonguierma A. and Bado I. Surveillance et cartographie des plans d’eau et des zones humides et inondables en rgions arides avec l’instrument VEGETATION embarque sur Spot 4, International Journal of Remote Sensing, 2004, 25,5. pp. 987- 1004. • Landsat satellite image: SWIR (5), Red (3) and NIR (4) spectral bands ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  12. Input S2 OP3 Input SG Output OP1 Input OP2 S1 Input Hypergraphs The hypergraph is a complex graph that combines operators (OP), services (S) and subgraphs (SG). The subgraph could be a simple or complex graph. ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  13. Satellite Image oriented Operators • Basic Operators • Arithmetic operators - addition, division, multiplication, subtraction (applied to two input image files), exponent, complement, and negation (applied to a single input image file) • Radiometric transformations - histogram equalization, mean, histogram scaling, linear scale and threshold based filtering of an image • Spatial domain transformations – blurring, convolution, sharpening, and median filters • Edge, line and spot detection - gradient transformation • Color conversion operators - transform image from a color model (such as RGB, CYMK, HSV) to another one, pseudo coloring multispectral image • Geometric transformations - rotation, scale, translation, and warp • Statistics – compute histogram, mean, and standard deviation pixel values of an image • Non-spatial domain transforms - forward and inverse Fourier transformation, spectrum computation of a Fourier domain image ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  14. Satellite Image oriented Operators • Multispectral Imagery Operators • Vegetation indices: DVI, NDVI, NDVI, IPVI, RVI, OSAVI, GEMI • Water indices: NDWI, VWC, WI • Non-linear interpolation • Change detection ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  15. Band 1 Add Sub Band 2 Not Mply Band 3 Band 1 Process Description Graph (PDG) ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  16. gProcess – Diagrammatic Process Description ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  17. ESIP based App Development Methodology • Algorithm identification and analysis • Data model definition • Identify atomic parts of the computation/algorithms • Parallel and serial processing • Atomic computation Implementation as services, procedures, distinct applications • Algorithm implementation • gProcess based process description • Workflow based description (i.e. PDG) • Workflow instantiation (i.e. iPDG) • Execution over Grid • Develop the application interface • Build up the GUI and link the interactive components to the Grid application entities ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  18. User Interface Prototyping • Simple process description graph: Enhanced Vegetation Index (EVI) It is an optimized index designed to enhance the vegetation signal with improved sensitivity in high biomass regions and improved vegetation monitoring through a de-coupling of the canopy background signal and a reduction in atmosphere influences. Input: NIR, RED and BLUE spectral bands of the Landsat image EVI = 2.5 * (NIR - RED) / (NIR + 6 * RED + 7.5 * BLUE + 1) ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  19. Simple Process Description Graph ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  20. Simple Process Description Graph ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  21. Interface Prototyping • Hierarchical process description: Normalized Difference Drought Index (NDDI) NDDI combines information from both the NDWI and NDVI vegetation indices. Input: NDVI and NDWI computation are described as sub-graphs NDDI = (NDVI-NDWI) / (NDVI+NDWI) ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  22. Hierarchical Process Description Graph ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  23. Hierarchical Process Description Graph ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  24. Conclusions • gProcess provides the workflow execution on both Grid middleware: Globus TK4 and gLite • Process scheduling and monitoring facilities will be included in the platform kernel • ESIP will use spatial data provided by the GENESI-DR project’s services, and will be a basic platform for the GISHEO (ESA-PECS) and EnviroGRIDS (FP7) projects • ESIP will support the development of the GreenView application in the SEE-GRID-SCI project. GreenView is the refinement of surface and vegetation parameters in SEE and CE regions based on satellite images • New operators, Grid services and algorithms will be experimented over the Grid • Compatibility with other satellite data and images will be considered as well ESIP Platform, 4th EGEE User Forum Catania - 5 March 2009

  25. Many thanks. Questions Dorian Gorgan Computer Science Department Technical University of Cluj-Napoca dorian.gorgan@cs.utcluj.ro

More Related