1 / 17

A Concept of a Monitoring Infrastructure for Workflow-Based Grid Applications

A Concept of a Monitoring Infrastructure for Workflow-Based Grid Applications. Bartosz Baliś, Marian Bubak, Włodzimierz Funika, Tomasz Szepieniec, Marcin Radecki, Roland Wism ü ller, Tomasz Arodź, Marcin Kurdziel. Plan. Workflow-based applications Application Monitoring

mariah
Download Presentation

A Concept of a Monitoring Infrastructure for Workflow-Based Grid Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. A Concept of a Monitoring Infrastructure for Workflow-Based Grid Applications Bartosz Baliś, Marian Bubak, Włodzimierz Funika, Tomasz Szepieniec, Marcin Radecki, Roland Wismüller, Tomasz Arodź, Marcin Kurdziel

  2. Plan • Workflow-based applications • Application Monitoring • Architecture of the monitoring infrastructure • Performance analysis • Monitoring of multilingual applications • Summary

  3. Workflow-based Grid Applications • Applications composed of a workflow of components • Components are independent Grid services

  4. Application Monitoring • Goal: provide monitoring services to • Obtain debug / performance-related information • Possibly enable manipulations on the target application • Detect events and program actions to be executed when they occur • Consumers • Tools for performance analysis / debugging • Other tools and systems, e.g., for fault-tolerance, load balancing, etc.

  5. Monitoring infrastructure for Wf-based apps • Additional monitoring interface for each grid service (component) to be monitored • Obtain monitoring info for that component • Global monitoring service • Separate grid service • One per application • Obtain collective monitoring info

  6. Monitoring Wf-based Applications Client Global monitoring (grid) service. Collective monitoring information. Additional monitoring interface (monitoring info related to the component) Monitoring service

  7. Startup • User submits a workflow application and requests monitoring, possibly subsequently • Monitoring service is created • It must discover the workflow components • Workflow subsystem must provide a mechanism for this • Workflow registry?

  8. Component monitoring interface – how? • Monitoring functionality inherent part of each component • How to provide the additional monitoring interface? • Take monitoring interface into account at the design stage • Component developers involved? • This approach (hopefully) enables instrumentation of grid services

  9. Performance analysis service • Combined with the monitoring service • Supports performance analysis of • Grid infrastructure • Application • Intra-component • Inter-component

  10. Infrastructure and intra-component analysis • Grid infrastructure • Uses existing tools to monitor the status of grid environment of the components • Intra-component operations • Measure some quantities describing the status and performance of a service used in the application workflow • Depend on the monitoring interface defined by the component developers • Reuse existing approaches to monitoring and analysis

  11. Inter-component performance monitoring • Adds a new level to the performance analysis of grid applications • Accounts for the concept of application as a workflow of services • Monitors the state and performance of the cooperation of the services to capture the state and performance of the whole application • Two modes: • Structural, semantic monitoring mode • Activity monitoring mode

  12. Structural, semantic monitoring • Captures information on workflow components: • Component status • Component usage • Custom component properties • Uses the intra-component monitoring data on: • Functions • Code regions • Synchronisation objects • Combines the information on various components to describe the status of the whole workflow based-application • Proposed techniques: • Source/byte code instrumentation • Dynamic instrumentation

  13. Activity monitoring • Captures information on the behaviour of the workflow • Analysed activity properties: • Caller/callee relationships • End-to-end response time for activity invocation • Success rate of invocation • Synchronisation time in activity • Volume of data exchanged in activity • Proposed techniques: • Distributed instrumentation of workflow components • Insertion of probes

  14. Event Service • Automatically sends notificationsto other grid services when some performance conditions are met. • Allows for a reaction on undesired changes of grid environment status as well as in the activity of the application. • Will be integrated with performance data provider in a uniform framework. • This integration will enable the notified service to inspect a wide range of performance information before deciding on the actions that should be taken.

  15. Two levels of performance information • A proven concept form the G-PM tool. • Used in both application performance data provider and event service. • Lower level performance information: • a consistent, well-defined set of metrics, • used, by system administrators, application developers,users andother parties. • Higher level performance information: • more abstract, numerical, quantitative metrics, • defined on the basis of the lower-level metrics with a special definition language, • allowing for creation of various metrics suitable for a particular purpose.

  16. Monitoring of multi-lingual applications • Services can be implemented using multi-lingual approach e.g. as a computational kernels written in Fortran and wrapped by Java code. • A higher level abstraction of metrics must be worked out which could handle different programming approaches and languages. • The low-level monitoring system must be designed in a way that enables a simultaneous monitoring of Java, C, C++ and other language-based service parts using an uniform approach: • the specification of monitoring services must be the same for all languages, • the implementations of the monitoring services may vary.

  17. Summary • Workflow-based approach to applications still immature • Architecture for a monitoring infrastructure proposed • Integration of monitoring services into components • Global grid monitoring service • Performance analysis • Resuse of existing approaches for infrastructure and intra-component analysis • New level of inter-component analysis

More Related