1 / 8

OSG Accounting System Design Overview

OSG Accounting System Design Overview. Sudhir Borra Philippe Canal Matteo Melani. VO Center. Grid Operation Center. Web Presenter. Statistical Analyzer. Web Presenter. Statistical Analyzer. Meter. Meter. Data Store Access Layer. Collector. Collector. Data Store Access Layer. Meter.

halle
Download Presentation

OSG Accounting System Design Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OSG Accounting SystemDesign Overview Sudhir Borra Philippe Canal Matteo Melani

  2. VO Center Grid Operation Center Web Presenter Statistical Analyzer Web Presenter Statistical Analyzer Meter Meter Data Store Access Layer Collector Collector Data Store Access Layer Meter Meter Meter Meter Repository of Accounting Records Repository of Accounting Records WSAPI Meter Data Store Access Layer Collector Web Presenter Meter Meter Statistical Analyzer Repository of Accounting Records Resource Provider Site

  3. Collectors • Main Functionalities • Keep track of which meter are reporting • Receive information from the Meters • Correlate or complete the information • For example looking up which VO a UID is mapped to. • Apply filters and policies • Store the information in the ‘local accounting data store’

  4. Meters • Ideally they would be the data producers themselves • Fall back solution is the typical log parsing with all it drawbacks • When is it run? Where is the log file? How long is it kept? Does it really contain all the information needed? etc. • Another alternative is active query • condor_q for example • but there are performance issues in some cases • In either cases, the ‘Meter’ should only depend • on the ‘tool’ (condor, pbs, etc.) • and the simple, lightweight Accounting Interface Library • (and should be eventually distributed with the tool)

  5. Meter – Collector Connection • A Meter uses a secure channel to send usage information to the Collector, the usage data header contains: • creation time of the record • time interval covered by the accounting data • Resource/Service ID • UID or DN • Optional Global Request ID • Structure of the accounting data that follow and size • Main information entered (mostly) via a name/value pair • The accounting connection library will format this into an XML files

  6. Meter – Collector Connection • Each Meter instance needs the Collector hostname or IP address and a Token. • A token is a string that the Meter instance and the Collector perform mutual authentication based on a secret-key protocol • The tokens are contained in a file managed by the Collector • A token uniquely identify a Meter instance such that the Collector can distinguish when a Meter instance has never started or when there is no usage information to report. • System administrator can add Meter Instances to the Collector configuration file. Each token must be a unique string for the site.

  7. Avoiding Data-loss • One of our goals is to reduce as much as possible the amount of data loss due to transient problems. • Another goal is to make the Accounting Interface as simple to use as possible • In consequence we will provide a library which will establish and manage the network connections • In case where the connection between two elements is broken the information that would have been sent is record in a local data file. • As soon as the connection is restored the local data file is send to the next element. • Example of elements pairs: • Meter-Collector • Collector – Local Accounting store

  8. Conclusion • More Information • Project Definition, Requirements and Design Documents • OSG Accounting TWiki and mailing list (osg-accounting@openscience.org) • Any Questions, Comments, etc?

More Related