Federated hierarchical filter grids
This presentation is the property of its rightful owner.
Sponsored Links
1 / 7

Federated Hierarchical Filter Grids PowerPoint PPT Presentation


  • 47 Views
  • Uploaded on
  • Presentation posted in: General

Federated Hierarchical Filter Grids. STTR-funded project with Indiana, Caltech and Deep Web Technologies A Grid infrastructure for Data Analysis Integrates with the LHC Tiered Computing Model Directly supports general Scientific Analysis

Download Presentation

Federated Hierarchical Filter Grids

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Federated hierarchical filter grids

Federated Hierarchical Filter Grids

  • STTR-funded project with Indiana, Caltech and Deep Web Technologies

  • A Grid infrastructure for Data Analysis

  • Integrates with the LHC Tiered Computing Model

  • Directly supports general Scientific Analysis

  • In the HEP case, the Gridlet is instantiated as a Rootlet

The FHFG Architecture Composed of Information Service Gridlets managed by general Grid system services with a portlet-based portal user interface


Federated hierarchical filter grids

SS

Database

SS

SS

SS

SS

SS

SS

SS

Raw Data  Data  Information  Knowledge  Wisdom

AnotherGrid

Decisions

AnotherGrid

SS

SS

SS

SS

FS

FS

OS

MD

MD

FS

Portal

OS

OS

FS

OS

SOAP Messages

OS

FS

FS

FS

AnotherService

FS

FS

MD

MD

OS

MD

OS

OS

FS

Other

Service

FS

FS

FS

FS

MD

OS

OS

OS

FS

FS

FS

MD

MD

FS

Filter Service

OS

AnotherGrid

FS

MetaData

FS

FS

FS

MD

Sensor Service

SS

SS

SS

SS

SS

SS

SS

SS

SS

SS

AnotherService


Filter grids

Filter Grids

Three Features:

  • Information services present data through traditional interfaces

  • Filters that accept data between these interfaces, transform and re-present

  • Streaming connections between all services:

    • High performance

    • Archiving

    • Security

    • Fault tolerance

    • Narada Brokering

Filter Grids are built from Information resources wrapped as Web Services and Basic Filters that either transform or aggregate Information. Information Services and Filters support identical Service Interfaces.


Federated hierarchical filter grids

Information Resource

IS

=

Request/Select

Status

MultiResolution Get

BFS

=

InformationService

Issue Queries

MultiResolution Put

Filter Resource

Basic FilterService

Request/Select

Status

MultiResolution Get

Filters either transform or Aggregate Information


Hep event analysis using filter grids

HEP Event Analysis using Filter Grids

Analysis tool of choice is Root.

Typical analysis activity is

  • Loading many files containing event data

  • Passing each event through a selection filter

  • Subjecting each selected event to a set of algorithms

  • Creating summary information in the form of histograms/tables/files

    Analysis: starts with small event samples, then applied to much larger samples

    Frequently these are remotely located in the Grid

    Our HEP implementation is a Filter Grid consisting of Clarens-hosted “Rootlets”.

    Each Rootlet is a full instance of the Root application, but limited in scope:

  • The user’s Root loads a Clarens plug-in

  • The Clarens interface to the Dataset Location Service allows a list of remote datasets to be generated

  • The client contacts each remote Grid node, connects to the Clarens server there, and instantiates a Rootlet

  • The user’s analysis selection code is passed over the network to the Rootlet

  • The list of event data files is passed to the Rootlet

  • The Rootlet executes, and terminates.

  • The output histograms/tables/files are then made available via the Clarens server, and fetched, aggregated and processed as required.


Rootlets

Physicist at Tier3 using Root on GBytes of ntuples

Loads Clarens Root plugin. Connects to Clarens at Tier2. Sends analysis code (.C/.h files).

Clarens creates Rootlet, passes it .C/.h files

Rootlet runs analysis code on TBytes of ntuples, creating high statistics output data.

Root at Tier3 receives and plots data

Root embedded in a Clarens server

Clarens Plugin

GBytes

Rootlets

Tier3

Tier2

Analysis.C, Analysis.h

XML/RPC

Root nTuples

Root nTuples

~10 TBytes


Higgs diphoton analysis using rootlets

Higgs diphoton Analysis using Rootlets


  • Login