The gridcc instrument element from prototypes to production environments
This presentation is the property of its rightful owner.
Sponsored Links
1 / 36

The GRIDCC Instrument Element: from prototypes to production environments PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on
  • Presentation posted in: General

EU FP6 Project. The GRIDCC Instrument Element: from prototypes to production environments. Roberto Pugliese On Behalf of the GRIDCC Collaboration. The GRIDCC project: Goals & Objectives.

Download Presentation

The GRIDCC Instrument Element: from prototypes to production environments

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The gridcc instrument element from prototypes to production environments

EU FP6 Project

The GRIDCC Instrument Element: from prototypes to production environments

Roberto Pugliese

On Behalf of the GRIDCC Collaboration


The gridcc project goals objectives

The GRIDCC project: Goals & Objectives

  • The GRIDCC - Grid Enabled Remote Instrumentation with Distributed Control and Computation – project has the main aim to bring Instruments to the GRID

  • It is a 3-years EU FP6 project started in September 2004

  • Web site: www.gridcc.org

  • More in particular the project goals are:

    • Definition of a “Instrument Element” allowing a standard remote access to any type of instrumentation

    • Tight integration between instrument grid and classical computational grid

    • Human interaction with Grids via Virtual Control Room (collaborative environment)

    • Enactment of complex workflows


The gridcc partners

The GRIDCC partners


Gridcc architecture i

Instruments Grid

Computational Grid

CE

IE

IE

DATA

Instrument Element

CE

Instrument

Manager

Information &

Monitoring System

Grid

SE

Problem Solver

VCR

Workflow

VCR

ES

VCR

VCR

Collaborative Environment

Execution Services

GRIDCC: Architecture (I)

  • The GRIDCC services are Web Services compliant, according to the WS-I convention.

  • Any type of client WS-I compliant, can access the GRIDCC services without any specific software library.

  • X509 based security is used

  • When performances are required , a kerberos based mechanism is used to access IEs.

Service Oriented Architecture – SOA


Gridcc architecture

Instrument

elements

(IE)

Storage

Element

(SE)

Instrument

elements

(IE)

Storage

Element

(SE)

Instrument

Element

(IE)

Storage

Element

(SE)

Virtual

Control

Room

Storage

Element

(SE)

Storage

Element

(SE)

Storage

Element

(SE)

Grid

Virtual

Control

Room

Compute

element

(CE)

Compute

element

(CE)

Computing

Element

(CE)

Execution

Service

WfMS

WMS

AS

GRIDCC Architecture

Information

and Monitoring

Services

(IMS)

Global Problem

Solver

Security

Services

Collaborative

Services

(CS)

Web Service

Interface

WMS Work Management System

WfMS Work Flow Mng System

AS Agreement Service

Information

System

(BDII)

Broker


Gridcc permanenttest bed

GridCC PermanentTest Bed

VCR, IE

WfMS

VOMS

IE, CE/CREAM,SE, LFC

AS, KrbServer

WMProxy

VCR, SE, IE, BDII


Gridcc main target areas

GRIDCC main target areas

Remote process control

Accelerator control

(Tele-) Biomedicine

Robotics

Automotive

Electronic microscopes

(Large-scale) scientific experiments

High energy particle physics

(Radio-) Telescopes

GRIDCC

Middleware

Widely Sparse Instrumentation

Power Grids

Monitoring of the territory

Monitoring of the sea

Geo-hazard prediction

Distributed laboratories

Transportation monitoring

Sensor network


Gridcc pilot applications

GRIDCC pilot applications

PowerGrid

Particle Accelerator

High Energy Physics

CMS

Meteorology

Device Farm

Geohazard Monitoring


Accessing grid enabled instrumentation requirements

Accessing GRID Enabled Instrumentation: Requirements

  • Interactive access to allow the control and the monitor of the instrumentation. Uniform access to the physical devices.

  • Fast Data Publishing to allow the publishing acquired by the instrumentation

  • Information (logs, errors, etc.) Publishing to track the behaviour of the instrumentation and possibly to correct the malfunctioning

  • GRID integration. The acquired data should be visible by the GRID protocols both to :

    • Move them to large GRID data storage (Storage Element)

    • Process them from the GRID computing power (Computing Element)


A new 4 channels element to fit the requirements the instrument element ie

A new 4 channels element to fit the requirements: The Instrument Element (IE)

Custom (JMS, ..... )

Discovery

Data/Info Pubblishing

I’m

here

Instrument

Element

Instrumentation Contorl

Web Service

GRID Access

SRM/GRIDFTP

Instruments Access

Advance Reservation

Custom

QoS


Instrument element versatile i o multichannel

Instrument Element: Versatile I/O multichannel

  • The IE permits

    • SOA accessible operations to control and monitor the instruments (via VIGS), such as:

      • execute a command

      • get / set parameters

    • different data outputs:

      • data mover to/from a grid Storage Element (via SRM)

      • high bandwidth channel for data publishing (via IMS).

      • low bandwidth channel for logs, states etc. (via IMS).

Logs, Errors,

States, Monitors

Data Subscribers

VCR

IMS

Grid Interaction

IE

Commands

Storage

Element

(SE)

Storage

Element

(SE)

VIGS

Storage

Element

(SE)

SRM

ES

Status

Parameters

Instruments

VIGSVirtual Instrument Grid Service

IMSInformation and Monitor Service

SRMStorage Resource Management


Instrument element architecture

Data Flow

State Flow

Error Flow

Monitor Flow

Control Flow

Access Control Manager

Instrument Element Architecture

IMS

The term Instrument Element describes a set of services that provide the needed interface and implementation that enables the remote control and monitoring of physical instruments.

IMS

Resource

Service

SRM/SE

GridFTP

SE

Instrument

Element

Problem

Solver

VIGS

Data

Publisher

IMS

Instrument

Manager

IMS

Proxy

Control

Manager

Data

Collector

Control Manager

Event

Processor

FSM

Engine

Instrument Protocols

Input

Manager

Resource

Proxy

Real Instruments


Configuration examples

Access Control Manager

Configuration Examples

IMS

Resource

Service

Configuration A, B, C

can run concurrently

sharing services like

RS, PS, IMS

SE

SRM/SE

GridFTP

Instrument

Element

Problem

Solver

IMS

Data

Publisher

Instrument

Manager

Top

IM

Top

IM

B

C

Top

IM

A

IM

IM

IM

IM

IM

IM

Resource Service Instruments Pool


Instrument element framework characteristics

IE

Instrument

Instrument

Instrument

Instrument

Instrument Element Framework Characteristics

Resource

Service

IMS

  • The IE framework is modular, fully customizable and can be adapted to different environments and different type of instruments to be controlled.

  • Customizable plug-in modules are used to interface the framework with the communication protocol (both hardware and software) used by the instruments.

  • the dynamic behaviour of the control that has to be performed is programmable via the built in Finite State Machine (FSM).

  • relatively fast control feedback loops can be provided using the customizable built in Event Processor (EP).

  • Error repairing actions due to malfunctioning of the instruments can even be handled by EP or by the Local Problem Solver (LPS).

Data

Mover

Local

Problem

Solver

VIGS

Access Control Manager

Instrument

Manager

Instrument

Manager

Data

Collector

IMS

Proxy

Control

Manager

Control Manager

Event

Processor

FSM

Engine

Input

Manager

Resource

Proxy


Instrument manager

Access Control Manager

Instrument Manager

IMS

IMS

Resource

Service

SRM/SE

GridFTP

SE

Instrument

Element

Problem

Solver

Data

Publisher

VIGS

IMS

Instrument

Manager

Instrument

Manager

IMS

Proxy

Control

Manager

Data

Collector


Instrument manager1

Instrument Manager

Control Manager

IMS

Proxy

Event

Processor

FSM

Engine

Data

Collector

Input

Manager

Resource

Proxy

Instrument Manager

Plug-in modules to interface to the instruments

Customizable

Control Flow

Data Flow

Monitor Flow

State Flow

Error Flow

Instruments

  • IM is composed by 3 main components:

    • - Control Manager:

  • - Input Manager. It handles all the input events of the IM. These includes commands from GUIs or other IMs,errors/state/log/monitor messages.

  • - Event Processor. It handles all the incoming message and decide where to send them. It has processing capability

    • - FSM. A finite state machine is implemented

    • - Resource Proxy. It handles all the outgoing connections with the resources.

  • Data Collector. It get data from the controlled instruments and make them available to the data mover. A local storage of the data is even foreseen.

  • - IMS Proxy. It receives error/state/log/monitor information from the controlled resources and forward them to IMS


Data collector

Data Subscribers

(Monitor, data analysis, etc.)

Access Control Manager

Control Manager

Event

Processor

FSM

Engine

Instrument

Instrument

Instrument

Input

Manager

Resource

Proxy

Instrument

DB persistency

Data Collector

RMM-JMS

Resource

Service

IMS

Data

Mover

enable/disable

data stream

Local

Problem

Solver

Instrument

Element

VIGS

IE File

System

Instrument

Manager

IMS

Proxy

Control

Manager

Data

Collector

  • Status

  • data collection

  • data publishing on RMM-JMS

  • data on file in progress

  • data on DB to be started

  • sync commands (enable/disable

  • done trough the “set param” of the

  • IE façade (VIGS)

data stream


Resource service

Access Control Manager

Resource Service

IMS

IMS

Resource

Service

SRM/SE

GridFTP

SE

Resource

Service

Instrument

Element

Problem

Solver

Data

Publisher

VIGS

IMS

Instrument

Manager

IMS

Proxy

Control

Manager

Data

Collector


Resource service architecture

Resource Service Architecture

Partition/Configuration

retrieve methods

Discovery

Manager

Available

Resources

Partition and Lock

setting methods

Subscribe

Manager

Partition

Definitions

RS Data Bases

  • The Resource Service (RS) handles all the resources of an IE and manages their partition (if any).

  • A resource can be any hardware or software component involved in the IE (instruments, Instrument Managers, IMS components)

  • RS stores the configuration data of the resources and download them to resource target when necessary

  • Resources can be discovered, allocated and queried.

  • It is the responsibility of the RS to check resource availability and contention with other active partitions when a resource is allocated for use.

  • A periodic scan of the registered resources keeps the configuration database up to date.

Configuration

setting methods

Partition&Lock

Manager

Configuration

Definitions

Discovery

methods

Configuration

Manager


Information and monitor service

Access Control Manager

Information and Monitor Service

IMS

IMS

Resource

Service

SRM/SE

GridFTP

IMS

SE

Instrument

Element

Problem

Solver

Data

Publisher

VIGS

IMS

Instrument

Manager

IMS

Proxy

Control

Manager

Data

Collector


Information and monitor system ims

Information and Monitor System (IMS)

IMS

Proxy

  • The Information and Monitor Service (IMS) collects messages and monitor data coming from the IMS Proxy of the IMs.

  • The messages are catalogued according to their type, severity level and timestamp. Data can be provided in numeric formats, histograms, tables and other forms.

  • The IMS collects and organizes the incoming information either in a database or a pub/sub system or both. It publishes the data to subscribers. These subscribers can register for specific messages categorized by a number of selection criteria, such as timestamp, information source and severity level.

Publish / Subscrib

IMS

Proxy

PERSISTENCY

SUBSCRIBERS

SUBSCRIBERS

IMS

Proxy


Ims fast data publishing rmm jms

IE

IMS Fast Data Publishing: RMM-JMS

Same data are sent to several subscribers.

Multicast protocols can have a benefic impact on performances

IE Data Producer

Data Subscribers

  • JMS provides a standard set of

  • APIs for the communication

  • Many commercial and academic

  • JMS implementations

  • both in C/C++ and Java

  • (NaradaBrokering, Sun, IBM)

  • GRIDCC (IBM Haifa lab) has

  • implemented a Reliable Multicast

  • protocol (RMM) JMS compliant

  • RMM-JMS works within a LAN

  • but an efficient bridge technology

  • has been developed to allow

  • inter-LAN multicast communication


Embedded srm compliant grid storage element se

Access Control Manager

Embedded SRM compliant Grid Storage Element (SE)

Resource

Service

IMS

SRM

SE

Backend

GRID

accessibile data

Local

Problem

Solver

Instrument

Element

VIGS

IE File

System

Instrument

Manager

  • Status: installed and running

  • SRM 2.2

  • STORM implemetation (INFN)

  • Bestman implementation (Berkeley))

  • XFS IE file system


Discovery instruments i

IE

IE

IE

Discovery Instruments (I)

  • The discovery of the instruments or of the IEs is an issue when the number of elements is high

  • We can have two cases:

    • Quasi static cases

      • The number of IEs are well defined and the single IE is quite complex with a good hardware support

      • In this case a register based discovery mechanism can be used.

      • GRIDCC tesbed is using the LCG BDII (Berkely Database Information Index) based on LDAP

      • The information collected in the BDII follow a GLUE schema and can be used for match making querries

Querring

Client

Querring

Client

Querring

Client

BDII

Local BDII


Discovering instruments ii

IE

IE

IE

IE

M1

M2

Querring

Client

Querring

Client

Querring

Client

Discovering Instruments (II)

  • Dynamic cases

    • The number of IEs can change very quickly, they are very simple devices, often with poor hardware support

    • The discovery is just use to know which are the online IEs

    • A new approach has been developed based on Peer to Peer (P2P) protocols

    • See demo on this workshop


Instrument elements at work

Instrument Elements at Work


Pilot applications cms i

DAQ

IM

DAQ

RS

DAQ

IMS

FB

RB

FF

xdaq

xdaq

xdaq

Pilot applications: CMS - I

  • The GridCC middleware has been deployed to control the run of the CMS (Compact Muon Solenoid), one of the four high energy experiments in LHC (Large Hadron Collider) at CERN laboratory.

  • CMS Magnet Test and Cosmic Challenge (MTCC) I and II, a milestone in the CMS construction, positively carried out.

CMSDetector

CMS Control

Structure

User

Interface

Top IE

CSC

IE

Tracker

IE

HCAL

IE

DAQ

IE

RPC

IE

Trigger

IE

DT

IE

ECAL

IE


Cms mtcc phase i and phase ii carried out in 2006

CMS MTCC phase I and phase II carried out in 2006

Scale MTCC versus CMS

FEDs: 20 out of 600 3%

EVB RUs: 6 out of 6001%

Filter Farm: 14 out of 20000.3%

Trigger rate: 100 Hz out of 100 kHz 0.1%

Event size: 200 kB out of 1 MB20%

IEs15 out of 150 10 %


Pilot applications power grid

Virtual

Control

Room

Virtual

Control

Room

Pilot applications: Power grid

  • GRIDCC deployed to monitor:

  • A 50kW generator

  • A 1 kw Photo-Voltaic array

See DEMO on this workshop

Power Grid. GRIDCC deployed to monitor a 50kW generator and a 1 kW Photo-Voltaic array

Instrument Manager

Power Grid V.O.

Instrument Element

Solar Panel

...

Gas


Pilot applications remote operation of an accelerator

Elettra Synchrotron

Pilot applications: Remote Operation of an Accelerator

See DEMO on this workshop


Gridcc other applications

GRIDCC: other applications

  • Meteorology (Ensemble Limited Area Forecasting)

    • Weather forecasting system to detect hazardous weather

  • Device Farm for the Support of Cooperative Distributed Measurements in Telecommunications and Networking Laboratories

    • The Device Farm consists of a pool of Measurement Instruments for Telecommunication Experiments

  • Geo-hazards: Remote Operation of Geophysical Monitoring Network

    • The monitoring net will be characterized by different levels of activity: stand-by, pre-alert, alert, plus a control modality

    • An event worth to be monitored is for example the evolution of a “landslip”


Instrument element scalable on embedded systems

Custom Board

Xilink Virtex IV

Grid

Custom Electronics

FPGA

PPC

405

JavaVM

Web Service

GridCC IM

Linux

Instrument Element: Scalable on embedded systems

1 Gbps Ethernet

IE

Instrument Manager

Custom Logic

Standalone Axis

Montavista www.montavista.com

USER INTERFACE

JamVM http://jamvm.sourceforge.net


Ie technologies

IE technologies

  • Web Service compliant (WS-I)

  • Tomcat + Axis (and Java) and Axis standalone are the main technologies of the IE

  • All the services are deployed on a single or multiple instances of Tomcat, according to the needs of the application

  • Message oriented middleware (Pub/Sub) is based on the Java Messaging System (JMS). The following implementations are used in the project

    • Sun

    • Narada Brokering

    • RMM - JMS (GridCC IBM)

  • MySQL and Oracle are used as Data Base for the RS

  • SRM interface version 2.2 used


Conclusion i

Conclusion - I

  • The GridCC project is integrating instruments into the “classic” computational/storage Grids.

  • One of the novel concepts introduced by GridCC are:

    • The Instrument Element, allowing a virtualisation of the instruments to be controlled and their insertion in a Grid

    • hard (reservation of IE) and soft (statistical prediction) guarantees of the IE’s methods execution times

    • Fast Data Publication via a Message Oriented middleware (RMM-JMS) to distribute data and information from an IE to the world wide Grid.

  • The IE is highly customizable and can be adapted in different environments.


Conclusion ii

Conclusion - II

  • 3 pilot applications deployed, one of them in a real production environment

  • The IE can be shrunk down into a chip allowing grid enabled embedded control of the instrumentation

  • Several heterogeneous pilot applications are deploying and running the IE middleware

  • We support and encourage the adoption of our middleware in other projects/experiments.


  • Login