Wp 10
This presentation is the property of its rightful owner.
Sponsored Links
1 / 87

WP 10 PowerPoint PPT Presentation


  • 74 Views
  • Uploaded on
  • Presentation posted in: General

WP 10. Information Services. Giovanni Aloisio, Massimo Cafaro, Italo Epicoco. [email protected] [email protected] [email protected] University of Lecce, Italy. Outline. Grid Information Services Globus Toolkit 2.x MDS Configuring & adding information providers

Download Presentation

WP 10

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Wp 10

WP 10

Information Services

Giovanni Aloisio, Massimo Cafaro, Italo Epicoco

[email protected]@[email protected]

University of Lecce, Italy


Outline

Outline

  • Grid Information Services

  • Globus Toolkit 2.x MDS

  • Configuring & adding information providers

  • GridLab WP 10 activities

  • Work in progress

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Grid information services

Grid Information Services

  • System information is critical to operation of the grid and construction of applications

    • What resources are available?

      • Resource discovery

    • What is the “state” of the grid?

      • Resource selection

    • How to optimize resource use

      • Application configuration and adaptation?

  • We need a general information infrastructure to answer these questions

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Examples of useful information

Examples of useful Information

  • Characteristics of a compute resource

    • IP address, software available, system administrator, networks connected to, OS version, load

  • Characteristics of a network

    • Bandwidth and latency, protocols, logical topology

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Grid information facts of life

Grid Information: facts of Life

  • Information is always old

    • changing system state

    • need to provide quality metrics

  • Distributed state hard to obtain

    • complexity of global snapshot

  • Component will fail

  • Scalability and overhead

  • Many different usage scenarios

    • heterogeneous policy, different information organizations, different queries, etc.

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Grid information service

Grid Information Service

  • Provide access to static and dynamic information regarding system components

  • A basis for configuration and adaptation in heterogeneous, dynamic environments

  • Requirements and characteristics

    • Uniform, flexible access to information

    • Scalable, efficient access to dynamic data

    • Access to multiple information sources

    • Decentralized maintenance

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


The gis problem many information sources many views

VO C

?

?

?

?

VO A

VO B

The GIS Problem: Many Information Sources, Many Views

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


What is a virtual organization

What is a Virtual Organization?

  • Facilitates the workflow of a group of users across multiple domains who share (some of) their resources to solve particular classes of problems

  • Collates and presents information about these resources in a uniform view

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Outline1

Outline

  • Grid Information Services

  • Globus Toolkit 2.x MDS

  • Configuring & adding information providers

  • GridLab WP 10 activities

  • Work in progress

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Two classes of mds servers

Two Classes of MDS Servers

  • Grid Resource Information Service (GRIS)

    • Supplies information about a specific resource

    • Configurable to support multiple information providers

    • LDAP as inquiry protocol

  • Grid Index Information Service (GIIS)

    • Supplies collection of information which was gathered from multiple GRIS servers

    • Supports efficient queries against information which is spread across multiple GRIS server

    • LDAP as inquiry protocol

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds 2 2 architecture

MDS 2.2 Architecture

Customized Aggregate Directories

Users

A

A

Enquiry

Protocol

Registration

Protocol

R

R

R

R

Standard Resource Description Services

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Information protocols

Information Protocols

  • Grid Resource Registration Protocol

    • Support information/resource discovery

    • Designed to support machine/network failure

  • Grid Resource Inquiry Protocol

    • Query resource description server for information

    • Query aggregate server for information

    • LDAP V3.0 in Globus

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds 2 2 service architecture

MDS 2.2 Service Architecture

VO-specific Aggregate Directories

discovery (GRIP?)

?

A

A

lookup (GRIP)

registration (GRRP)

R

R

R

R

standard Resource Description services

  • Dynamic Registration via Reg. Protocol (GRRP)

  • Resource Inquiry via Info. Protocol (GRIP)

    • Co-located with resource on network

  • Resource Discovery (via GRIP or other)

    • Using GRIP allows resource/directory hierarchy

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Distributed services

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

registration

R

fault-partition

messages

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

R

replicated directories

divergent directories

VO-A

VO-B

Distributed Services

D

D

D

D

  • Service scales with Grid growth

  • Loose consistency model tolerates failures

  • Interoperability by GRIP/GRRP protocols

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Metacomputing directory service

Metacomputing Directory Service

  • Uses LDAP as Inquiry

  • Access information in a distributed directory

    • Directory represented by collection of LDAP servers

    • Each server optimized for particular function

  • Directory can be updated by:

    • Information providers and tools

    • Applications (i.e., users)

    • Backend tools which generate info on demand

  • Information dynamically available to tools and applications

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Ldap details

LDAP Details

  • Lightweight Directory Access Protocol

    • IETF Standard

    • Stripped down version of X.500 DAP protocol

    • Supports distributed storage/access (referrals)

    • Supports authentication and access control

  • Defines:

    • Network protocol for accessing directory contents

    • Information model defining form of information

    • Namespace defining how information is referenced and organized

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds components

MDS Components

  • LDAP 3.0 Protocol Engine

    • Based on OpenLDAP with custom backend

    • Integrated caching

  • Information providers

    • Delivers resource information to backend

  • APIs for accessing & updating MDS contents

    • C, Java, PERL (LDAP API, JNDI)

  • Various tools for manipulating MDS contents

    • Command line tools, Shell scripts & GUIs

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Grid resource information service

Grid Resource Information Service

  • Server which runs on each resource

    • Given the resource DNS name, you can find the GRIS server

      • well known port = 2135

  • Provides resource specific information

    • Much of this information may be dynamic

      • Load, process information, storage information, etc.

      • GRIS gathers this information on demand

  • “White pages” lookup of resource information

    • Ex: How much memory does machine have?

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Grid index information service

Grid Index Information Service

  • GIIS describes a class of servers

    • Gathers information from multiple GRIS servers

    • Each GIIS is optimized for particular queries

      • Ex1: Which Alliance machines are >16 process SGIs?

      • Ex2: Which Alliance storage servers have >100Mbps bandwidth to host X?

    • Akin to web search engines

  • Organization GIIS

    • The Globus Toolkit ships with one GIIS

    • Caches GRIS info with long update frequency

      • Useful for queries across an organization that rely on relatively static information (Ex1 above)

  • “Yellow pages” lookup

    • Ex: Which machines have large memory?

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Server registration

Server Registration

  • A GRIS or GIIS server can be configured to (de-) register itself during startup/shutdown

    • Targets specified in configuration file

  • Soft-state registration protocol

    • Good behavior in case of failure

  • Allows for federations of information servers

    • E.g. Argonne GRIS can register with both Alliance and DOE GIIS servers

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Soft state registration

Soft-state Registration

  • Periodic notification

    • “Service/resource is available”

    • Expected-frequency metadata

  • Automatic directory construction

    • Add new resources to directory

    • Invite resources to join new directory

  • Self-cleaning

    • Reduce occurrence of “dead” references

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Hierarchical mds deployment

Hierarchical MDS Deployment

GIIS

GIIS

GRISes

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds commands

MDS Commands

  • LDAP defines a set of standard commands

    ldapsearch, etc.

  • Globus Toolkit defines MDS-specific commands

    • grid-info-search, grid-info-host-search

  • APIs are defined for C, Java, etc.

    • C: OpenLDAP client API

      • ldap_search_s(), …

    • Java: JNDI

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Information services api

Information Services API

  • RFC 1823 defines an IETF draft standard client API for accessing LDAP databases

    • Connect to server

    • Pose query which returns data structures contains sets of object classes and attributes

    • Functions to walk these data structures

  • Globus does not provide an LDAP API and recommends the use of OpenLDAP, an open source implementation of RFC 1823.

  • LDAP APIs available in other languages

    • E.g. Java JDNI, Perl, Python, etc.

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds 2 2 features

MDS 2.2 Features

  • Security Mechanisms

    • GSI mutual-authentication

    • Fine-grained access control by GSI name

  • Performance Enhancements

    • Better query speeds

    • Less stale information

  • New Information Model (schema)

    • Better representation of computers

    • Cleaner namespace management

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds 2 2 software stack

MDS 2.2 Software Stack

  • OpenLDAP 2.0.x

    • Implements LDAPv3 protocol

    • Client and server components

  • Cyrus-SASL

    • Generic security

    • We provide loadable SASL/GSS plugin

  • Globus GSI

    • Provides GSS-API interface to PKI

    • Shared library used by our SASL plugin

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds 2 2 information model

MDS 2.2 Information Model

  • Structural information

    • Resource hierarchy maps to objects

    • Named positions in LDAP DIT

  • Merged information

    • Some parents “join” child data

    • Simplifies common query patterns

  • Auxiliary information

    • Uniform representation of leaf/parent data

    • Uses LDAP auxiliary objectclasses

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gris host objects

hostname

hn=

CPU

RAM

DISK

CPU

VM

NET

software=

OS

dev group=

CPUs

CPU

CPU

dev group=

memory

dev group=

disk

dev group=

net

RAM

DISK

NET

dev=

cpu 0

dev=

cpu 1

CPU

CPU

VM

dev=

RAM

dev=

VM

dev=

/scratch1

dev=

eth0

RAM

VM

DISK

NET

GRIS Host Objects

OS

OS

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gris object hierarchy

GRIS Object Hierarchy

Mds-Host-name=hostname

Mds-Software-Deployment=operating system

Mds-Device-Group-name=processors

Mds-Device-name=cpu 0

Mds-Device-Group-name=memory

Mds-Device-name=physical memory

Mds-Device-name=virtual memory

Mds-Device-Group-name=filesystems

Mds-Device-name=/scratch1

Mds-Device-name=/scratch2

Mds-Device-Group-name=networks

Mds-Device-name=eth0

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gris structural class hierarchy

GRIS Structural Class Hierarchy

Mds

Attr: Mds-validfrom (like createtime)

Attr: Mds-validto (accuracy metadata)

Attr: Mds-keepto (discard metadata)

MdsHost

MdsDevice

MdsDeviceGroup

MdsSoftwareDeployment

  • Every MDS object: name, time metadata

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gris auxiliary class examples

GRIS Auxiliary Class Examples

  • Once per CPU

  • Once per CPU

  • Once per SMP

  • Once per MPP

MdsCpu

Attr: Mds-Cpu-vendor

Attr: Mds-Cpu-model

Attr: Mds-Cpu-speedMHz

MdsCpuCache

Attr: Mds-Cpu-Cache-L1kB

MdsCpuSmp

Attr: Mds-Cpu-Smp-size

MdsCpuTotal

Attr: Mds-Cpu-Total-count

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gris auxiliary class examples1

GRIS Auxiliary Class Examples

MdsCpuFree (once per SMP)

Attr: Mds-Cpu-Free-1minX100

Attr: Mds-Cpu-Free-5minX100

Attr: Mds-Cpu-Free-15minX100

MdsCpuTotalFree (once per MPP)

Attr: Mds-Cpu-Total-Free-1minX100

Attr: Mds-Cpu -Total-Free-5minX100

Attr: Mds-Cpu -Total-Free-15minX100

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Client tools

Client Tools

  • Globus Toolkit includes 2 command line client tools for querying MDS services

    • grid-info-search: General purpose client

      • grid-info-search –h <host> -p <port> -b <base> \ -T <timeout> [<filter>] [<attributes>]

      • -x: Anonymous access

    • grid-info-host-search: Same as grid-info-search, but defaults to GRIS standard port

      • E.g. grid-info-host-search –h localhost

  • Both clients can search for specific system information and filter results.

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Filtering

Filtering

  • Filters allow selection of object based on relational operators

    • =, ~=,<=, >=

      • grid-info-search “cputype=*”

  • Compound filters can be construct with Boolean operations

    • &, |, !

      • grid-info-search “(&(cputype=*)(cpuload1<=1.0))”

      • grid-info-search “(&(hn~=sdsc.edu)(latency<=10))”

  • Hints:

    • white space is significant

    • use -L for LDIF format

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Outline2

Outline

  • Grid Information Services

  • Globus Toolkit 2.x MDS

  • Configuring & adding information providers

  • GridLab WP 10 activities

  • Work in progress

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Wp 10

Configuration files for Registration

dc-n1.isi.edu

grid-info-site-policy.conf

grid-info-resource-register.conf

grid-info-resource-ldif.conf

GIIS

GRIS

dc-n2.isi.edu

grid-info-resource-register.conf

grid-info-resource-ldif.conf

GRIS

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Configuration files for registration

Configuration files for Registration

  • A resource that is hosting a GIIS

    • grid-info-site-policy.conf

      • Determines whether to accept incoming registrations

      • Accept everything, or only registrations from the resources explicitly defined in this conf file (can use wildcards)

        • Default policy is to only accept registrations from self, and from port 2135

        • In a hierarchical GIIS environment, this file must be modified from the default

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Grid info site policy conf

grid-info-site-policy.conf

Default Policy Data:

objectclass: MdsRegistrationPolicy

policydata: (&(Mds-Service-hn=mako.isi.edu)(Mds-Service-port=2135))

Change ‘policydata’ to this for completely open policy:

(Mds-Service-hn=*)

Change ‘policydata’ to this to restrict to 2 specific hosts:

(&(|(Mds-Service-hn=dc-n2.isi.edu)(Mds-Service-hn=dc-n3.isi.edu))(Mds-Service-port=2135))

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Configuration files for registration1

Configuration files for Registration

  • A resource registering GRIS information with a GIIS

    • grid-info-resource-register.conf

      • Which GIIS’s this GRIS should register to, and how

      • GIIS could be on the same machine, but may not be

    • grid-info-resource-ldif.conf

      • Determines which GRIS providers are active and available to send data to the GIIS’s that this GRIS is registering with

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Registering a gris with a giis

Registering a GRIS with a GIIS

  • dn: <LDAP add object DN>

  • regtype: mdsreg2

  • reghn: <host to send reg to>

  • regport: <port to send reg to>

  • regperiod: <how often to send reg (seconds)>

  • [service attribute/value]...

    • where service attribute entries depend on the type of LDAP object being published

$GLOBUS_LOCATION/etc/grid-info-resource-register.conf

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Example gris on dc n2 registers with giis on dc n1

ExampleGRIS on dc-n2 registers with GIIS on dc-n1

  • dn: Mds-Vo-Op-name=register, Mds-Vo-name=site, o=grid

  • regtype: mdsreg2

  • reghn: dc-n1.isi.edu

  • regport: 2135

  • regperiod: 600

  • type: ldap

  • hn: dc-n2.isi.edu

  • port: 2135

  • rootdn: Mds-Vo-name=local, o=grid

  • ttl: 1200

  • timeout: 20

  • mode: cachedump

  • cachettl: 30

$GLOBUS_LOCATION/etc/grid-info-resource-register.conf

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Registration control parameters

Registration Control Parameters

  • regperiod

    • How often this GRIS will send a message to the GIIS noting its existence

  • ttl

    • How long the recieving GIIS should keep the registration information before assuming that the GRIS is no longer available

    • In general ttl should be: ttl = 2 x (regperiod)

  • cachettl

    • Recommendation to the GIIS about how long to maintain in cache, the GRIS information provided by this resource

$GLOBUS_LOCATION/etc/grid-info-resource-register.conf

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds 2 2 gris providers

MDS 2.2 GRIS Providers

  • grid-info-cpu reports CPU/load info

  • grid-info-fs reports filesystem info

  • grid-info-mem reports RAM/VM info

  • grid-info-net reports NIC/net info

  • grid-info-os reports OS info

  • grid-info-platform reports arch. info

  • grid-info-merged merges all host info*

  • Extensible for other sources, e.g. GRAM

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gris dispatch logic

GRIS Dispatch Logic

  • For each provider:

    • Could search intersect provider? No, then skip.

    • Is provider cache stale? Yes, then refill.

    • Apply search filter to cache data.

  • Combine all intersecting providers’ results

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gris response issues

GRIS Response Issues

  • MDS 2.2 is lazy

    • Probes are not issued unless queried

  • Some system probes are slow

    • “Best” probe may take several seconds

  • How to avoid stale data?

    • Clients set time-out per query

    • GRIS/GIIS define time-out per source

    • Fresh data found before timeout is returned

    • Cache fill continues after client time-out

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


New information providers

New Information Providers

  • Decide what information to publish into MDS

  • Create a program that implements the I/O interface requirements of a GRIS Information Provider

  • Enable the information provider for an MDS installation

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


New information providers1

New Information Providers

  • Decide what information to publish into MDS

    • OID: conflict avoidance

      • OID Registered with IANA Private Enterprise Numbers

        • http://www.iana.org/cgi-bin/enterprise.pl

        • 1.3.6.1.4.1.3536.* Globus OID subspace

        • 1.3.6.1.4.1.3536.2.* Globus Info Services OID subspace

        • 1.3.6.1.4.1.3536.2.6.* MDS OID subspace

  • GridLab project OID: 1.3.6.1.4.1.3536.2.6.3536.9

    • Object Naming: conflict avoidance

      • ISI will coordinate prefix naming. Request a prefix from:

        • mailto:[email protected]

    • GridLab prefix: “GridLab”

    • Schema

      • Data Modeling problem

      • LDAP schema syntax problem

      • $GLOBUS_LOCATION/etc/grid-info-resource.schema

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


New information providers2

New Information Providers

  • Create your program

    • Any language. The only requirements are the I/O interface of your program:

      • Must be callable by fork and exec from the slapd process

        • You can pass in arguments to your program

      • Must return data in LDIF format

        • Data returned must match the LDAP schema

        • RFC 2849

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


New information providers3

New Information Providers

  • Enable your new information provider by adding a configuration block

  • $GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf

    • # generate memory info every minute

    • dn: Mds-Device-Group-name=memory, Mds-Host-hn=mako.isi.edu, Mds-Vo-name=local, o=grid

    • objectclass: GlobusTop

    • objectclass: GlobusActiveObject

    • objectclass: GlobusActiveSearch

    • type: exec

    • path: /globus/libexec

    • base: grid-info-mem-linux

    • args: -devclassobj -devobjs -dn Mds-Host-hn=mako.isi.edu,Mds-Vo-name=local,o=grid -validto-secs 60 -keepto-secs 60

    • cachetime: 60

    • timelimit: 10

    • sizelimit: 3

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


New information providers4

New Information Providers

  • dn

    • Where the object lives in the DIT

  • These lines must be included:

    • objectclass: GlobusTop

    • objectclass: GlobusActiveObject

    • objectclass: GlobusActiveSearch

    • type: exec

  • path

    • Path to the information provider program

  • base

    • Name of the information provider program

  • args

    • Arguments to be passed to the information provider program

  • cachetime

    • In seconds, how long GRIS will consider the data to not be stale

  • timelimit

    • In seconds, how long the GRIS should wait for the information provider to return data before giving up on it

  • sizelimit

    • Max number of LDIF objects to be read from the output of the information provider

$GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Outline3

Outline

  • Grid Information Services

  • Globus Toolkit 2.x MDS

  • Configuring servers & adding information providers

  • GridLab WP 10 activities

  • Work in progress

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gridlab wp 10 tasks

GridLab WP 10 Tasks

  • T10.1 Analysis of current state of the art GIS. Completed

  • T10.2 Comparison of GIS capabilities and GridLab dynamic grid computing requirements. Completed

  • T10.3 Specification of information model. Completed

  • T10.4 Security mechanisms & policies.

  • T10.5 Development of extended GIS

  • T10.6 Development of high-level APIs

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gridlab wp 10 deliverables

GridLab WP 10 Deliverables

  • D10.1 Analysis report. Delivered

  • D10.2 Requirements & specification. Delivered

  • D10.3 Prototype release. Delivered

  • D10.4 Extended GIS release

  • D10.5 First release report

  • D10.6 Second release report

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gridlab wp 10 milestones

GridLab WP 10 Milestones

  • M10.1 Month 3 Requirements analysis report. On time

  • M10.2 Month 6 Technical specification. On time

  • M10.3 Month 12 Prototype GIS server. On time

  • M10.4 Month 24 First release of GIS

  • M10.5 Month 36 Documented second release of GIS

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Requirements of gridlab mds

Requirements of GridLab MDS

  • Performance

  • Scalability

  • Security

  • Uniformity

  • Expressiveness

  • Extensibility

  • Multiple sources

  • Dynamic data

  • Flexible access

  • Deployability

  • Decentralized control

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Wp 10 current release

WP 10: Current release

  • MDS schema extension to provide information about:

    • Services

    • Software

    • Users

    • Firewalls

    • Virtual Organizations

    • Certification Authorities

  • Access to MDS

    • We developed a GSI enabled version of gSOAP supporting mutual authentication and delegation of credentials

    • Developed GSI enabled Web Services in order to:

      • Register information

      • Unregister information

      • Lookup information

      • Generic MDS search

    • Developed GSI enabled clients for the above functionalities

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Implementation

Implementation

  • GT 2.x based

  • C based

    • Uses LDAP API

  • WEB SERVICE

    • GSI enabled; uses gSOAP Toolkit

    • WSDL description for clients

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


The gsoap tooolkit

The gSOAP Tooolkit

  • by Robert Van Engelen, Florida State University, USA

  • development environment for C/C++ Web services

  • easy-to-use RPC compiler: produces the stub and skeleton routines to integrate (existing) C or C++ applications into SOAP/XML Web services

  • automatically maps native C/C++ application data types to semantically equivalent XML types and vice versa

  • simple API relieving the user from the burden of SOAP details

  • SOAP 1.1/1.2 and WSDL 1.1

  • Zlib compression, SSL encryption, and streaming direct internet message encapsulation (DIME) attachments

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


The gsi plugin for gsoap

The GSI plugin for gSOAP

  • By Massimo Cafaro, Daniele Lezzi (University of Lecce) and Robert Van Engelen (FSU)

  • Overrides gSOAP callbacks to provide transport-level GSI security

  • Provides mutual authentication and delegation of credentials

  • Very simple API relieving the user from the burden of GSI details

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


A gsi enabled client

A GSI enabled client

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


A gsi enabled threaded web service

A GSI enabled, threaded Web Service

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Information about services

Information about Services

  • GridLab-Mds-Service-name: service name

  • GridLab-Mds-Service-port: service port

  • GridLab-Mds-Service-type: service protocol

  • GridLab-Mds-Service-description: service description

  • GridLab-Mds-Service-publisher: service publisher

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Information about software

Information about Software

  • GridLab-Mds-Software-name: name

  • GridLab-Mds-Software-version: version

  • GridLab-Mds-Software-path: pathname

  • GridLab-Mds-Software-totalLicences: number of total software licences

  • GridLab-Mds-Software-freeLicence: number of available software licences

  • GridLab-Mds-Software-licenceInfo: information about software licence

  • GridLab-Mds-Software-startupEnvironment: multi-valued software startup environmental variables

  • GridLab-Mds-Software-executable: software executable

  • GridLab-Mds-Software-arguments: multi-valued software arguments

  • GridLab-Mds-Software-description: software description

  • GridLab-Mds-Software-helpURL: software help URL

  • GridLab-Mds-Software-usage: software usage

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Information about users

Information about Users

  • GridLab-Mds-User-ID: user’s login name on local resource

  • GridLab-Mds-User-Mapped-DN: multi-valued attribute representing the Distinguished Namemapped on the user

  • GridLab-Mds-User-homedir: user’s home directory

  • GridLab-Mds-User-shell: user’s shell

  • GridLab-Mds-User-UID: user’s UID

  • GridLab-Mds-User-GID: user’s GID

  • GridLab-Mds-User-comment: a short comment about the user

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Information about firewall

Information about Firewall

  • GridLab-Mds-Firewall-hostname: firewall hostname

  • GridLab-Mds-Firewall-ports: multi-valued attribute representing open ports (range)

  • GridLab-Mds-Firewall-validityTime: time frame during which open ports ranges are valid

  • GridLab-Mds-Firewall-adminDN: Distinguished Name of firewall administrator

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Information about vos

Information about VOs

  • GridLab-Mds-Vo-name: Virtual Organization to which a specified computational resourcebelongs to

  • GridLab-Mds-Vo-helpDeskPhoneNumber: multi-valued help desk phone number

  • GridLab-Mds-Vo-helpDeskURL: URL pointing to a Virtual Organization’s web page

  • GridLab-Mds-Vo-adminName: administrator name of the VO

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Information about cas

Information about CAs

  • GridLab-Mds-Certificate-Subj: Distinguished Name of the recognized certification authority

  • GridLab-Mds-Certificate-version: CA’s certificate version

  • GridLab-Mds-Certificate-serialNumber: CA’s certificate serial number

  • GridLab-Mds-Certificate-signatureAlgorithm: CA’s certificate signature algorithm

  • GridLab-Mds-Certificate-issuer: CA’s certificate issuer

  • GridLab-Mds-Certificate-validity-from: beginning date of the CA’s certificate validity

  • GridLab-Mds-Certificate-validity-to: end date of the CA’s certificate validity

  • GridLab-Mds-Certificate-publicKeyAlgorithm: CA’s certificate public key algorithm

  • GridLab-Mds-Certificate-RSAPublicKey: CA’s certificate RSA public key

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds web service

MDS Web Service

  • gridlab_search

    • this method can be used to query an arbitrary MDS server to retrieve specific information

  • gridlab_getServiceDescription

    • this method can be used to get the description of the service

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds web service1

MDS Web Service

  • gridlab_register_service

    • this method allows the user to register a new service.

  • gridlab_unregister_service

    • this method removes an existing service from the GIS

  • gridlab_lookup_service

    • this method lookups an existing service and returns the information about the services that have been found. It contacts the supplied MDS server on the default port

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds web service2

MDS Web Service

  • gridlab_register_software

    • this method allows the user to register a new software package

  • gridlab_unregister software

    • this method removes an existing software package from the GIS

  • gridlab_lookup software

    • this method lookups an existing software package and returns all the information about thesoftware. It contacts the supplied MDS server on the default port

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds web service3

MDS Web Service

  • gridlab_register_firewall

    • this method allows the registration of information about a firewall installed on the grid resource.

  • gridlab_unregister_firewall

    • this method removes information about a registered firewall from the GIS

  • gridlab_lookup_firewall

    • this method lookups firewall information. It contacts the suppliedMDS server on the default port

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds web service4

MDS Web Service

  • gridlab_register_vo

    • this method allows the user to register the virtual organization towhich the grid resource belongs.

  • gridlab_unregister_vo

    • this method removes information about a Virtual organization from the GIS

  • gridlab_lookup_vo

    • this method lookups information about a specified virtual organization. It contacts the suppliedMDS server on the default port

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Mds web service5

MDS Web Service

  • gridlab_lookup_user

    • this method lookups the information about a specified user. It contacts the supplied MDS serveron the default port

  • gridlab_lookup_cert

    • this method lookups the information about a specified certification authority. It contacts thesupplied MDS server on the default port

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Outline4

Outline

  • Grid Information Services

  • Globus Toolkit 2.x MDS

  • Configuring & adding information providers

  • GridLab WP 10 activities

  • Work in progress

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Wp10 current activities

WP10: Current Activities

  • Configuration of the GridLab MDS to allow GSI connections through SASL binding

  • Development of C APIs (library) to access MDS

    • Anonymous Binding and GSI-SASL Binding

  • Analisys of the GT3 Index Service

    • To migrate GridLab MDS when GT3 is ready for production

  • Development of information providers for:

    • Information about jobs submitted to PBS scheduler

    • Information about PBS queues

  • Development of GSI-plugin for gSOAP

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gridlab mds sasl configuration

GridLab MDS – SASL Configuration

  • MDS supports the following security features

    • Authentication through Cyrus SASL

    • Privacy and integrity protections through the use of TLS

    • Access Control to database entries based on LDAP authorization information

      • IP address

      • Domain name

      • Others

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gridlab mds sasl configuration1

GridLab MDS – SASL configuration

  • grid-info-slapd.conf

    • it sets basic access control rules and sets anonymous binding.Additionally, this file designates the GIIS and GRIS provider components to OpenLDAP,establishes LDAP and MDS information schema, and defines back ends supported by theslapd server. The anonymousbind and access to parameters in this file set anonymousbinding and access control, respectively.

  • grid-info-site-policy.conf

    • it controls the acceptance of registration messages by aGIIS. This file can be used to create an open policy where all registrants are welcome, ora closed system whereby only specified resources can register with a GIIS.

  • grid-info-resource-register.conf

    • it lists the GIIS servers to which a GRIS or “child”GIIS will register directly. The default is to register to the local GIIS on the host. Thisfile identifies host names, ports, and time values that control registration messages from aGRIS or GIIS to a GIIS server.

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gridlab mds sasl configuration2

GridLab MDS – SASL configuration

  • Mutual authentication between GRIS-GIIS and GIIS-GIIS in a hierarchy

    • In file grid-info-resource-register.conf set:

      • bindmethod: AUTHC-ONLY

  • SASL authorization is performed through:

    • Screening against the grid-mapfile

    • Static ACLs stored in file grid-info-slapd.conf

    • To activate SASL authorization set, in file grid-info-slapd.conf:

      • Anonymousbind no

      • Access to <what> [by <who> <access>]+

        • What: select the entries and/or attributes to which the access apply

        • Who: which entities are granted access (DN)

        • Access: specifies the type of access granted (search, read, write, compare, …)

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gt3 index service

GT3 Index Service

  • MDS functionality appears throughout GT3

    • GT3 GRAM service data (GRAM reporter)

    • GT3 Index Service (GIIS)

    • OGSI Service Data interface

  • Information services capabilities are now more tightly integrated in (the right places in) GT3 than they were in GT2.

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gt3 index service1

GT3 Index Service

  • Dynamic service data creation and management via information provider programs

  • Aggregation of service data from multiple instances

  • Registration of Grid service instances

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gt3 servicedataproviders

GT3 ServiceDataProviders

  • SimpleSystemInformation

    • Java native system probe. This provider enumerates the following data; CPU count, Memory statistics, OS type, and Logical Disk Volumes.

  • HostScriptProvider

    • Linux-specific set of shell scripts that monitor system-specific host data.

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gt3 index service2

GT3 Index Service

  • Three main mechanisms

    • Poll

    • Automated cache refilling

    • Subscribe to / receive notifications from arbitrary services

  • The Index Service will aggregate Service Data it receives

  • No fixed schema for resource data

  • Information Providers now output XML fragments

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gt2 3 index service differences

GT2 & 3 Index Service differences

  • Subscription & notifications

    • This subsumes the role of the GRIS backend server module in GT2

  • GRAM, GridFTP, … Service Data providers

    • These Service Data sources subsume the role of the sample GRAM and host-information provider scripts in GT2

  • Service Data Descriptions in Service Type WSDL

    • These subsume the role of MDS schema written in the RFC2252 LDAP schema format.

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


C apis for accessing mds

C APIs for accessing MDS

  • struct mdsInfo *gridlab_search (char *giis, int port, char *base_dn, int scope, char **attributes, char *filter, int verbose)

    • Binds anonimously to an MDS server and queries it

  • struct mdsInfo *gridlab_search_auth()

    • Binds through GSI-SASL to an MDS server and queries it

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


New information providers5

New Information Providers

  • To publish queues information

  • Currently working on the OpenPBS scheduler

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


Gsi plugin for gsoap

GSI plugin for gSOAP

  • Continuous development to ensure compatibility with the latest gSOAP version

  • Addition of new features, as requested by the users

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


References

References

  • GridLab WP 10 Information Services

    • http://www.gridlab.org/WorkPackages/wp-10/index.html

  • Globus Toolkit MDS

    • http://www.globus.org

  • gSOAP Toolkit

    • http://www.cs.fsu.edu/~engelen/soap.html

  • GSI plugin for gSOAP

    • http://sara.unile.it/~cafaro/gsi-plugin.html

2nd GridLab Conference

Eger - Hungary, March 31 - April 1 2003


  • Login