An Introduction to Grid Technologies at NERSC
Download
1 / 38

An Introduction to Grid Technologies at NERSC June 24, 2004 David Turner NERSC User Services Group - PowerPoint PPT Presentation


  • 109 Views
  • Uploaded on

An Introduction to Grid Technologies at NERSC June 24, 2004 David Turner NERSC User Services Group [email protected] 510-486-4027. Topics. Definitions Certificates Acquiring Using Tools and Services Current Soon Future Resources. What is “The Grid”?. The Globus view

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' An Introduction to Grid Technologies at NERSC June 24, 2004 David Turner NERSC User Services Group' - dugan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

An Introduction to Grid Technologies at NERSC

June 24, 2004

David Turner

NERSC User Services Group

[email protected]

510-486-4027


Topics
Topics

  • Definitions

  • Certificates

    • Acquiring

    • Using

  • Tools and Services

    • Current

    • Soon

    • Future

  • Resources


What is the grid
What is “The Grid”?

  • The Globus view

    • Grids are persistent environments that enable software applications to integrate instruments, displays, computational and information resources that are managed by diverse organizations in widespread locations.(http://www.globus.org)

  • The NERSC implementation

    • File transfer and data management

    • Remote job submission

    • Portals


What is globus
What is “Globus”?

  • The Globus Alliance is a research and development project focused on enabling the application of Grid concepts to scientific and engineering computing.

    • Argonne National Laboratory’s Mathematics and Computer Science Division

    • University of Chicago’s Distributed Systems Laboratory

    • University of Southern California’s Information Sciences Institute

    • University of Edinburgh

    • Swedish Center for Parallel Computers


What is globus really
What is “Globus”, Really?

  • The Globus Toolkit is a middleware package from the Globus Alliance

    • De facto standard for Grid “platform”

    • Client software

      • Command-line tools

      • Application development libraries

      • Installed on seaborg, escher, newton, and PDSF

    • Server software

      • Installed on seaborg, escher, newton, PDSF, HPSS, and web servers

  • Version 2.4.3 supported on most systems (PDSF is between 2.2.4 and 2.4.3)

  • Available for “most” desktop systems


Globus software tools
Globus Software Tools

  • Client software

    • Certificate management tools

      • grid-cert-info, grid-proxy-init, grid-proxy-info, grid-proxy-destroy

    • File transfer commands

      • globus-url-copy, uberftp, pftp_gsi

  • Server software

    • GridFTP, Gatekeeper, MyProxy

  • “Portals”

    • Combine characteristics of client and server


Using globus at nersc
Using Globus at NERSC

  • To use client software:

    % module load globus

    % echo $GLOBUS_LOCATION

    /usr/common/globus/gt243

  • GridFTP servers run on:

    seaborg-g1.nersc.gov

    garchive.nersc.gov

    escher.nersc.gov

    pdsfgrid[1-3].nersc.gov

    newton0[1-4].eth1.nersc.gov


Security definitions
Security Definitions

  • Authentication

    • Verifying that someone is who they claim to be

    • Required to run client software

    • Based on certificates

  • Authorization

    • Determining if an authenticated person has access to a particular resource or service

    • Typically implemented with a grid-mapfile

  • Globus security model based on Grid Security Infrastructure (GSI)


Certificate definitions
Certificate Definitions

  • A document attesting to the truth of certain stated facts.

  • A document that is used to certify that a user or organization is who they say they are. They contain information about who it belongs to, who it was issued by, expiry date and information that can be used to check out the contents of the certificate.

  • Implements ISO X.509

    • Public Key Infrastructure (PKI)


Certificate characteristics
Certificate Characteristics

  • Opaque

  • Issued by recognized Certificate Authority (CA)

  • Stored inside a web browser

    • Netscape/Mozilla

      Edit->Preferences->Privacy&Security->Certificates->Manage Certificates

    • Internet Explorer

      Tools->Internet Options->Content->Certificates

    • Exported from browser to disk file

    • Globus tools to query contents of file

  • Can be exported from browser into file

    • Move from system to system


Types of certificates
Types of Certificates

  • Personal certificate

    • a.k.a. client certificate

  • Host certificate

    • a.k.a. server certificate

  • CA certificate

    • a.k.a. root certificate


Certificates at nersc
Certificates at NERSC

  • Acquired from a CA

    • Local institution

    • DOEGrids (administered by ESnet)

    • NERSC

  • Enter Distinguished Name (DN) of Subject and Issuer into NERSC Information Management (NIM) system

    • Required for authorization (to access servers)

    • Propagates to local grid-mapfiles

    • http://nim.nersc.gov


Doegrids certificate process
DOEGrids Certificate Process

usercert.pem

Proxy cert

CA

Browser

“p12” file

x509up_u12345

userkey.pem

NIM

Subject DN

Issuer DN


Getting a doegrids certificate
Getting a DOEGrids Certificate

  • To allow your browser to trust DOEGrids:

    http://www.doegrids.org/pages/How-To-Import.html

    • Follow directions, including restarting browser

  • To acquire personal certificate:

    https://pki1.doegrids.org/

    • Provide all requested fields, click “Submit”

    • Wait for email with further instructions

      • Click on “Import Your Certificate”

  • Personal Certificate now stored in browser

  • Valid for 1 year


Doegrids certificate process1
DOEGrids Certificate Process

usercert.pem

Proxy cert

CA

Browser

“p12” file

x509up_u12345

userkey.pem

NIM

Subject DN

Issuer DN


Got a cert now what
Got a Cert; Now What?

  • Put certificate into “p12” file

    • Netscape Navigator

      Edit->Preferences->Privacy&Security->Certificates->Manage Certificates

      • Select certificate, click “Backup”

      • Provide file name, password(s)

    • Internet Explorer

      Tools->Internet Options->Content->Certificates

      • Select certificate, click “Export”

      • Provide file name, password(s)

    • File name extension: .p12 or .pfx

    • PROTECT THIS FILE!


Doegrids certificate process2
DOEGrids Certificate Process

usercert.pem

Proxy cert

CA

Browser

“p12” file

x509up_u12345

userkey.pem

NIM

Subject DN

Issuer DN


Have p12 file now what
Have “p12” File; Now What?

  • If necessary, copy file to machine with globus installed

    • seaborg, escher, newton, pdsf

  • Extract certificate (which contains public key)

    openssl pkcs12 -in YourCert.p12-clcerts -nokeys -out ~/.globus/usercert.pem

  • Extract private key

    openssl pkcs12 -in YourCert.p12-nocerts -out ~/.globus/userkey.pem

  • Set permissions to owner read/write

    chmod go-rwx ~/.globus/user*.pem

  • Protect these files!


Doegrids certificate process3
DOEGrids Certificate Process

usercert.pem

Proxy cert

CA

Browser

“p12” file

x509up_u12345

userkey.pem

NIM

Subject DN

Issuer DN


Querying your certificate
Querying Your Certificate

% grid-cert-info

Certificate:

Data:

Version: 3 (0x2)

Serial Number: 1918 (0x77e)

Signature Algorithm: sha1WithRSAEncryption

Issuer: DC=org, DC=DOEGrids, OU=Certificate Authorities, CN=DOEGrids CA 1

Validity

Not Before: Jun 17 17:32:11 2004 GMT

Not After : Jun 17 17:32:11 2005 GMT

Subject: DC=org, DC=doegrids, OU=People, CN=David Turner 460392

Subject Public Key Info:

Public Key Algorithm: rsaEncryption

RSA Public Key: (2048 bit)

Modulus (2048 bit):

00:f5:b9:2a:d1:e3:89:cb:49:6d:99:00:93:b1:fe:

4a:56:9e:c4:a2:59:00:3e:0f:02:56:c2:5c:2d:ce:

bb:4f:f9:44:89:01:9b:42:4b:ad:8f:25:53:07:c1:

8e:88:c2:3d:0c:1e:bd:6b:5f:12:1d:b7:57:bd:df:

22:6e:50:88:63:c5:59:d6:e5:65:9e:34:cd:c9:dd:

00:2e:bd:ef:8d:65:03:20:d0:dc:86:33:b3:d9:2d:

15:cd:20:d3:14:d3:63:05:21:82:e3:ab:b5:12:f4:

2e:45:5b:50:72:a3:71:d6:10:cb:e3:8d:ec:1a:e0:

3c:16:9c:f1:71:4e:45:8c:04:49:9c:4d:eb:be:79:

f6:0c:76:cb:66:48:54:a6:d9:94:3c:54:82:c8:46:

72:74:36:99:c1:e4:45:c9:c5:f8:d4:74:a1:6b:fe:

ff:2c:ad:43:13:b0:47:76:5d:7f:65:65:b0:b9:02:

e2:3f:53:5d:77:cb:30:50:42:1a:dd:3e:df:a3:2f:

49:27:66:d9:63:bf:c5:56:07:c4:8d:de:78:c0:08:

fd:f8:5f:b4:ae:5d:61:e4:6d:52:86:d3:cb:d6:76:

28:95:8e:1c:b7:3f:d6:63:87:82:4b:c7:d1:b6:18:

b3:76:99:0c:a9:13:d6:2f:20:56:5d:56:a8:dd:87:

f6:fd

Exponent: 65537 (0x10001)

X509v3 extensions:

Netscape Cert Type:

SSL Client, SSL Server, S/MIME

X509v3 Key Usage: critical

Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment

X509v3 Authority Key Identifier:

keyid:CA:19:1D:12:8E:6E:A4:38:5D:42:D4:31:0E:08:DB:D9:8D:17:0D:5D

X509v3 Subject Alternative Name:

email:[email protected]

Signature Algorithm: sha1WithRSAEncryption

16:80:67:ef:45:d3:0d:c8:15:86:64:16:ca:b1:0e:64:e7:46:

b6:da:b7:cc:e5:3c:54:3f:c6:7d:62:f9:b4:02:b5:e4:3e:74:

ed:5f:61:91:98:df:1f:45:cb:16:2a:17:48:28:8e:56:11:e6:

52:93:73:71:44:bf:7d:24:f8:c8:69:15:e2:7b:dd:38:68:15:

a7:de:e4:d6:7a:c7:da:41:c5:3e:dd:a9:b2:15:2b:74:6f:87:

32:4a:e5:38:db:fc:e6:e4:49:b6:95:25:c6:f8:77:32:e2:b2:

b7:29:46:21:f9:7d:93:11:3e:97:b7:6c:5b:ac:a1:a8:b0:28:

eb:44:99:10:d7:16:6a:39:6a:dc:9e:ef:94:47:2f:ab:1b:02:

be:94:96:a8:3a:83:5c:56:58:13:c7:d6:ae:4b:25:49:d1:aa:

20:c4:d8:f2:0a:01:40:67:18:11:11:d3:5b:69:43:4f:5c:61:

71:45:2c:06:fb:95:ae:2d:b9:f7:d1:0f:eb:5b:91:73:7d:8e:

4b:8b:f9:a6:e4:78:c0:51:14:a5:f6:f6:00:db:00:91:75:f9:

b5:42:5b:43:46:1a:ca:36:25:21:73:b0:1b:21:bb:35:6f:f6:

3e:21:df:c6:50:96:5b:ac:0f:a5:48:3b:88:68:c6:eb:8d:0a:

a5:fe:97:b9


Certificate information in nim
Certificate Information in NIM

  • Login in to NIM

  • Click “Grid Certificates” tab in lower frame


Informing nim of your cert
Informing NIM of Your Cert

% grid-cert-info -subject

/DC=org/DC=doegrids/OU=People/CN=David Turner 460392

% grid-cert-info -issuer

/DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1

  • In NIM, click “Add existing certificate to NIM”

  • Cut-and-paste Subject and Issuer DN

  • Click “Add Certificate”


Doegrids certificate process4
DOEGrids Certificate Process

usercert.pem

Proxy cert

CA

Browser

“p12” file

x509up_u12345

userkey.pem

NIM

Subject DN

Issuer DN


Have pem files now what
Have “pem” Files; Now What?

  • Create “proxy certificate”

    • Usually has a limited lifetime

      • Default 12 hours

      • Maximum 1 year

    • Creates $HOME/.globus/x509up_u12345

      grid-proxy-init [-valid h:m]


Doegrids certificate process5
DOEGrids Certificate Process

usercert.pem

Proxy cert

CA

Browser

“p12” file

x509up_u12345

userkey.pem

NIM

Subject DN

Issuer DN


Querying your proxy cert
Querying Your Proxy Cert

% grid-proxy-info

subject : /DC=org/DC=doegrids/OU=People/CN=David Turner 460392/CN=proxy

issuer : /DC=org/DC=doegrids/OU=People/CN=David Turner 460392

identity : /DC=org/DC=doegrids/OU=People/CN=David Turner 460392

type : full legacy globus proxy

strength : 512 bits

path : /usr/common/homes/d/dpturner/.globus/x509up_u17931

timeleft : 11:58:33


Proxy certificate issues
Proxy Certificate Issues

  • Managing Grid credential files (“pem” files containing certificate/public key and private key) is a nuisance.

  • Security

    • Keys can be stolen if account compromised.

    • Copies on multiple machines increase exposure.

  • One solution: MyProxy server


Using myproxy
Using MyProxy

  • To place Grid credentials into MyProxy server:

    myproxy-init [-t hours]

    • Default lifetime on server is one week.

    • Can now remove “pem” files from local file system.

  • To retrieve proxy delegation:

    myproxy-get-delegation

  • To query credentials on server:

    myproxy-info

  • To remove credentials from server:

    myproxy-destroy


Using myproxy delegation
Using MyProxy Delegation

  • A MyProxy delegation is another form of a proxy certificate:

    grid-proxy-info

    grid-proxy-destroy

  • Once delegation in place, user can run Globus client software.


Isn t there a simpler way
Isn’t There a Simpler Way?

  • NERSC-managed certificates

    • Issued by “NERSC CA” (actually “DOEGrids CA 2”)

    • Created entirely in NIM

    • Delivered directly to MyProxy server

    • One-year lifetime

    • User never handles “p12” file or “pem” files

    • Not widely trusted (YET!)


Getting a nersc certificate
Getting a NERSC Certificate

  • Login to NIM

  • Click “Grid Certificates” tab

  • Click “Create a new NERSC-managed certificate in NIM” link


But what does globus do
But What Does Globus Do?

  • GridFTP file transfer commands

    • globus-url-copy

      globus-url-copy -nodcau gsiftp://seaborg-g1.nersc.gov/path1/file1gsiftp://garchive.nersc.gov/path2/file2

    • uberftp

      • Complete GridFTP interactive client

      • Third-party transfers using “lopen” command

        uberftp -a GSI -P 2811 -H garchive.nersc.gov

    • pftp_gsi

      • NERSC-developed pftp client with GSI authentication

        pftp_gsi garchive.nersc.gov


Other file transfer tools
Other File Transfer Tools

  • GridFTP API

    • Transfer data directly from batch jobs to visualization system at PPPL

  • Grid File Yanker (GFY) portal

    • Reliable transfer

  • VisPortal


Globus future at nersc
Globus Future at NERSC

  • Remote job submission and monitoring

  • NERSC Portal

  • Continuing evolution of authentication and authorization

    • GridLogon


Resources
Resources

  • Web

    http://www.nersc.gov/

    http://www.nersc.gov/nusers/help/access/globus.php

    http://nim.nersc.gov/

    http://www.doegrids.org/

    http://www.globus.org/

    http://dims.ncsa.uiuc.edu/set/uberftp/

    http://www.google.com/

  • Human

    • [email protected]

    • 1-800-66-NERSC, option 3

    • 1-510-486-8600, option 3


The end
The End

This page intentionally left blank.


Still the end
Still The End

This page intentionally left blank.


Still the end really
Still The End, Really

This page intentionally left blank.


ad