Charon syst e m http egee cesnet cz en voce charon html
This presentation is the property of its rightful owner.
Sponsored Links
1 / 24

CHARON Syst e m  egee.cesnet.cz/en/voce/Charon.html PowerPoint PPT Presentation


  • 60 Views
  • Uploaded on
  • Presentation posted in: General

Petr Kulh á nek , 1,2 Jan Kmun íč ek, 1,3 and Martin Pet ř ek 1,2 1. CESNET, Czech Republic 2. National Centre for Biomolecular Research, Masaryk University in Brno, Czech Republic 3.Institute of Computer Science, Masaryk University in Brno, Czech Republic.

Download Presentation

CHARON Syst e m  egee.cesnet.cz/en/voce/Charon.html

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Charon syst e m http egee cesnet cz en voce charon html

Petr Kulhánek,1,2 Jan Kmuníček,1,3 and Martin Petřek1,2

1. CESNET, Czech Republic

2. National Centre for Biomolecular Research, Masaryk University in Brno, Czech Republic

3.Institute of Computer Science, Masaryk University in Brno, Czech Republic

CHARON Systemhttp://egee.cesnet.cz/en/voce/Charon.html


Charon syst e m egee cesnet cz en voce charon html

CHARON System

  • Contents

    • GRID Vocabulary

    • Summary of User Authentication/Authorization in GRID

    • Job Submission in Native LCG Middleware

    • Charon Extension Layer

      • Overview

      • Module System

      • Job Submission and Management

      • Configuration

    • Conclusions

  • Examples are in:

    skurut4.cesnet.cz:/home/kulhanek/cesnet_seminar/<example>

    • input directory contains fresh input data

    • test directory contains results of test run


  • Charon syst e m egee cesnet cz en voce charon html

    CHARON System

    • GRID Vocabulary

      • VOCE– Virtual Organization for Central Europe

      • UI – user interface

      • SE – storage element

      • CE – computing element

      • WN – worker node


    Charon syst e m egee cesnet cz en voce charon html

    CHARON System

    • Summary of User Authentication/Authorization in GRID

      • grid-proxy-init orgrid-proxy-init-sc

        • grid-proxy-info

      • grid-proxy-destroy

      • myproxy-init or myproxy-init-sc

        • myproxy-info

        • myproxy-get-delegation

      • myproxy-destroy

        NOTE: grid-proxy-init-sc and myproxy-init-sc are not a regular part of LCG distribution. They serve for proxy certificate initialization using secure cards (HW Tokens).


    Charon syst e m egee cesnet cz en voce charon html

    CHARON System

    • Job Submission in Native LCG Middleware

      Example #01:01_lcg_test

      edg-job-submitlcg_job.jdl

      edg-job-statushttps://skurut3.cesnet.cz:9000/AXyE-vQ04qcFfM-kQFe6QQ

      edg-job-get-output–dir ./ https://skurut3.cesnet.cz:9000/AXyE-vQ04qcFfM-kQFe6QQ

    # LCG Test Job (lcg_job.jdl)

    Type = "Job";

    JobType = "Normal";

    Executable = "/bin/hostname";

    StdOutput = "lcg_job.stdout";

    StdError = "lcg_job.stdout";

    OutputSandbox = {"lcg_job.stdout"};

    Arguments = "";

    RetryCount = 2;


    Charon system

    CHARON System

    What is Charon?

    uniform and modular approach for (complex) computational jobs submission and management

    generic system for use of application programs in the Grid environment (LCG middleware, …)

    Why Charon?

    many various batch systems & scheduling components used in grid environment

    each batch system has unique tools and different philosophy of its utilization

    LCG provided tools are quite raw and simple

    many additional tasks to use computer resources properly


    Charon system1

    CHARON System

    Scheme of Charon (CEL - Charon Extension Layer)


    Charon syst e m egee cesnet cz en voce charon html

    CHARON System

    • Activation of CEL on UI of VOCE

      1.login to skurut4.cesnet.cz

      2.type following command:

      /afs/ics.muni.cz/software/ncbr/voce/migrating/install_charon

      3.logout from skurut4.cesnet.cz

      4.login to skurut4.cesnet.cz again

      NOTE: CEL installer modify .profile and .bashrc files.

    • Deactivation of CEL on UI of VOCE

      1.login to skurut4.cesnet.cz

      2.type following command:

      /afs/ics.muni.cz/software/ncbr/voce/migrating/uninstall_charon


    Charon system2

    CHARON System

    Module system

    manages application software

    each software package is described by a specific module

    configuration information is internally stored in XML format

    find the architecture and parallel mode that best fit available computational resources

    solve conflicts or dependencies between individual modules

    list available modules sorted into categories

    use pre-installed modules on WNs or install them on the fly

    if they are missing


    Charon system3

    CHARON System

    • Commands of Module System

      • module command: module [action] [module1 [module2] …]

        Most Important Actions:

        add (load)

        remove (unload)

        avail

        list*

        * list is default action

        NOTE: Type module -h for short summary info about module command.

      • modview command

        • menu driven configuration of visualization of modules


    Charon system4

    CHARON System

    • Module Name Completion

      Example:

      amber amber:8.0m:auto:auto  amber:8.0m:pn3:single

      user  default values  resolved final name

      pn3 – module is optimised for Pentium III architecture

      single – module is built for sequential execution


    Charon syst e m egee cesnet cz en voce charon html

    CHARON System

    • Module Activation

      $ module add amber

      Module specification: amber (add action)

      ===========================================================

      Architecture : pn3

      Number of CPUs : 1

      Max CPUs per node : 1

      Exported module : amber:8.0m

      Complete module : amber:8.0m:pn3:single

      INFO: Module is taken from local software repository.


    Charon system5

    CHARON System

    • Charon System – Overview of Commands

      Job Submission and Management

      • psubmit

      • pinfo

      • psync

      • pgo(does not work in GRID environment)

        Charon Setup

      • pconfigure

      • palias (deprecated, it will be substituted with pconfigure)

      • pprofile (deprecated, it will be substituted with pconfigure)

      • psum


    Charon syst e m egee cesnet cz en voce charon html

    CHARON System

    • Charon Job Restrictions

      • each job has to be in separate directory

      • job directories must not overlap

      • job is described by script

      • only relative paths to job directory contents have

        to be used in job script

      • software should be activated by Module System


    Charon system6

    CHARON System

    • Example of Charon Job (Example #02: 02_isomaltose)


    Charon system7

    CHARON System

    psubmit <VO/alias> <jobscript> [NCPU] [syncmode]


    Charon system8

    CHARON System

    • pinfo


    Charon system9

    CHARON System

    • psync


    Charon system10

    CHARON System

    • Results


    Charon system11

    CHARON System

    • Other Examples

      • Example #03: 03_complex_job

        psubmit skurut get_pdbs

      • Example #04: 04_parallel_sander

        psubmit skurut md_test 2


    Charon syst e m egee cesnet cz en voce charon html

    CHARON System

    • Charon Configuration

      • Sync Mode – option for data transfer between UI and WN

        • gridcopy

          – all data within job directory as input

          – all data within job directory as result

        • stdout

          – all data within job directory as input

          – only standard output as result (other data are discarded)

      • Resources – identification of particular CE

      • Properties – fine grained selection of computational resources (throught Requirements item in JDL)

      • Alias -uniform combination of above setup in a single word


    Charon syst e m egee cesnet cz en voce charon html

    CHARON System

    • palias

    $ palias

    Alias Queue/VO Syncmode Properties Resources

    --------------------------------------------------------------------------------------

    stdout voce stdout - -

    tuke voce gridcopy - ce.grid.tuke.sk:2119/jobmanager-pbs-voce

    polgrid voce gridcopy - ce.polgrid.pl:2119/jobmanager-lcgpbs-voce

    poznan voce gridcopy - ce.egee.man.poznan.pl:2119/jobmanager-lcgpbs-voce

    skurut voce gridcopy - skurut17.cesnet.cz:2119/jobmanager-lcgpbs-voce

    zeus voce gridcopy - zeus02.cyf-kr.edu.pl:2119/jobmanager-lcgpbs-voce


    Charon syst e m egee cesnet cz en voce charon html

    • Next Release of Charon System (2006)

      • gLite support

      • more batch systems support within single Charon installation

      • simplification of configuration (e.g. only pconfigure will remain)

      • better handling of error states

      • documentation in WIKI

    • What Do We Need ?

      • What type of application software is required by users?

      • What data (type and size) does this software produce or operate with?

      • Do you require parallel execution of application software?

      • suggestions and criticism of Charon System


    Charon syst e m egee cesnet cz en voce charon html

    CHARON System

    • Conclusions

      • Single job management

        • encapsulation of a single computational job

        • minimization of overhead resulting from direct middleware usage

          (JDL file preparation, etc.)

        • easy submission and navigation during job lifetime

      • Application programs management

        • powerful software management and administration

        • comfortable enlargement of available application portfolio

    • Acknowledgements

      • European Commission (contract number IST-2003-508833)

      • Ministry of Education, Youth, and Physical Training of the Czech

        Republic (contract number MSM0021622413)


  • Login