gridlab 2003 4 steady leadership in changing times l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
GridLab 2003/4 „Steady leadership in changing times!” PowerPoint Presentation
Download Presentation
GridLab 2003/4 „Steady leadership in changing times!”

Loading in 2 Seconds...

play fullscreen
1 / 31

GridLab 2003/4 „Steady leadership in changing times!” - PowerPoint PPT Presentation


  • 356 Views
  • Uploaded on

GridLab 2003/4 „Steady leadership in changing times!” Jarek Nabrzyski Project Coordinator Poznan Supercomputing and Networking Center GridLab Project Funded by the EU (5+ M € ), January 2002 – December 2004 Application and Testbed oriented

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'GridLab 2003/4 „Steady leadership in changing times!”' - paul


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
gridlab 2003 4 steady leadership in changing times

GridLab 2003/4„Steady leadership in changing times!”

Jarek Nabrzyski

Project Coordinator

Poznan Supercomputing and Networking Center

Meeting with Sun Microsystems at PSNC: Exploitation, 13 May 2004

gridlab project
GridLab Project
  • Funded by the EU (5+ M€), January 2002 – December 2004
  • Application and Testbed oriented
    • Cactus Code, Triana Workflow, all the other applications that want to be Grid-enabled
  • Main goal: to develop a Grid Application Toolkit (GAT) and set of grid services and tools:
    • Resource management (GRMS),
    • Data management (GDMS),
    • Monitoring (Mercury) and information services,
    • Adaptive components (Pythia),
    • Mobile user support and remote visualization,
    • Security services (GAS),
    • Portals (GridSphere),
  • ... and test them on a real testbed with real applications
gridlab members
GridLab Members
  • PSNC (Poznan) - coordination
  • AEI (Potsdam)
  • ZIB (Berlin)
  • Univ. of Lecce
  • Cardiff University
  • Vrije Univ. (Amsterdam)
  • SZTAKI (Budapest)
  • Masaryk Univ. (Brno)
  • NTUA (Athens)
  • Sun Microsystems GmbH
  • HP France
  • ANL (Chicago, I. Foster)
  • ISI (LA, C.Kesselman)
  • UoWisconsin (M. Livny)
  • collaborating with:
    • Users!
      • EU Astrophysics Network,
      • DFN TiKSL/GriKSL
      • NSF ASC Project
    • other Grid projects
      • Globus, Condor,
      • GrADS,
      • PROGRESS, Clusterix
      • GriPhyn/iVDGL,
      • European Grid Projects (GRIDSTART)
      • KISTI, Canadian Grid, LSU…
review issues and concerns
Review Issues and Concerns
  • It is GridLab’s first direct contact with the new PO and 2 out of 3 reviewers are new to the project,
  • Difficult to review a project of this size and complexity in several hours,
  • Decision to present the project from theintegration angle,
  • Problems with synchronization of deliverables with the last-minute amendment (details in the WP presentations)
  • Presentations: Short time per workpackage, short time for the demo,
  • Innovative GAT, the portal and services presented in a practical demo during SC2003 in Phoenix
  • The more features we add (to every single WP out of 12) the more complex the integration gets
gridlab aims
GridLab Aims
  • Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community, ...),
  • Make it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizations
  • Dream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.
what gridlab isn t
What GridLab isn’t
  • We are not developing low level Grid infrastructure - we do more actually, we want to be infrastructure independent,
  • We do not want to repeat work which has already been done (want to incorporate and assimilate it …)
    • Globus APIs,
    • OGSA/OGSI/…,
    • ASC Portal (GridSphere/Orbiter),
    • GPDK,
    • GridPort,
    • DataGrid,
    • GriPhyn,
    • ...
project goals for 2003
Project Goals for 2003
  • Integration of all the GridLab software,
  • “Zakopane Scenario” driving all the developments and integration,
  • First release of GridLab software scheduled for December 2003,
  • Make the testbed more stable and ready for any demo action and/or development,
  • SC2003 was chosen to demo the GridLab software,
  • Strong dissemination of the project in order to get attention of interesting Grid applications,
  • Start the GAT and other GridLab standardization work under GGF,
  • Strengthen management of the project, with clear responsibilities to ensure better control of resources, fast identification of the risks, ensure progress,
  • GridLab Open Source License,
  • Prepare the plan for exploitation of the results,
  • Adjust the Technical Description of Work to the current needs of the project
what have we done during last 8 months
What have we done during last 8 months?
  • Response to the reviewers report sent to all
    • Open Source License
    • GT2 vs. GT3 issues (finally it was wise decision)
    • Strategy for dissemination in 2004
    • Deliverables, some delays here (see details in the WP presentations)
    • Contract amendment is on the way
  • GridLab meeting in Eger (April) and Olomouc (October)
    • More developments and integration
    • More and more WPs brought to the scenario
    • Today most of the WPs were integrated with each other
  • SC2003 in Phoenix (2 big „demos”, many „small” demos)
    • Painful demo, but successful: GRMS + portal + GAT application + Testbed + WP5 information service + Adaptive with some last minute problems with other services
    • „The devil is in the details” - the most true sentence.
major achievements
Major achievements
  • Integration efforts and devotion of project staff,
    • Meeting in Eger in March/April, meeting in Olomouc in October,
    • Many small integration meetings (Poznan-Berlin-Potsdam, Potsdam-Budapest, Potsdam-Prague, Cardiff, GGF meetings and at other confs.),
  • Getting a major attention towards the GAT of many application communities,
    • GRIDSTART, GGF, application communities and 6FPP (EGEE, DEISA, HPC-Europa, …)
    • Expectations are high!
  • Many successful implementations of services
    • Single GridLab services being used by other projects,
    • Most of the services are fully integrated with each other, but can be configured as stand alone
    • GAT can connect to most of the services. All will be available in a month.
major achievement cont
Major achievement (cont.)
  • GridLab middleware being exploited by other projects and production testbeds
    • GridSphere: tens of projects (Grid Infrastructure Project, GEON, HPC-Europa, Physics Portal Development Project, …)
    • GRMS (Clusterix, SGI Grid, Progress, Canadian Grid)
    • Mercury Monitor (DataGrid, P-GRADE)
    • WP8 Visualization - GriKSL
  • Open Source License
    • GridSphere has it’s own Open Source license
major achievements standards collaboration
Major achievementsStandards, collaboration
  • Global Grid Forum
    • Positions in GGF
      • Alexander Reinefeld - GGF Advisory Board
      • Jarek Nabrzyski – step down from the SC and AD for Apps and now take care of the GSA RG),
      • Ed Seidel, Thilo Kielman chairing the Apps RG
      • Thilo Kileman chairing Grid CPR WG
      • Andre Merzky chairing the Persistent Archive WG and secretary of Data Area and Persistent Environments
    • GAT architecture put at the Apps and APM groups
    • Many WG talks about GridLab
    • Putting the GAT to the BOF in October 2003
      • Very successful
    • Participation in Gridstart!
      • GridLab chairs at least half of the TWGs
  • Other
    • Projects: GriPhyN, GRIKSL, CrossGrid, Reality Grid (and some others) have a certain level of collaboration with GridLab
    • Companies: Grid Systems, Platform Computing, IBM, HP, Sun
major achievements dissemination
Major achievementsDissemination
  • Improved the www.gridlab.org
  • Several new papers accepted for books, journals and conferences
  • Many talks have been given at major conferences and events
  • Events organized
    • GridLab Conference and Workshop in Eger, March-April 2003
    • Grid Portals Conference, August 2003, Edinburgh
    • Grid Application Workshop in Czestochowa, September 2003
    • GridLab Conference and Workshop in Olomouc, October 2003
    • Grid Resource Management Workshop, Poznan, October 2003
    • Workshops at international conferences and symposia
  • Tutorials on major conferences, also via AccessGrid
  • 2 GridLab brochures
  • In 2004 we will produce 14 different GridLab brochures featuring all the services, tools and advanced scenarios (see first three issues)
book on grid resource management
Book on Grid Resource Management
  • 4 GridLab chapters:
    • Application requirements for resource brokering
    • Multicriteria resource management (GRMS)
    • Triana peer-to-peer RM
    • Workflow management
  • ISI/Argonne/Wicsonsin contributions
  • Published in November by Kluwer Academic Publishers
technical progress
Technical Progress
  • GAT API specification and GAT adaptor developer’s guide
  • GAT tested by Cactus and a GAT subset implementation for Triana (GAP)
  • Strenghten the collaboration with ISI, ANL, Wisconsin
  • New Portal Framework (GridSphere) - 1st release in 2003 (2nd in 2004) - being used by tens of projects!
  • Participation in the Grid Technology Repository (Jarek Nabrzyski serves as the Int. Advisory Board member)
  • Strong participation in the GRIDSTART project (GridLab was one of the main contributors to the Technical Roadmap)
  • Services ready and interoperable in spite of changing requirements
  • GAT ready for standardization process, already under the community comment process
technical progress cont
Technical Progress (cont.)
  • Extending the GridLab scenario
  • Successful meeting in Eger (new application users involved, new developments) and in Olomouc (integration)
  • Problems/Issues
    • GAT development delayed - underestimated efforts needed for the GAT development
      • Should be fully ready in a month
      • Will be revisited and refactored! User’s input is needed. Good docs is needed.
      • Hiring a new person, experienced with Cactus development to speed up the CGAT integration.
    • GAT documentation is behind! We are shifting efforts a bit to speed up the process.
    • Some requirements and scenarios have changed dramatically: new ones appeared (WP7)
    • Some unexpected problems with hackers on one of the test machines
issues
Issues
  • GAT development delayed - underestimated efforts needed for the GAT development
    • Should be fully ready in a month
    • Will be revisited and refactored! User’s input is needed. Good docs is needed.
    • Hiring a new person, experienced with Cactus development to speed up the CGAT integration.
  • GAT documentation is behind! We are shifting efforts a bit to speed up the process.
  • Some deliverables are delayed
    • New Annex1 caused some problems
    • GAT development delayed (CGAT, GAT documentation)
  • Some requirements and scenarios have changed dramatically: new ones appeared (WP7)
  • Some unexpected problems with hackers on one of the test machines
plans for 2004
Plans for 2004
  • Continue the development of GAT and services
    • more info in the following presentations
  • More complicated scenarios
    • collaborative environments,
    • submitting, controling and steering jobs from mobile devices,
    • more dynamic behavior of applications
  • Two more GridLab Workshop Meetings (Lecce: 16-22 May, Zakopane: early December)
  • Organizing the GridLab/GT3.2/WSRF integration meeting withUS partners (to ensure GAT compatibility)
  • Preparing for the Supercomputing demos
  • GGF BOF (GAT) and finally GGF WG
  • GGF Scheduling Architecture Working Group now started (GRMS)
  • Exploitation of the project results
    • Close work with GridLab’s commercial partners
    • Global Grid Application Alliance with GridLab’s leadership
    • GGF activities (long term)
    • GridSuit (based on GridLab plus Gridstart Open Source, PSNC plus partners)
      • Open Source + commercial support
    • European Grid Support Centre (PSNC)
summary
Summary
  • GAT is important
    • We will put yet more effort into the GAT
    • GAT will be revisited after finalizing the 1st release (in one month)
    • Refactoring
    • Lecce Meeting to bring application people, TWG8, tutorial, exploitation
    • Documentation!
    • GGF/Gridstart
  • There was a great interest and we are talking to many groups
  • The project works better than ever!
    • Project is unique in this sense (this might be an impression of ours only)
  • Integration was much more difficult than expected
  • We had to deal with many (too many) unexpected problems in 3rd party software.
collaboration with argonne
Collaboration with Argonne
  • OGSA C bindings
  • GridLab/GT3.2 Inegration Workshop (4 Argonne GT3 developers/experts are coming to help us with the integration)
  • Help with developing the GT3.2 adaptors
  • GAT review
exploitation plan

Exploitation Plan

Jarek Nabrzyski

Meeting with Sun Microsystems at PSNC: Exploitation, 13 May 2004

pillars of exploitation
Pillars of exploitation
  • Close work with GridLab’s commercial partners
  • Global Grid Application Alliance with GridLab’s leadership
  • GGF activities (long term)
  • GridSuite (based on GridLab plus Gridstart Open Source, PSNC plus partners)
  • European Grid Support Centre (PSNC, HP)
  • European Grid Technology Broker
exploitation of the results through commercial partners
Exploitation of the results through commercial partners
  • HP
    • Grid Resource Toolkit. HP investigates which GL software can be adopted.
    • Distribution through server/software distribution
  • Sun
    • This meeting!
  • Actions to be taken:
    • GridLab will work closely with the companies to ensure that the companies evaluate the software and make right decisions on which software is supposed to be part of their distributions
    • Responsible: WP managers will provide software plus documentation to a contact person at the vendor side.
global grid application alliance
Global Grid Application Alliance
  • Global
    • Europe - GridLab
    • US - Ed Seidel
    • Asia Pacific - both Satoshis
    • Korea - KISTI
  • Tasks
    • Bring the GAT and services to the community
    • Train people to make then use the GAT
    • People will write adaptors for their favorite services
    • Work on the new scenarios that the Alliance will bring
    • Work on portable applications on a global scale using the GridLab technologies
  • Actions:
    • The initial meeting of the founders took already place
    • SC will be responsible for the actions: meetings soon to be scheduled
ggf gridstart activities
GGF, GRIDSTART activities
  • SAGA
  • GSA
  • and more will be continued beyond the timeframe of the project
  • Gridstart TWGs will be used to work with other Grid projects in order to deploy the GAT and GridLab tools in these projects
  • TWG chairs (from GridLab) are responsible to make it happen (GridLab WP managers, SC members)
gridsuite at psnc
GridSuite at PSNC
  • GridSuite (based on GridLab plus Gridstart open source, PSNC plus partners) - Grid out-of-the-box complete solution
    • Open Source + commercial support
  • PSNC has declared to hire a team of 6-8 persons to build the GridSuite this year
  • PSNC will work with GridLab partners to provide an expertise to build the GridSuite and to design a business plan for the effort.
  • Release roadmap is under way.
  • This might be a background for the commercial company selling the Grid services for European industry
  • Actions to be taken:
    • PSNC will discuss this with the GridLab Steering Committee soon,
    • This will be more open to other GridLab partners (not limited to PSNC)
    • Strategy is being developed by PSNC now
    • Initial money from the European infrastructure funds (under way) and PSNC’s own funds.
european grid support centre
European Grid Support Centre
  • European Grid Support Centre (HP, PSNC plus UK eScience, CERN, PDC)
  • Institutions’ own funds
  • The task is to support different user groups (currently DataGrid, EGEE, GridLab, NorduGrid)
  • The tools and the procedures are now being established.
  • The ground for GridLab’s support for its users and infrastructure builders,
  • Currently PSNC and HP from GridLab only. More will be involved with increase of the number of users.
  • Actions:
    • Build the procedures and assure that GridLab user’s know about the Center and use its services,
    • To be discussed by the SC in more details in 6 weeks time,
    • Extension of the group with a time and load (currently one person half-time at PSNC, similarly at HP, 3-4 persons at PSNC soon).
exploitation by partners general strategy
Exploitation by partners - general strategy
  • Every GL partner will work with the users that the SC agrees to work with closely,
  • In such a case particular institution will make sure that appropriate resources are assigned to work with the user in order to deploy GridLab software in the user’s environment,
  • The appropriate resolutions will be worked out in details and decided by the Steering Committee by the existence of the GAT release, no later than by the end of April.
  • At the beginning we will focus on strategic users to ensure the added value to the project.
  • We will work closely with the GridLab application communities first of all (Cactus and Triana) on the GAT usage.
beyond the project
Beyond the project
  • Seek for local funding by every partner
  • Survive the gap after the GridLab
  • Continue work, bring more users, get new requirements, analyze scenarios, work with the GGAA (Global Grid Application Alliance)