slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Content PowerPoint Presentation
Download Presentation
Content

Loading in 2 Seconds...

play fullscreen
1 / 33

Content - PowerPoint PPT Presentation


  • 122 Views
  • Uploaded on

GRACE Grid parallelization and tests Roberta Faggian Marque (CERN) with contribution from Jan Fiete Grosse-Oetringhaus GRACE General Project Meeting Stuttgart, 14-15 December 2004. Two GRACE Grid integration models: M1, M2 Pre-conditions for the tests

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Content' - brian


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

GRACEGrid parallelization and testsRoberta Faggian Marque (CERN)with contribution from Jan Fiete Grosse-OetringhausGRACE General Project MeetingStuttgart, 14-15 December 2004

content
Two GRACE Grid integration models: M1, M2

Pre-conditions for the tests

Planning of the Grid tests and first results

M1 test results/performances

Simulation of M2

Parallelization study

M2 tests results/performances

Comparison

Conclusions

EGEE collaboration - update

Content
slide5

Pre-conditions for running the tests

  • GRACE components have been tested and validated: We adopted the components which were made available at the beginning of November on the CVS repository, SANDRA release 4.45. These components have been later on improved and optimized by the partners.
  • A convenient testing corpus of documents for is used: Provided by CERN and approved by partners.
  • Problems with replica manager in GILDA have been solved: Solved after CERN investigation & feedback to GILDA team & site administrators.
  • Only English documents are considered.
  • Search result set size is in average between 0.1 and 4 MB of text.
  • The Usage of DAG for the job model in GILDA has been investigated: DAG will not be used in the implementation of M2 since the current installation on the GILDA testbed is not completed.
work performed
Preparation of a test plan and report template

Creation of the testing corpus of documents

Verification of testing pre-conditions

Creation of the test scripts for semi-automatic testing

Testing on Gilda testbed

Creation of scripts for validation of output and parsing of logging

Collection and analysis of the results

Work performed
testing job submission
Testing: job submission
  • general tests started on 20.10.2004
  • main testing period from 05.11 to 25.11.04
  • submitted more than 1000 jobs - about 1 million Java API calls
functional tests
Functional tests
  • The functional tests were successful. Anyhow, during the testing period some Grid related problems were experienced and fixed:
  • RB Configuration Problems
  • RM/SE Configuration Problems
  • Java API call failures (very low percentage < 0,1%)
performance tests i
Performance tests (I)

Depends on input data size

On empty queues

Depends on GRACE performance

Variable

Depends on output data size

I = Input Size in MB

performance tests iii
Performance tests (III)

I = Input Size in MB

parallelized model m2
Split search results outside the Grid

Launch N parallel jobs for text normalization

Monitor jobs status

Store results on the Grid (using Replica Manager)

Launch Categorization job:

Pick up documents from SEs and merges them

Perform Categorization

Monitor and get results from Categorization job

Parallelized model M2
simulation of m2 performances
Simulation of M2 performances

Increase due

to job submission

overhead

splitting study based on simulation
Rules for splitting parameters

Minimize user interface waiting time  Kopt

Save “unnecessary” resources by splitting less than optimal value  Keff (see graph)

Calculated formulas for splitting parameters

Apply constraints: number of available worker nodes, number of simultaneous users, size of the biggest input file

Implemented in Java class for GRACE application

Splitting study based on simulation

α: % increasing of waiting time at user interface

conclusions
The Grid parallelization work is the extension of

WP6 activity on Grid integration and testing

Conclusions
  • The Grid parallelization has been completed successfully
  • Parallelization proved to improve application performances and low the query failure rate (GL measure this?
  • Grid performed well: low failure rate, prompt reply of Grid administrators to problems, good coordination with Gilda team
egee collaboration update
EGEE collaboration - update
  • Feedback submitted to EGEE user requirements db
  • Grace presented at the EGEE EGAAP commission at the second EGEE conference
  • Grace will be supported in Gilda until the end of the Project (installation in SHU?)
  • GRACE requirements entered in the EGEE requirements db (input from GL missing)
  • Contacts with other projects, collaboration with EGEE networking activities: provided input requirements for project support
  • Member of EGEE user group (supporting action for EGEE user communities)