grid parallelization and tests n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Grid parallelization and tests PowerPoint Presentation
Download Presentation
Grid parallelization and tests

Loading in 2 Seconds...

play fullscreen
1 / 22

Grid parallelization and tests - PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on

Grid parallelization and tests. CERN GRACE Final Review Amsterdam, 15-16 February 2005. Contents. Two GRACE Grid integration models: M1, M2 Pre-conditions for the tests Work performed General test results Model 1 test results Simulation of Model 2 Model 2 tests results Comparison

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Grid parallelization and tests' - gryta


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
grid parallelization and tests

Grid parallelization and tests

CERNGRACE Final ReviewAmsterdam, 15-16 February 2005

contents
Contents
  • Two GRACE Grid integration models: M1, M2
  • Pre-conditions for the tests
  • Work performed
  • General test results
  • Model 1 test results
  • Simulation of Model 2
  • Model 2 tests results
  • Comparison
  • Conclusions

GRACE Review February 2005 - Amsterdam

slide3

Application workflow

Single search Grid workflow

Approach used: M1 - M2

M1

M2

GRACE Review February 2005 - Amsterdam

slide4

Pre-conditions

  • Adopted Content and Categorization Engines release 4.45. These components have been later on improved and optimized by the partners
  • A convenient testing corpus of documents has been selected (English documents, correct pdf to txt conversion, small and large sizes)
  • Configuration problems of GILDA replica manager have been solved (intervention of site administrators)
  • Search result set size is considered in average between 0.1 and 4 MBs of text
  • The Usage of DAG for the job model in GILDA has been discarded

GRACE Review February 2005 - Amsterdam

slide5

Work performed

  • Preparation of a test plan and report template
  • Creation of the testing corpus of documents
  • Verification of testing pre-conditions
  • Creation of the test scripts for semi-automatic testing
  • Testing on Gilda testbed
  • Creation of scripts for validation of output and parsing of logging
  • Collection and analysis of the results

GRACE Review February 2005 - Amsterdam

slide6

General tests

Model 1

Model 2

Testing: job submission

  • general (RM, RB, functional, etc.) tests started in October 2004
  • main testing period November 2004
  • submitted more than 1000 jobs

GRACE Review February 2005 - Amsterdam

slide7

Variable Parameters

GRACE Review February 2005 - Amsterdam

slide8

Graphs

GRACE Review February 2005 - Amsterdam

slide9

Results

Results collected and published on a study and test report

GRACE Review February 2005 - Amsterdam

general tests
General tests

GRACE Review February 2005 - Amsterdam

slide11

Functional tests

  • The functional tests were successful. Problems related to the Grid nodes configuration were experienced and fixed:
  • RB Configuration Problems
  • RM/SE Configuration Problems

GRACE Review February 2005 - Amsterdam

slide12

Performance tests (I)

Depends on input data size

On empty queues

Depends on GRACE performance

Variable

Depends on output data size

I = Input Size in MB

GRACE Review February 2005 - Amsterdam

slide13

Grid overhead

retrieving

queuing

brokering

submission

Grid overhead is 3 minutes in average

GRACE Review February 2005 - Amsterdam

slide14

Performance tests (II)

The Grid performed well, job success rate > 80%

GRACE Review February 2005 - Amsterdam

model 1
Model 1

GRACE Review February 2005 - Amsterdam

slide16

Normalization

Categorization

M1 performaces: execution time/input size

Tests performed on machines with different specifications

The normalization job is the most demanding

GRACE Review February 2005 - Amsterdam

model 2
Model 2

GRACE Review February 2005 - Amsterdam

slide18

M2 description

  • Search results are split outside the Grid
  • Grid parallel jobs execute Text normalization
  • Jobs are monitored for status
  • Results are stored on the Grid (Replica Manager)
  • Grid Categorization job executes:
    • normalized documents merging from SEs
    • categorization processing
  • Job is monitored and results retrieved

GRACE Review February 2005 - Amsterdam

slide19

Waiting time at UI

Computing time

M2 Simulation

Increase due to

job submission

overhead

α{

α

GRACE Review February 2005 - Amsterdam

Kopt1

Kopt2

Kopt

slide20

UI waiting time

Normalization

Grid overhead

Categorization

M2 performances

execution time/n. of parallel jobs

Input size=2MB

execution time/input size

Splitting parameter = 9

GRACE Review February 2005 - Amsterdam

slide21

Model 1

Model 1

Model 2

Model 2

Comparison M1 and M2

execution time/input size

computing time/input size

GRACE Review February 2005 - Amsterdam

slide22

Conclusions

  • Parallelization proved to improve application performances and lower the query failure rate
  • Grid performed well: low failure rate, prompt reply of Grid administrators to problems, good coordination with Gilda team

GRACE Review February 2005 - Amsterdam