neutron science teragrid gateway update n.
Skip this Video
Loading SlideShow in 5 Seconds..
Neutron Science TeraGrid Gateway Update PowerPoint Presentation
Download Presentation
Neutron Science TeraGrid Gateway Update

Loading in 2 Seconds...

play fullscreen
1 / 17

Neutron Science TeraGrid Gateway Update - PowerPoint PPT Presentation

  • Uploaded on

Neutron Science TeraGrid Gateway Update. Vickie Lynch, Meili Chen, John Cobb Oak Ridge National Laboratory AUS Technical Presentation October 8, 2009. Outline. Sinergia Startup Project Fitting Service Transition to SNS. Sinergia Startup Project.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Neutron Science TeraGrid Gateway Update' - karlyn

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
neutron science teragrid gateway update

Neutron Science TeraGrid Gateway Update

Vickie Lynch, Meili Chen, John Cobb

Oak Ridge National Laboratory

AUS Technical Presentation

October 8, 2009

  • Sinergia Startup Project
  • Fitting Service
  • Transition to SNS
sinergia startup project
Sinergia Startup Project
  • Collaboration between the University of Zürich, ETH Zürich and Oak Ridge National Laboratory (ORNL)
  • Swiss Sinergia grant supports PostDoc at Zürich and graduate student at ORNL
diffuse scattering modeling
Diffuse Scattering Modeling
  • 3D fitting of large datasets
  • Needed for Single Crystal instruments
    • Topaz (SNS)
    • Snap (SNS)
    • MaNDi (SNS)
    • Corelli (SNS)
    • Four-circle Diffractometer (HFIR)
    • Imagine (HFIR)
sinergia july 09 kickoff in zurich
Sinergia July 09 Kickoff in Zurich
  • Organized by Hans-Beat Bürgi
  • Michel Chodkiewicz (PostDoc at Zürich) will develop new C++ diffuse scattering code over 3 years
  • Tara Clark (Grad student at ORNL) working with Vickie Lynch will get previous code running on TeraGrid and test
  • Jürg Hauser from University of Bern will provide a reference dataset with less noise
original diffuse scattering code
Original diffuse scattering code
  • Used for publications:
    • Weber T.,Bürgi H.B., Determination and refinement of disordered crystal structures using evolutionary algorithms in combination with Monte Carlo methods, Acta Crystallographica A 58, 526-540 (2002)
    • H.B. Bürgi, J. Hauser, T. Weber, R.B. Neder, Supramolecular Architecture in a Disordered Perhydrotriphenylene Inclusion Compound from Diffuse X-ray Diffraction Data. Crystal Growth & Design, 5 2073-2083 (2005)
  • Master/slave code was written mainly in Perl with telnet to communicate between workstations (Calls Fortran and C codes for FFTs)
  • Much file communication between master and slave
    • Code assumed unique disk space for each slave
  • One run for over 200 generations took ~30 days on ~10 workstations
vop code by phil bentley
VOP code by Phil Bentley
  • Vitess Optimisation Program written for optimizing parameters in instrument simulations
  • Genetic algorithm with tournament selection; Swarms
  • Written in C++ using MPI for parallel computing (Allreduce to find min; Bcast to sent best parameters)
  • Command line input for Vitess with parameters to be optimized preceded by @
  • Limits file read by code
  • Written to maximize flux from Vitess, but easily modified to minimize c2
code port to teragrid
Code port to TeraGrid
  • Replaced master telnet code with VOP MPI code
  • VOP calls Perl slaves with pipes
  • Unique directories created for each slave
  • Ported and ran successfully on NSTG
  • Needed faster turnaround and 1048 clones
  • Got Kraken Startup allocation
  • Popen for Perl slaves did not work with CNL
code port to ranger
Code port to Ranger
  • Tested that popen worked on Ranger
    • Transferred part of Startup to Ranger
  • Too much file communication in Perl slaves for 16 cores/node
  • Put OpenMP directives in FFT codes
  • Using 4 slaves/node with 4 OpenMP threads/slave
sinergia accomplishments
Sinergia accomplishments
  • Add differential evolution option to vop master
  • Compared differential evolution, genetic algorithm and swarm for disordered crystal data. Genetic algorithm converged fasted, but each generation takes twice as long using tournament selection
  • Added hybrid OpenMP directives
  • Have code running on NSTG and Ranger
  • Have Levenberg-Marquardt nonlinear least squares driver for slaves for further optimization
  • Running using reference dataset on Ranger now (160 cores)
future sinergia work
Future Sinergia work
  • Install PGPLOT Perl module and helper scripts
  • Testing with reference dataset
    • Number of individuals
    • Number of clones
    • Optimize mutation and crossover constants
  • Scale to thousands of cores
  • Automatically switch between genetic algorithms and least squares using history of chisq
  • Write slave code in C++, not Perl
  • Communicate between modules with memory, not files
fitting service
Fitting Service
  • Fast data fitting, the major and the most time consuming procedure of the data analysis, is the key of high performance data analysis and real-time data processing.
  • Uses NL2SOL or Dakota to fit experimental data
  • Instrument scientists writes functions
    • Read data
    • Model to fit data
fitting service results
Fitting Service results
  • Fitting service used for experimental reflectometer data is shown
  • NL2SOL results show 5 times smaller Chisq than previous fitting code used for this data
fitting service uses teragrid
Fitting Service uses TeraGrid
  • New fitting service
    • going into production
    • available from the development portal
    • will be “monitored” for health in the future
  • Calculation scales linearly on TeraGrid on Mercury at NCSA
  • See TG09 slides by Meili Chen for more details.
nstg cluster transition to sns
NSTG Cluster Transition to SNS
  • Cluster: dual processor 3.06 GHz Intel Xeon nodes
  • 14 nodes with 2.5GB of memory are compute nodes
  • 4 nodes with 4GB memory are dedicated to GridFTP
  • Transitioning to Scientific Linux 5.2
  • Move planned from ORNL to SNS machine room (~ 3 miles)
  • Cluster will link SNS with TeraGrid