1 / 17

Early Experience of Running WRF and CAM on Ranger

Early Experience of Running WRF and CAM on Ranger. Siddhartha Ghosh Wei Huang Juli Rew National Center for Atmospheric Research June 11, 2008 Contact: huangwei@ucar.edu. Ranger Nodes 4X Quad Core 2GHz AMD Barcelona Processors InfiniBand Interconnection 3,936 16-way SMP nodes

iain
Download Presentation

Early Experience of Running WRF and CAM on Ranger

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Early Experience of Running WRF and CAM on Ranger Siddhartha Ghosh Wei Huang Juli Rew National Center for Atmospheric Research June 11, 2008 Contact: huangwei@ucar.edu National Center for Atmospheric Research Teragrid 08, Las Vagas

  2. Ranger Nodes • 4X Quad Core • 2GHz AMD • Barcelona Processors • InfiniBand Interconnection • 3,936 16-way SMP nodes • Memory Hierarchy • 64KB L1 Cache (per core) • 2MB L2 Cache • 2MB L3 Cache (shared) • 32GB (per node) • 4 FP/clock, 8GF TP National Center for Atmospheric Research Teragrid 08, Las Vagas

  3. Pingpong Test Ranger vs IBM HPS (Federation) National Center for Atmospheric Research Teragrid 08, Las Vagas

  4. WRF Variables and Domain Size • Variables Defined in Registry # table entries are of the form #<Table> <Type> <Sym> <Dims> <Use> <NumTLev> <Stagger> <IO> <DNAME> <DESCRIP> <UNITS> state real u ikjb dyn_em 2 X i01rhusdf=(bdy_interp:dt) "U" "x-wind component" "m s-1" state real v ikjb dyn_em 2 Y i01rhusdf=(bdy_interp:dt) "V" "y-wind component" "m s-1" state real w ikjb dyn_em 2 Z irhusdf=(bdy_interp:dt) ”W" "z-wind component" "m s-1” • Domain Size Defined in “namelist.input” &domains max_dom = 1, s_we = 1, 1, 1, e_we = 74, 112, 94, s_sn = 1, 1, 1, e_sn = 61, 97, 91, s_vert = 1, 1, 1, e_vert = 28, 28, 28, • Developed Python program to read Registry and process “namelist.input” • Get Memory Usage before run National Center for Atmospheric Research Teragrid 08, Las Vagas

  5. Memory Usage (MB) for WRF Dataset 425*300*35 National Center for Atmospheric Research Teragrid 08, Las Vagas

  6. Average Wall-Clock (seconds) Used for Each WRF Integration National Center for Atmospheric Research Teragrid 08, Las Vagas

  7. Memory Usage (MB) for WRF Dataset 1501*1201*35 National Center for Atmospheric Research Teragrid 08, Las Vagas

  8. Average Wall-Clock (seconds) Used for Each WRF Integration National Center for Atmospheric Research Teragrid 08, Las Vagas

  9. WRF Timing on Blueice and Ranger(x-procs, y-seconds/time-step) National Center for Atmospheric Research Teragrid 08, Las Vagas

  10. Community Atmospheric Model (CAM) • A Global Atmospheric Model developed at NCAR in collaboration with researchers elsewhere • Supports many dynamical cores • FV core is becoming popular • Supports few standard out of the box resolutions • In this study we considered 1x1.25 and 0.5x0.625 (lat x lon in degrees) • 2D domain decomposition National Center for Atmospheric Research Teragrid 08, Las Vagas

  11. Performance • Reported in Model-years estimated to be computed per day • Compared with IBM 1.9GHz power5-HPS system at NCAR • NCAR System (Blueice) • Dual core 16-way 1.9GHz IBM pwr5+ nodes • IBM HPS 2-SNI/node with 8 micro-seconds latency and 2GB/ps each SNI each way peak bw National Center for Atmospheric Research Teragrid 08, Las Vagas

  12. CAM perturbation growth test * Give the model a small perturbation * The perturbation should not grow fast National Center for Atmospheric Research Teragrid 08, Las Vagas

  13. Performance comparison at resolution of 1x1.25 National Center for Atmospheric Research Teragrid 08, Las Vagas

  14. Performance comparison at resolution 0.5x0.625 National Center for Atmospheric Research Teragrid 08, Las Vagas

  15. Conclusion • Able to run WRF, CAM, and CCSM on Ranger successfully • WRF scales pretty good on Ranger • Small data set to 512 processors • Larger data set to 2048 processors • CAM and CCSM doesn’t scale so well National Center for Atmospheric Research Teragrid 08, Las Vagas

  16. Acknowledgement and Reference • Acknowledgement • We’d like to thank Rich Loft for providing encouragement in this study and including us in the team of early Ranger users. • We’d also like to thank John R. Boisseau for providing early user access and Karl W. Schulz for providing performance tips and initial hand holding needed to compute on Ranger. • Reference • http://www.tacc.utexas.edu • http://www.ccsm.ucar.edu • http://www.wrf-model.org • Contact • Siddhartha Ghosh: sghosh@ucar.edu • Wei Huang: huangwei@ucar.edu • Juli Rew: juliana@ucar.edu National Center for Atmospheric Research Teragrid 08, Las Vagas

  17. Add small perturbation to the initial field • The perturbation should not grow fast (at the order of 10-9 within 2 days • Used to check the impact of compiler/platform and optimization on model National Center for Atmospheric Research Teragrid 08, Las Vagas

More Related