SPEC Benchmarks for Large Systems
Download
1 / 22

SPEC Benchmarks for Large Systems - PowerPoint PPT Presentation


  • 111 Views
  • Uploaded on

SPEC Benchmarks for Large Systems Matthias Mueller, Kumaran Kalyanasundaram, G. Gaertner, W. Jones, R. Eigenmann, R. Lieberman, M. van Waveren, and B. Whitney SPEC High Performance Group [email protected] What is SPEC?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' SPEC Benchmarks for Large Systems' - stormy


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

SPEC Benchmarks for Large Systems

Matthias Mueller, Kumaran Kalyanasundaram, G. Gaertner, W. Jones, R. Eigenmann, R. Lieberman, M. van Waveren, and B. Whitney

SPEC High Performance Group

[email protected]


What is spec
What is SPEC?

The Standard Performance Evaluation Corporation (SPEC) is a non-profit corporation formed to establish, maintain and endorse a standardized set of relevant benchmarks that can be applied to the newest generation of high-performance computers. SPEC develops suites of benchmarks and also reviews and publishes submitted results from our member organizations and other benchmark licensees.


Spec high performance group
SPEC High-Performance Group

  • Founded in 1994

  • Mission: To establish, maintain, and endorse a suite of benchmarks that are representative of real-world high-performance computing applications.

  • SPEC/HPG includes members from both industry and academia.

  • Benchmark products:

    • SPEC OMP (OMPM2001, OMPL2001)

    • SPEC HPC2002 released at SC 2002


Currently active spec hpg members
Currently active SPEC HPG Members

  • Fujitsu

  • HP

  • IBM

  • Intel

  • SGI

  • SUN

  • UNISYS

  • University of Purdue

  • University of Stuttgart


Where is spec relative to other benchmarks
Where is SPEC Relative to Other Benchmarks ?

There are many metrics, each one has its purpose

Raw machine performance: Tflops

Microbenchmarks: Stream

Algorithmic benchmarks: Linpack

Compact Apps/Kernels:NAS benchmarks

Application Suites: SPEC

User-specific applications: Custom benchmarks



SPEC OMP

  • Benchmark suite developed by SPEC HPG (High Performance Group)

  • Benchmark suite for performance testing of shared memory processor systems

  • Uses OpenMP versions of SPEC CPU2000 benchmarks and candidates


Spec omp cpu2000 differences
SPEC OMP/CPU2000 differences

  • Larger working set sizes, 1.6GB for OMPM2001, 6.5 GB for OMPL2001; it is 200MB for CPU2000

  • Longer run times (>1000 s/cpu for CPU2000 vs >10,000 s/cpu for OMP2000 medium)

  • Focus on SMP systems, and issued by HPG

  • SPEC OMP based on work for CPU2000, SPEC OMP mixes integer and FP in one suite

  • OMPM is focused on 4-way to 16-way systems

  • OMPL is targeting 32-way and larger systems


Spec omp results
SPEC OMP Results

  • 66 submitted results for OMPM

  • 24 submitted results for OMPL





Spec hpc2002 benchmark
SPEC HPC2002 Benchmark

  • Full Application benchmarks(including I/O) targeted at HPC platforms

  • Serial and parallel (OpenMP and/or MPI)

  • Currently three applications:

    • SPECenv: weather forecast

    • SPECseis: seismic processing, used in the search for oil and gas

    • SPECchem: comp. chemistry, used in chemical and pharmaceutical industries (gamess)

  • All codes include several data sizes


Specenv execution models on a sun fire 6800
SPECenv execution models on a Sun Fire 6800

Medium scales better

OpenMP best for small size

MPI best for medium size



Specseis execution models on a sun fire 6800
SPECseis execution models on a Sun Fire 6800

Medium scales better

OpenMP scales better than MPI



Specchem execution models on a sun fire 6800
SPECchem execution models on a Sun Fire 6800

Medium shows better scalability

MPI is better than OpenMP




Current and future work
Current and Future Work

  • SPEC HPC:

    • Update of SPECchem

    • Improving portability, including tools

    • Larger datasets

  • New release of SPEC OMP:

    • Inclusion of alternative sources

    • Merge OMPM and OMPL on one CD

  • Adoption of new benchmark codes:

    • Should represent a type of computation that is regularly performed on HPC systems

    • We currently examine CPU2004 for candidates


Conclusion and summary
Conclusion and Summary

  • Results of OMPL and HPC2002:

    • Scalability of many programs to 128 CPUs

  • Best choice of programming model (MPI,OpenMP, hybrid) depends on:

    • Hardware

    • Program

    • Data set size

  • SPEC HPC will continue to update and improve the benchmark suites.


ad