Idc hpc user forum 09 08 2009
This presentation is the property of its rightful owner.
Sponsored Links
1 / 13

IDC HPC User Forum 09/08/2009 PowerPoint PPT Presentation


  • 68 Views
  • Uploaded on
  • Presentation posted in: General

IDC HPC User Forum 09/08/2009. Manuel Hoffmann, Vice President, Channel Development [email protected] +1.408.342.0337. ScaleMP at a Glance. 100+ Deployments Worldwide. V irtualization for high-end computing

Download Presentation

IDC HPC User Forum 09/08/2009

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Idc hpc user forum 09 08 2009

IDC HPC User Forum 09/08/2009

Manuel Hoffmann, Vice President, Channel Development

[email protected] +1.408.342.0337


Scalemp at a glance

ScaleMP at a Glance

  • 100+ Deployments Worldwide

Virtualization for high-end computing

deliveringhigher performance and lower Total Cost of Ownership through aggregation of multiple x86 off-the-shelves servers into a single large virtual shared memory system

  • Founded in 2003

  • Product shipping since 2006

  • Sold through Tier-1 and Tier-2 OEMs


Virtualization

Virtualization ?

  • “a technique for hiding the physical characteristics of computing resources from the way in which other systems, applications or end users interact with these resources”

    • Wikipedia

Partitioning

Providing a virtual resource that is a subset of the physical resource

Aggregation

Providing a virtual resource that is a concatenation of several physical resources

Utilization

(Disk Partitioning)

Flexibility and Capability

(RAID and Volume Manager)

Flexibility

(VLANs)

Availability and Capability

(Link Aggregation)

Utilization

(Server Virtualization)

???


Server virtualization

Server Virtualization

AGGREGATION

Concatenation of physical resources

PARTITIONING

Subset of the physical resource

Virtual Machines

Virtual Machine

App

App

App

App

OS

OS

OS

OS

Hypervisor or VMM

Hypervisor or VMM

Hypervisor or VMM

Hypervisor or VMM

Hypervisor or VMM


How does it work

How Does It Work?

Multiple off-the-shelf x86 servers, with processors and memory

Processors speed/amount or memory amount does not have to be same across all boards

InfiniBand HCAs, cables and switch

vSMP Foundation™ Devices

The flash-devices plug into the boards and used as bootable device. vSMP Foundation is booted to present an aggregate coherent view to the OS

HIGH-END X86 SYSTEM, BASED ONSTANDARD X86 COMPONENTS


Behind the scenes

Behind The Scenes

SEAMLESS INTEGRATION

One System

  • Software interception engine creates a uniform execution environment

  • vSMP Foundation creates the relevant BIOS environment to present the OS (and the SW stack above it) as single coherent system

    Coherent Memory

  • vSMP Foundation maintains cache coherency between boards

  • Multiple concurrent memory coherency mechanisms, on a per-block basis, based on real-time memory activity access pattern

  • Leverage board local-memory for caching

    Shared I/O

  • vSMP exposes all available I/O resources to the OS in a unified PCI hierarchy

  • No need for cluster file systems

HIGHEST X86 SMP MEMORY BANDWIDTH!


Server virtualization aggregation

Server Virtualization Aggregation

  • SMP

Cost Savings &

Performance

Virtual Machine

App

OS

High core-count / Large memory

  • Cluster

Manageability

Hypervisor or VMM

Hypervisor or VMM

Hypervisor or VMM

Hypervisor or VMM

New management paradigm for small clusters: 4 to 64 nodes

AGGREGATION

Concatenation of physical resources

  • Cloud

Flexibility

On-the-fly provisioning for compute grids: unlimited scaling


Vsmp foundation aggregation platform

vSMP Foundation Aggregation Platform

  • SMP

  • Cluster

  • Cloud

Cost Savings & Performance

Manageability

Flexibility

  • Cost Savings

  • Up to 5X cost savings

  • Performance

  • Leveraging latest Intel processors

  • Best x86 solution by SPEC CPU2006

  • #7th best shared-memory by STREAM (memory bandwidth)

  • Reliability

  • Fault detection and component isolation

  • Redundant backplane support

  • Capabilities

  • Up to 128 cores and 4TB RAM

  • Manageability

  • Single Operating System (OS) for up to 16 nodes

  • OS driven job scheduling and resource management

  • Performance

  • InfiniBand performance with zero management and knowhow

  • Storage

  • Built-in cluster file system

  • Installation

  • Unboxing to production in less than 3 hours

  • Flexibility

  • On-the-fly aggregated VM provisioning and tear-down

  • Scaling memory or CPU

  • Utilization

  • Resource fragmentation reduction

  • Support any programming model (serial, throughput, multi-threaded, large-memory) without machine boundary

  • Integration

  • Network installation provides seamless integration with any grid management system


Target environments and applications

Target Environments and Applications

Target Environments

Typical end-user applications

Manufacturing

CSM (Computational Structural Mechanics)

ABAQUS/Explicit

ABAQUS/Standard

ANSYS Mechanical

LSTC LS-DYNA

ALTAIR Radioss

NASTRAN

CFD (Computational Fluid Dynamics)

FLUENT

ANSYS CFX

STAR-CD

AVL FIRE

Tgrid

Other

inTrace OpenRT

Life Sciences

Gaussian

VASP

AMBER

Schrödinger Jaguar

Schrödinger Glide

NAMD

DOCK

GAMESS

GOLD

mpiBLAST

GROMACS

MOLPRO

OpenEye FRED

OpenEye OMEGA

SCM ADF

HMMER

Energy

Schlumberger ECLIPSE

Paradigm GeoDepth

3DGEO 3DPSDM

Norsar 3D

  • Users seeking to simplify cluster complexities

  • Applications that use large memory footprint (even with one processor)

  • Applications that need multiple processors and shared memory

EDA

Mentor

Cadence

Synopsys

Finance

Wombat

KX

Others

The MathWorks MATLAB

R

Octave

Wolfram MATHEMATICA

ISC STAR-P

Weather Forecasting

MM5

WRF


Example vsmp foundation cluster

Example vSMP Foundation Cluster

Challenges:

  • Need to run MPI as well as OpenMP codes

  • System needs to be deployed remotely, and hence needs to be simple to manage

  • Data processing flow is complex and requires transferring large amounts of data between steps

    Applications: MM5, WRF, MAWSIP, Home-grown code for data transformation

    Solution:

  • 4 Intel Nehalem dual socket blades, total of 8 sockets (32 cores) and 192GB RAM

  • Internal storage

  • Solution was extended to 8 blades, total of 16 sockets (64 cores) and 384GB RAM

    Benefits:

  • Performance: 2.5 X better performance on same # of cores (32)

  • Simpler solution: Significantly reduced capital expense, allowedthe customer to have a higher # of cores

  • Simplicity: Simple to manage by domain experts (weather forecast scientists)

  • Dataflow remains within the system, leveraging internal storage

WEATHER FORECASTING SERVICE PROVIDER

SIMPLE AND FLEXIBLE COST EFFECTIVE SOLUTION


Example vsmp foundation smp

Example: vSMP Foundation SMP

Challenges:

  • Need to generate large mesh as part of pre-processing of whole-carsimulation (FLUENT TGrid). Mesh requirements are ~200GB in size

  • Expect to grow significantly within 12 months after initial deployment

  • Would like to standardize on x86 architecture due to lower costs and open standards

    Solution:

  • 12 Intel dual-processor Xeon systems to provide 384GB RAM single virtual system running Linux with vSMP Foundation

    Benefits:

  • Better performance: Solution evaluated and found to be faster thanalternative systems (x86 and non-x86)

  • Cost: Significant savings compared to alternative system

  • Versatility: Also being used to run FLUENT (MPI) as part of large cluster

  • Investment protection: Solution can grow

FORMULA1 TEAM

SCALEUPAT SCALEOUTPRICING


Example vsmp foundation cluster1

Example: vSMP Foundation Cluster

Challenges:

  • A single 4-socket server did not provide enough performance required for customer business targets

  • Multiple 4-socket servers required complex decomposition and introduced challenges in transferring data between processes in a short and deterministic time (low latency and small jitters)

  • Co-location at exchanges for a solution comprised of multiple systems is complicated

    Applications: KX, WOMBAT, Home-grown code

    Solution:

  • 16 Intel dual-processor Xeon systems to provide 0.5TB RAM, 32 sockets (128 cores) single virtual system running Linux with vSMP Foundation

    Benefits:

  • Reduced latency and latency variance

  • Simpler solution: Deploy and management of a single system

  • Better utilization: Single system reduces resources fragmentation

  • Simpler programming model: No need for specific InfiniBandprogramming

FINANCIAL SERVICES

SIMPLIFYING INTER-PROCESS COMMUNICATION


Example vsmp foundation cloud

Example: vSMP Foundation Cloud

Problems:

  • Need to provision systems for MPI as well as OpenMP (shared memory) codes

  • Large shared memory jobs currently require dedicated proprietary hardware

  • Low utilization on shared memory systems

    Applications: A variety of commercial and customer codes

    Solution:

  • Original: 4 systems, total of 8 sockets (32 cores) and 128GB RAM

  • Solution was extended to 16 nodes

    Benefits:

  • Utilization: Rely on standard commodity hardware

  • Flexibility: Using same system for both shared memoryand cluster benchmarks, resulting in high utilization

HOSTED HPC RESOURCE PROVIDER

COST EFFECTIVE FLEXIBLE SOLUTION WITH HIGH UTILIZATION


  • Login