windows compute cluster server 2003 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Windows Compute Cluster Server 2003 PowerPoint Presentation
Download Presentation
Windows Compute Cluster Server 2003

Loading in 2 Seconds...

play fullscreen
1 / 29

Windows Compute Cluster Server 2003 - PowerPoint PPT Presentation


  • 176 Views
  • Uploaded on

Windows Compute Cluster Server 2003. Ming Xu Program Manager Windows HPC Microsoft Corporation. HPC Market Trend. Windows dominate the x86 server shipments. WW Clustered Servers Forecast. Linux is replacing Unix Windows market is growing. HPC clusters more than 50% revenue, x64 dominates.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Windows Compute Cluster Server 2003' - mallika


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
windows compute cluster server 2003

Windows Compute Cluster Server 2003

Ming XuProgram ManagerWindows HPCMicrosoft Corporation

hpc market trend
HPC Market Trend

Windows dominate the x86 server shipments

WW Clustered Servers Forecast

Linux is replacing Unix

Windows marketis growing

HPC clusters more than 50% revenue,

x64 dominates

hpc market segments and growths
HPC Market Segments And Growths

2004 Systems

Sold (units)

2004 – 9 CAGR

Operated By

Capability, Enterprise$1M+

Standalone Labor cost center

1,167

4.2%

Divisional$250K–$1M

IT organization

3,915

5.7%

Departmental$50–250K

Administrator

22,712

7.7%

Workgroup<$50K

Power user

127,802

13.4%

growth

Source: IDC, 2005

<$250K – 97% of systems, 52% of revenue

In 2004 clusters grew 96% to 37% by revenue

opportunity
Opportunity
  • Departmental and Workgroup segmentis the under-represented majority
    • “Takes too long to setup”
    • “Hard to maintain and update”
    • “Don’t integrate with end-user apps”
  • Needs
    • Customers require simple job submissionand compute performance
    • Administrators require simplified IT environment
    • Developers require standards-basedenvironment and advanced tools

Source – WW Server Forecast IDC, 2005, MS Analysis

how microsoft partners with the industry
How Microsoft Partners With The Industry
  • Provide a complete Microsoft platform solution for HPC
    • Enable rapid migration of ISVs to a standard, stable, and supported platform
    • Differentiate by providing turnkey clusters that securelyintegrate into Windows infrastructure
    • Integrate clusters into end user experience through application/desktop
  • Focus on departmental/workgroup customers in targeted verticals
    • Build deep expertise for Manufacturing, Geosciences,Life Sciences, and Federal
  • Grow the HPC eco-system
    • OEM and networking partners to sell pre-configured HPC clusters
    • Collaborate with HPC cluster ISVs for each targeted verticals
  • Establish credibility in the HPC community
    • Engage Open Source projects and technologies in HPC stack
    • Create world-wide Innovation Centers aroundtop HPC researchers and practitioners
windows compute cluster server 20031
Windows Compute Cluster Server 2003

Mission: Deliver the easiest to deploy and most costeffective solution for solving scaled-out business,engineering and scientific computational problems

Windows Server 2003, Compute Cluster Edition

Compute Cluster Pack

Microsoft Windows Compute Cluster Server 2003

+

=

Support for high performance hardware (x64bit architecture)

Winsock Direct support forRDMA for high performance interconnects (Gigabit Ethernet, InfiniBand, Myrinet,and others)

Support for Industry Standards MPI2

Integrated Job Scheduler

Cluster Resource Management Tools

Integrated Solution out-of-the-box

Leverages investmentin Windows administrationand tools

Makes cluster operation easyand secure asa single system

how ccs works

Job Mgmt

Cluster Mgmt

Head Node

Scheduling

Resource Mgmt

Data

Management

Input

Tasks

Cmd line

Domain\UserA

Job Execution

User App

MPI

How CCS Works

Active Directory

Policy, reports

Jobs

User

Desktop App

Admin Console

Job Mgr UI

Admin

Cmd line

High speed, low latencyinterconnect

DB/FS

Node Manager

integrated management
Integrated Management

Corporate IT Infrastructure

  • SystemsManagement
    • SMS
    • 3rd party

Windows Update

Monitoring

MOM

3rd party

AD

DNS

DHCP

PublicNetwork

Head Node

Compute Node

Compute Node

Admin / User Cons

Node Manager

Node Manager

RIS

MPI

MPI

Job Scheduler

Management

MPI

Management

Management

PrivateNetwork

NAT

MPINetwork

Compute Cluster

integrated job scheduling and management
Integrated Job Scheduling And Management
  • Integrated Queue monitoring
  • Integrated Node add/remove operations
  • Integrated administrative command and job scheduling
integrated development debugging

Head Node

Head Node

SDK

SDK

Compute Node

Compute Node

Compute Node

Compute Node

Compute Node

Compute Node

Integrated Development / Debugging

Developer Computeron a Cluster Node

Developer Computeron a Corporate Network

Developer Computer

Developer Computer

Remote Debug

Remote Debug

Remote Debug

Remote Debug

Remote Debug

integrated parallel programming
Integrated Parallel Programming
  • Message Passing Interface (MPI)
    • Works in shared memory (single chassis) and cluster (multiple chassis) situations
    • Requires moderate-advancedlevel of programming
  • OpenMP
    • Works in shared memory situationsonly – inside one physical chassis
    • Easy to program – simply identify parallelizable loops with pragma’s
technical drill down
Technical Drill Down
  • Simplified setup and deploymentfor OEM pre-installed clusters
  • Simplified and reliable networking infrastructure support for parallel applications
simple deployment and management
Simple DeploymentAnd Management
  • Prescriptive, simplified cluster setupand administration
  • Automated deployment and configuration
  • Integration with existing Windowsand management infrastructure
      • Reduces setup time
      • Requires less specialized skill set
end to end deployment
End-To-End Deployment

Copy

lay down the bits

Connect

Join Corporate AD

Configure

Build NetworkTopology

AD

DNS

DHCP

AD

DNS

DHCP

OEM Factory

Existing Windows

Install Procedure

Customer Site

Deployment/Addition of Compute Nodes

Opportunity for Professional Service

slide16
What OEM does

Reduce integration complexity byoffering preconfigured systems

Remove risks with patching by shipping with pre-installedhot-fixes

Ensure hardware compatibility (BIOS settings)

Reduces network configuration complexity by installing IB driversat the factory

Why is it easy

Same OS 64bit Windows → no new cert required

InfiniBand are logo’d

Use current installation methods

DedicatedTAM resources

One source for tested and proven drivers for all hardware

Copy

Copy

lay down the bits

connect
What customer does

Leverage ADfor single sign-onand accessto resources

Enable secureand transparentfile-sharing

Leverage existing auditing and group policy alreadyin place

Integrate with existing Windows management infrastructure

Why is it easy

80% enterprisehave deployed AD

Everything needed to implementthe solutionis includedin CCS

Leveragethe breadthof skill set – no needto train specialists

Connect

Connect

Join Corporate AD

AD

DNS

DHCP

configure
What customer does

Save customer time on network hardware configuration

Provide highly customizable network configuration options

Simplified future expansion of the cluster (additional servers or hardware upgrade can be performed without re-building the nodes)

Why is it easy

Use network topology configuration wizard

Automated building of application traffic to a particular network interconnect

Configure

Configure

Build Network Topology

networking challenges
Networking Challenges
  • Proliferation of MPI Libraries
  • Networking driver quality issues
proliferation of mpi libraries

MPICH

Vendor N

LAM

Vendor 1

GigE

GigE

GigE

GigE

Myrinet

Myrinet

Myrinet

Myrinet

IB

IB

IB

IB

Proliferation Of MPI Libraries

Application

MPI

Implementations

Interconnects

OS

Distributions

Kernel

Versions

customers as integrators
IT Manager’s application to node placement challenges

How many OS X or OS Y nodes?

What versions of the Kernel for node M?

If application X has an MPI error, which vendorshould I call for support?

If application X drops Kernel version Y support,which nodes do I need to upgrade?

If I remove a node for maintenance, what applicationswon’t be able anymore?

MPICH

Vendor N

LAM

MPICH

Vendor N

LAM

GigE

GigE

GigE

GigE

Vendor 1

Myrinet

Myrinet

Myrinet

GigE

IB

IB

GigE

GigE

GigE

GigE

GigE

Myrinet

IB

IB

RH

RH

Myrinet

Myrinet

Myrinet

Myrinet

SuSE

SuSE

IB

IB

IB

IB

SuSE

RH

RH

SuSE

RH

RH

RH

RH

RH

RH

RH

RH

RH

RH

RH

RH

RH

RH

RH

RH

SuSE

RH

RH

RH

RH

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

SuSE

Customers As Integrators

Application 2

Application 1

Application 3

ccs brings back the sanity
CCS Brings Back The Sanity
  • Use Winsock Directfor best performance(latency and throughput) and CPU efficiency
    • Increased flexibility
    • Increased utilization
  • Target industry standard interconnect fabrics
    • Gigabit Ethernet to minimize costs for a fast network
    • InfiniBand for latency-sensitive and bandwidth-intensive applications

Application

MSMPI

WSD

Any

Interconnect

W2k3

ms mpi leverages winsock direct
MS MPI Leverages Winsock Direct

HPC Application

User Mode

MPI

Switch trafficbased on sub-net

WinSock DLL

Winsock Switch

IBw/ RDMA

GigE

w/ RDMA

IB WinSock Provider DLL

Ethernet

GigE RDMA WinSock Provider DLL

User API (verbs based)

Manage hardware resources in user space (e.g., Send and receive queues)

User Host Channel Adapter Driver

TCP

IP

Kernel Mode

NDIS

Miniport (GigE)

Miniport (IPoIB)

Kernel API (verbs based)

OS component

Virtual Bus Driver

Host Channel Adapter Driver

IHV-provided component

Networking Hardware

windows logo program testing for infiniband
Background

Driven by Windows Networking Team

AvailableApril 21, 2006

Details

A test suite for WSD providers and IP over IB Miniport drivers

Include functionaltest only (no code coverage)

Signature covers networking only(no storage)

Windows Logo Program Testing For InfiniBand
call to action
Call To Action
  • Target departmental and workgroup cluster opportunities
  • Pre-install CCS including hotfixes,drivers and licensing scheme
  • Pre-test server and networking infrastructure
  • Build innovative systems forpersonal and workgroup clusters
  • Make sure your high speedinterconnect drivers are logo’d
resources
Resources
  • Microsoft HPC Web site

http://www.microsoft.com/hpc/

  • Windows Server x64 information

http://www.microsoft.com/64bit/, http://www.microsoft.com/x64/

  • Get the Facts Web site

http://www.microsoft.com/getthefacts

  • Windows HPC Community

http://www.windowshpc.net

  • Windows Vista Logo Program

http://www.microsoft.com/whdc/winlogo/hwrequirements.mspx

slide28

© 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.