slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Virginia Tech NDSSL-HPC Proposed System Diagrams 06-11-2013 PowerPoint Presentation
Download Presentation
Virginia Tech NDSSL-HPC Proposed System Diagrams 06-11-2013

Loading in 2 Seconds...

play fullscreen
1 / 6

Virginia Tech NDSSL-HPC Proposed System Diagrams 06-11-2013 - PowerPoint PPT Presentation


  • 141 Views
  • Uploaded on

Virginia Tech NDSSL-HPC Proposed System Diagrams 06-11-2013. Diagram Page Architecture Diagram 2 Equipment Rack Placement 3 Components 4,5 Management Network 6. Racks 1-4. Rack 5. 10 GbE Network. 1GbE Management

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Virginia Tech NDSSL-HPC Proposed System Diagrams 06-11-2013' - justin-kaufman


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Virginia Tech

NDSSL-HPC

Proposed System Diagrams

06-11-2013

DiagramPage

Architecture Diagram 2

Equipment Rack Placement 3

Components 4,5

Management Network 6

slide2

Racks 1-4

Rack 5

10 GbE Network

1GbE Management

Net Switches (stacked)

Racks 1-4, each with:

18 x C2112 2U

Quad Node Chassis

80 nodes each with:

2 x E5-2670 CPUs

64GB Memory (4GB/core)

1 x 1TB HDD

FDR IB

2 x 48-port 10GbE Switches

FDR IB Fabric

432 leaf ports expandable

to 648 ports

Racks 6

1GbE Mgmt switch

4 x C1104-RP5 1U

Xeon PhiNodes

2 x E5-2670 CPUs

64 GB Memory (4GB/core)

1 x 1TB HDD

2 x C2108-RP2 Admin nodes

2 x Intel Xeon E5 2670 2.6 GHz

128GB Mem 2x1TB HDD RAID-1

1 x PCIE-CONNX3-1P (FDR IB)

2 x 10GbE ports (SR)

Console/KVM

2 x C2108 VM servers,

2 x Intel Xeon E5 2670 2.6 GHz

128GB Mem 3x1TB HDD RAID-1

1 x PCIE-CONNX3-1P (FDR IB)

2 x 10GbE ports (SR)

2 x C2108-RP2 Login nodes

2 x Intel Xeon E5 2670 2.6 GHz

128GB Mem 2x1TB HDD RAID-1

1 x PCIE-CONNX3-1P (FDR IB)

2 x 10GbE ports (SR)

1 xC2108 archive node

2 x Intel Xeon E5 2670 2.6 GHz

128GB Mem 2x1TB HDD RAID-1

1 x PCIE-CONNX3-1P (FDR IB)

2 x 10GbE ports (SR)

2 x C2108-RP2 Gateway nodes

2 x Intel Xeon E5 2670 2.6 GHz

128GB Mem 2x1TB HDD RAID-1

1 x PCIE-CONNX3-1P (FDR IB)

2 x 10GbE ports (SR)

4 x UV20 fat nodes

4 x E5-4650 2.6GHz 8 core

1.5TB memory, 1 x 1TB HDD, FDR

Racks 9

1 x 1GbE Mgmt switch

4 x UV20 Oracle Servers each

with 10 x 3TB HDD

NFS Server

1 MIS Server with 72 3TB HDD

1 MIS JBOD Expansion with

72 3TB HDDs

Racks 7,8

Cable Legend

Racks 7,8 each with:

1 x 1GbE Mgmt switch

20 x C2110-RP5 2U

Xeon Phi Nodes

2 x E5-2670 CPUs

64GB Memory (4GB/core)

1 x 500GB HDD

1 x 5110P Xeon Phi

accelerators

FDR IB

10GbE Network

1GbE Management Network

FDR Infiniband Fabric

NDSSL HPC

System Architecture

slide3

C2112-4RP4 2U Quad Node Chassis

  • 4 x compute nodes in 2U, each with:
  • - 2 x E5 CPU sockets
  • 16 x 1600/1333/1066/800 DIMM Slots
  • - 3 x 3.5” HDD per server
  • 1 x PCIe 3.0 x16 supportint low profile slot
  • 1 x FDR Infiniband port on board
  • - 2 x 1GbE NICs
  • - 2 x 1200w redundant power supplies

C1104G-RP5 Accelerator node

1U 2 x E5 CPU sockets

8 x 1600/1333/1066/800 DIMM Slots

4 x 2.5” HDD

3 x PCIe 3.0 x16 supporting 3 GPGPU or Xeon Phi cards

1 x PCIe 3.0 x8 slot

2 x 1GbE NICs

2 x 1800w Platinum level redundant power supplies

  • C2108
  • - 2U dual-socket E5-2600 series server
  • - HPC management, head node, I/O nodes
  • - Up to 8 x 3.5” hot-swap drives
  • - Up to five expansion PCIe slots
  • - 1+1 Redundant AC Power Supplies
  • UV20 2U Quad Socket Server
  • - 4 socket E5-4600 series server
  • - 48 DIMM slots (1.5TB mem with 32GB DIMMs)
  • - Up to 8 x 2.5” HDDs
  • 8 PCIe slots
  • Redundant power

NDSSL HPC

Components

slide4

SGI IS2112 2U 12-drive JBOD

  • 12 3.5-inch HDD
  • SAS Interfaces

SGI Modular InfiniteStorage Server

  • Storage Server Options
    • Integrated into 4U chassis
      • 1 or 2 motherboards (Sandy bridge Platform)
      • 2 socket LGA ES-2600 (E5-2600 Series)
    • Each chassis supports SATA/SAS/SSD drive types:
      • Up to 72 x 3.5” drives
      • Up to 144 x 2.5” drives
    • Dual GbE onboard. Optional 2 port 10GbE,

2 port GbE, or 4 port 8Gb FC PCIe cards

  • JBOD Option
    • Up to 81 SATA/SAS/SSD drives (162 w/2.5” SFF)
    • Up to 8 SAS connections

NDSSL HPC

Components

slide5

Rack 1

Rack 2

Rack 3

Rack 4

Rack 5

Rack 6

Rack 7

Rack 8

Rack 9

Racks 1-4 each with:

19 x C2112 Quad-node compute chassis (total: 304 nodes)

Rack 5: IB switches, 10GbE switches

Rack 6: 4 x C2112 Quad compute chassis (16 nodes)

2 x admin nodes,

2 x VM servers,

2 x login nodes,

1 x archive node

4 x UV20 fat nodes

Rack 7-8: each with:

40 C1104-RP5 1U nodes with Xeon Phi ( total: 80 Xeon Phi nodes)

Rack 9: 4 x UV20 Oracle servers with local storage

NAS Storage: 1 x MIS server with 72 x 3TB HDD + 1 x MIS JBOD expansion with 72 x 3TB HDDs

Total: 144 x 3TB HDDs = 14 x 8+2 LUNs + 4 hot spares = 319.2TB usable capacity

NDSSL HPC

Equipment Rack Placement

slide6

1G sw1

1G sw1

1G sw1

1G sw1

1G sw2

1G sw2

1G sw2

1G sw2

Bond0

Active-Backup

Admin Nodes

10G Nodes x 8

10G Nodes x 8

10G Sw1

10G Sw2

1G sw1

1G sw1

1G sw1

1G sw1

Rack 1

Rack 2

Rack 3

Rack 4

Rack 6

Rack 7

Rack 8

Rack 9

Rack 5 not shown

10G switches are in rack 5

NDSSL HPC

Management Network