Parallel processing for AstroWise - PowerPoint PPT Presentation

amadahy
parallel processing for astrowise l.
Skip this Video
Loading SlideShow in 5 Seconds..
Parallel processing for AstroWise PowerPoint Presentation
Download Presentation
Parallel processing for AstroWise

play fullscreen
1 / 19
Download Presentation
Parallel processing for AstroWise
158 Views
Download Presentation

Parallel processing for AstroWise

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Parallel processing for AstroWise Some considerations for implementation of parallel processing AstroWise pre kick-off Meeting

  2. Current Developments • Beowulf for the masses • Oscar • User setup Linux cluster infrastructure • Add-on software PBS,PVM,MPI,C3 • Includes Itanium support • www.OpenClustergroup.org AstroWise pre kick-off Meeting

  3. Current Developments • In-a-Box initiative • NCSA Alliance layered software • Cluster-in-a-Box (CiB) • Grid-in-a-Box (Gib) • Display Wall-in-a-Box (DBox) • Access Grid-in-a-Box (AGiB) AstroWise pre kick-off Meeting

  4. Cluster-in-a-Box • Builds on OSCAR • Simplify installing and running Linux clusters • Compatible with Alliance’s production clusters • Software foundation for • Grid toolkits • Display Walls AstroWise pre kick-off Meeting

  5. Display Wall-in-a-Box • Tiled display wall • WireGL, VNC, NCSA Pixel Blaster • Building instructions AstroWise pre kick-off Meeting

  6. Example Sites • NIKHEF D0 Farm • D0 Monte Carlo production AstroWise pre kick-off Meeting

  7. Nodes • Dell Precision Workstation 220 • CPU: Dual Pentium III processor 800 MHz / 256 kB cache each • Memory: 512 MB PC800 ECC RDRAM • Hard Disk: 40 GB (7200 rpm) ATA-66 • CD-ROM: 20-48x EIDE • Floppy drive: 3.5" 1,44 MB • Graphics card: Diamond Viper V770D, 32 MBI AGP • Network: integrated 3Com Fast Ethernet card (10/100 TPO) Wake up on Lan functionality AstroWise pre kick-off Meeting

  8. Farm Server • Dell Precision Workstation 620 • CPU: Dual Pentium III Xeon processor 1GHz / 256 kB cache each • Memory: 512 (2x256) MB PC800 ECC RDRAM • Hard Disk: 72.8 Gbyte (10,000 rpm) U160/M SCSI • CD ROM: 20-48x EIDI • Floppy drive: 3.5" 1,44 MB • Graphics card: ELSA Synergy Force Graphics Card • Monitor: Dell Mainstream 17" (15.9 VIS) FST • Network: Integrated 3Com10/100 MB ethernet controller with Wake up on Lan functionality AstroWise pre kick-off Meeting

  9. File Server • ELONEX EIDE Server • CPU: Dual Pentium III 700 MHz • Memory: 512 (4 x 128) MB SDRAM DIMM ECC • Floppy drive: 3.5" 1,44 MB • Hard Disk: 2 x 10 Gbyte IDE (system disk) • RAID Controller: 3WARE 3W5800L 8 ports • Data Disks: 16 IBM EIDE 75 GB DTLA - 307075 EIDE (total of 1.2 Tbyte) • Network: Gigabit Netgear GA620 • Graphics card: IntegratedCirrus Logic GD5480 2 MB • Monitor: 17" flat screen AstroWise pre kick-off Meeting

  10. Switch • 3Com Superstack II Switch 3300 • 24 x 10/100 Base TX (autosensing) • 1 x 1 Gbps matrix port for stacking purpose • optional high speed slot; 1000BaseSX, 100BaseFX or Matrix module (4 x 1 Gbps) AstroWise pre kick-off Meeting

  11. Lorentz Institute • 64 AMD 1300 Mhz Rack mount PC’s • Monte Carlo Simulation • See tour • Additional conditions • Power consumption • Heat production AstroWise pre kick-off Meeting

  12. Configuration Considerations • Motherboard • FSB speed (200, 266 MHz) • Memory Rambus DRAM • CPU support > 1.5 Ghz • Pentium 4 (SMP?) • Itanium AstroWise pre kick-off Meeting

  13. Parallel implementation • Single OS, multiple CPU’s • MOSIX • Fork and forget • Does load balancing, adaptive resource allocation • Cluster of Machines • MPI, PMV (Message passing) • PVFS (parallel virtual file system) • PBS (Portable Batch system) • MAUI (Job scheduling) AstroWise pre kick-off Meeting

  14. Network • Latency • Overhead in packing data • Grand copy and wait (4kx2k in 4 sec at 100Mb/s) • Bandwidth • Physical capacity of connections • 200KPix/s • Switching technology, server at > 1 Gb/s AstroWise pre kick-off Meeting

  15. Parallellization • Simple scripting • Rendezvous problem • Load balancing • Code level • MPI Programming • How deep? • Loops • Matrix splitting • Sparse array coding AstroWise pre kick-off Meeting

  16. Granularity • Fine • Small tasks • Frequent comm. • Many processes • Coarse • Large tasks • Much comp. work • Infrequent comm. AstroWise pre kick-off Meeting

  17. Costs • Programmer’s time • Analyze, recode • Complicated debugging • Loss of portability • Total CPU time greater with parallel • Initialize & terminate tasks • Comminications among tasks • Replication of code, more memory AstroWise pre kick-off Meeting

  18. Reading/Writing data • One process distributes data • Lots of I/O, no local disk? • Each process can read/write • Read/write their own file • Read/write own section of big file • Master/Slave based on • Dataflow AstroWise pre kick-off Meeting

  19. Issues • Parallellization • On what level, at what cost • Should be serializable as well • What kind of cluster • Cluster of Machines • Single OS (large SMP) • Grid connectivity AstroWise pre kick-off Meeting