1 / 11

Clustering Technology For Scaleability

Clustering Technology For Scaleability. Jim Gray Microsoft Research http://www.research.Microsoft.com/~Gray. The Answer: BOTH SMP and Cluster?. Grow Up with SMP 4xP6 is now standard Grow Out with Cluster Cluster has inexpensive parts. Cluster of PCs. Clusters being built.

maryleej
Download Presentation

Clustering Technology For Scaleability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Clustering TechnologyFor Scaleability Jim Gray Microsoft Research http://www.research.Microsoft.com/~Gray

  2. The Answer:BOTH SMP and Cluster? Grow Up with SMP 4xP6 is now standard Grow Out with Cluster Cluster has inexpensive parts Cluster of PCs

  3. Clusters being built • Teradata 500 nodes (50k$/slice) • Tandem,VMScluster 150 nodes (100k$/slice) • Intel, 9,000 nodes @ 55M$ ( 6k$/slice) • IBM: 512 nodes @ 100m$ (200k$/slice) • PC clusters (bare handed) at dozens of nodes web servers (msn, PointCast,…), DB servers • KEY TECHNOLOGY HERE IS THE APPS. • Apps distribute data • Apps distribute execution

  4. So, What’s New? • When slices cost 50k$, you buy 10 or 20. • When slices cost 5k$ you buy 100 or 200. • Manageability, programmability, usability become key issues (total cost of ownership). • PCs are MUCH easier to use and program

  5. New MPP & NewOS New MPP & NewOS New MPP & NewOS New MPP & NewOS New App New App New App New App So, What’s New? • PCs create virtuous cycle Vicious Cycle No Customers! Virtuous Cycle: Standards allow progress and investment protection Apps Standard OS & Hardware Customers

  6. What is Wolfpack? • A consortium of 60 HW & SW vendors(everybody who is anybody) • A set of APIs for clustering and fault tolerance • An enhancement to NT™ Server (in beta test ) • Key concepts • System: a particular node • Cluster: a collection of systems working together • resource: a hardware or software module • resource dependency: one resource needs another • resource group: fails over as a unit: dependencies do not cross group boundaries

  7. Non Aware App What is Wolfpack? Cluster Management Tools Cluster Api DLL RPC Cluster Service Global Update Database Manager Manager Node Event Processor Manager Mgr Failover Communication App Manager Resource Mgr Resource Other Nodes DLL Open Online IsAlive LooksAlive Offline Close Resource Resource Monitors Management Interface Physical Logical App Resource Resource Resource DLL DLL DLL Cluster Aware App

  8. Cluster Advantages • Clients and Servers made from the same stuff. • Inexpensive: Built with commodity components • Fault tolerance: • Spare modules mask failures • Modular growth • grow by adding small modules • Parallel data search • use multiple processors and disks

  9. Single System Image:Is It Important? • Yes, if you don’t have it you fail • parallel MPPs vs Tandem, Teradata, VAXcluster. • NUMA & Cluster: • some things are farther away. • Must program in parallel to • utilize multiple cpus, disks, wires • OS, DBMS, TPmonitor, Web Server, ORB give transparency: load balance data and programs. • Administrator, Programmer, User • do not want to know about program & data location

  10. What Happens When a Component Fails? • Redundant disk or path: configure around it. • Non-redundant software: restart. • Non-redundant hardware: migrate software to surviving nodes. • Fault detection: 1 ms to 10 sec. • Failover .1 sec to 1 min. • This is standard in Tandem, Teradata, VMScluster

  11. What are Support Costs? • Cluster lowers support costs by • masking failures (instant repair via spare modules) • allowing online maintenance and upgrades. • Commodity parts are much cheaper • 10$/MIPS vs 10,000$/MIPS • 1k$/OS vs 30K$/month/OS • Moden OSs are easier to install, configure, manage • GUI • Self-tuning • Online and task-based help • Built in wizards

More Related