1 / 20

Implementation of Package Management in a Cluster Environment

Implementation of Package Management in a Cluster Environment. So, Jung-ki Distributed Computing System LAB School of Computer Science and Engineering Seoul National University. Introduction (1/2). Supercomputer High performance processor / high network bandwidth

roz
Download Presentation

Implementation of Package Management in a Cluster Environment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Implementation of Package Management in a Cluster Environment So, Jung-ki Distributed Computing System LAB School of Computer Science and Engineering Seoul National University

  2. Introduction (1/2) • Supercomputer • High performance processor / high network bandwidth • Expensive system but Beowulf system is cost-effective • Motivation • Focus on Cluster system • Cluster Management system • Manual method / add-on method / integrated method • Registry • Central repository of information about all aspects of the computer

  3. Introduction (2/2) • Challenge • Integrated method has low availability and reliability • Can’t manage computation nodes separately • When failure occurs, system can’t be rejuvenated • Goal ( using Registry ) • Improve availability and reliability of integrated method • Administrator can manage a cluster system easily • Restore cluster system with a backup snapshot

  4. 60.8% Supercomputer Domestic Supercomputer Quantity : 14 Cluster : 4 MPP : 4 Constellation : 6 ※ SNU : 2 (51/413)

  5. Cluster Management System • Manual approach • System administrator brings up entire system manually • Add-on method • Bring up a frontend node, then add cluster packages • OSCAR / Warewulf / OpenMosix • Integrated method • Cluster packages are installed and configured during the initial installation • Rocks / Scyld

  6. Application HPC SGE OS (Linux) Cluster Management System • Software Stack Parallel code / Grid / computer lab … Message passing / communication Layer Job Scheduling and Launching Cluster software management Cluster State management / Monitoring HPC Device Drivers Linux Environment Linux Kernel

  7. Rocks Overview • Identity • System to build and manage a Linux Cluster • Free : Open source project • Goal • Make clusters easy • Philosophy • Computation nodes are 100% automatically installed • Roll : set of packages • Graph / Kickstart • Run on heterogeneous system architecture • Doesn’t attempt to incrementally update software

  8. Rocks system • Architecture internet Front-end node eth1 eth0 Local Network eth0 eth0 eth0 eth0 node node node node

  9. What is Registry ? • Central repository of info about all aspects of the computer • Hardware, OS, applications, users information • Function • Retrieve system information • Update / add / delete software • Backup & restore system • Advantage • Easier for applications to access system • Storing large amounts of structured data (system info)

  10. Registry Design Aliases Original Relational Schema ID (primary key) Node Name H/W information Appended Relation Network Nodes ID (primary key) Node MAC IP Gateway Name Device Module Package ID (primary key) Name Membership CPUs Rack Rank Comment S/W information ID (primary key) Node Name Version Release Install Appliances Memberships Distribution ID (primary key) Name Graph Node ID (primary key) Name Appliance Distribution ID (primary key) Name Release Lang

  11. Strategy of management • Rocks Setup • Minimum modification • Take advantage of original Rocks system • Deploy cluster system easily • Modify related source codes • insert-ethers, kickstart.cgi, Kpp, Kgen, Rgen • Running System • Apply package modification • Package management program : add / update / delete packages • DB consistency management program

  12. Rgen Appended component Package variables Registry variables Collection Method

  13. Instruction : Add / update / delete add –c=compute-0-0 –i=amanda-2.4.5-2.i386 add –c=all –i=all del -c=compute-0-0 –i=amanda-2.4.5-2.i386 del -c=all -i=all Packages table Package name / version / release Compute Nodes Packages table Add / delete / update Modification Method Insert command

  14. Registry consistency • Setup time • When frontend node removes / updates computation node • Dependency : change node table → change package table • Modify Kickstart.cgi / kgen • Apply cascading tables change ※mysql not support transaction property • Running system • Package install / delete / update • Compute node rpm information = frontend node’s registry DB

  15. Experiment Data name capacity volume amanda 468KB 3 HPC 117MB 53 Rocks roll 1.5GB 479 Experiment Setup Public Ethernet Rocks.snu.ac.kr CPU 800Mhz RAM 768MB HDD 40G Frontend node Compute-0-(1~14) CPU 850Mhz RAM 1G HDD 10G Compute nodes (14)

  16. Network card DHCP request Original Rocks Evaluation average service time : 18min 14sec average transmit time : 11min 28sec

  17. Amanda Packages Evaluation average install time : 6.62 sec Average delete time : 5.57sec

  18. HPC Roll Evaluation average install time : 3min 38sec average delete time : 1min 18sec

  19. Conclusion • Registry takes advantage of cluster system • Improve availability and reliability using Registry • Administrator can manage cluster systems easily • Restore cluster systems with backup snapshots

  20. Q & A Questions or Comments ? Thank you !

More Related