1 / 15

Packaging & Testing: NMI & VDT

Packaging & Testing: NMI & VDT. In Brief: What is NMI?. NSF Middleware Initiative Funding for middleware infrastructure Build, package, and test grid software. In Brief: What is VDT?. Virtual Data Toolkit—GriPhyN project Grid middleware + tools for virtual data

cachet
Download Presentation

Packaging & Testing: NMI & VDT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Packaging & Testing:NMI & VDT

  2. In Brief: What is NMI? • NSF Middleware Initiative • Funding for middleware infrastructure • Build, package, and test grid software

  3. In Brief: What is VDT? • Virtual Data Toolkit—GriPhyN project • Grid middleware + tools for virtual data • Supports physicists in GriPhyN, usable by anyone • Currently in active use in CMS, Atlas, EDG, and more • Very easy to install

  4. Why do you care? • Condor participates in grid activities • Condor team members are part of NMI & VDT • More testing means better Condor and Condor-G • NMI & VDT make it easy for you to start with grid software

  5. NMI at Condor • Three full-time Condor team members work on NMI • Build & package grid software • Only quality software is included. • Ensure that versions that work together (not always easy to do!) • Test infrastructure

  6. NMI Testing: Verification • NMI does basic verification testing • Can we submit a Globus single job? • Can we submit a Condor-G job? • Can we transfer a single file?

  7. NMI Testing: Local Grids • NMI has seven computers that can be organized in a local-area grid • Will expand with more computers, more architectures

  8. NMI Testing: Local Grids • Stress testing with DAGMan • Coordinated tests of thousands of jobs • Jobs distributed across grid sites • Tests Condor-G, Condor, Globus • Real-life testing with CMS application • Tests everything: Globus, Condor-G, Condor, file transfers… • Real-life workload, high stress

  9. NMI Testing: Larger Grids • We can attach more resources to local grids for stress testing • UW Condor pool with hundreds of nodes • Remote resources • Distributed testing across NMI partner sites

  10. VDT at Condor • Two full-time Condor team members • Packaging and installation • NMI packages software • VDT installs it beautifully • Packaging and installation are not glamorous, but are essential • NMI & VDT work together and make each other better

  11. VDT installation • VDT installation goal: • You hit a button, the software is correctly installed and configured. • (You can customize it afterwards.) • What does this depend on? • Excellent packaging • Excellent installation • Excellent testing

  12. VDT example installation • Download Pacman • http://physics.bu.edu/~youssef/pacman • pacman –get VDT-Server • Answer a few questions • pacman –get VDT-Client • ls condor/ ftsh/ Pacman.db replica/ vdt/ doc/ globus/ perl/ setup.csh vdt-install.log edg/ gpt/ post-install/ setup.sh

  13. VDT Testing • Wouldn’t it be nice if VDT could share with NMI? • VDT and NMI people are both on the Condor staff • Collaboration to share: • Test harness • Tests • Test infrastructure

  14. Where can you learn more? • NMI: www.nsf-middleware.org Bill Taylor: bt@cs.wisc.edu • VDT: www.griphyn.org/vdt Alain Roy: roy@cs.wisc.edu

  15. Take-home message • NMI & VDT are pushing us towards: • Heavily tested grid middleware • Supported software infrastructure Better support, better software

More Related