1 / 14

General Purpose Computing with Condor

Condor Week 2004. General Purpose Computing with Condor. May 2004. Brooklin J. Gore Senior Fellow, Advanced Computing. 2003 Micron Technology, Inc. All rights reserved. Information is subject to change without notice. General Purpose Computing with Condor Agenda.

enye
Download Presentation

General Purpose Computing with Condor

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Condor Week 2004 General Purpose Computing with Condor May 2004 Brooklin J. Gore Senior Fellow, Advanced Computing 2003 Micron Technology, Inc. All rights reserved. Information is subject to change without notice.

  2. General Purpose Computing with CondorAgenda The Condor environment at Micron Micron’s Grid Computing Best Practices Parade of Applications (Benefits) Condor Enhancements Future Requirements for Condor Micron Confidential

  3. General Purpose Computing with CondorCondor Environment Micron Confidential

  4. General Purpose Computing with CondorCondor Environment Micron Confidential

  5. General Purpose Computing with CondorCondor Environment Micron Confidential

  6. General Purpose Computing with CondorBest Practices for Grid Computing • Infrastructure • Fast network • Centralized Data • Common system images • Lots of memory • Dual processors Micron Confidential

  7. General Purpose Computing with CondorBest Practices for Grid Computing • Philosophy • Articulate value proposition • “Low hanging fruit” first • Educate, educate, educate • Integrate with existing practices Micron Confidential

  8. General Purpose Computing with CondorParade of Applications (Benefits) • Automated Process Control (scalability, reliability) • DRAM row/column redundancy design (1.5x 24->16 hours) • Engineering extracts (Windows->*nix port in 50 v. 900 hours) • Engineering analysis (6x speedups, “I don’t know how” quote) • Enhanced SW Defect Analysis (our old friend) • Global planning and scheduling (scalability, reliability) Micron Confidential

  9. General Purpose Computing with CondorParade of Applications (Benefits) • MSI package creation (6x 12->2 hours) • Network device configuration (6x 24->4 hours) • RDA inspection processing (scalability, reliability) • Matched Routing (capability) • Integrated Data Analysis System (time window) • OOC chart generation (scalability, reliability Micron Confidential

  10. General Purpose Computing with CondorParade of Applications (Benefits) • SWR processing (scalability, reliability) • System reporting (15x 55->3.5 minutes) • Web log processing (8x 16->2 hours) • Trend charts (>300k charts in 5 hours) • Data conversion (1mil files, 8 hours) • OS patch management (12x 12->1 hour) Micron Confidential

  11. General Purpose Computing with CondorParade of Applications (Benefits) Quotes like this make it all worthwhile: I don’t know how we’d get all of our data extractions done without Condor -- Melissa Norgard Savings on order of $ millions Micron Confidential

  12. General Purpose Computing with CondorCondor Enhancements • Web-based • Host Management Interface • Pool, job, machine status • Pool performance metrics (hosts, idle jobs, queue delay) • Custom Unix/Windows machine ClassAds (via STARTD_CRON) • Admin tools (missing host reports, global capabilities) • Developer tools (msg-enabled job monitor, wait, cleanup) • Need: Stale job notification, cleanup Micron Confidential

  13. General Purpose Computing with CondorFuture Requirements • Queue performance • Need to start a job in < 3 minutes, ideally 1 minute • Handle deep/narrow queues, shallow/wide queues and most importantly a combination of the two • Scale • 10,000 hosts per pool • 100-200 submitters • Web Services protocols • SOAP, UDDI, WSDL, XML Micron Confidential

  14. Condor Week 2004 Micron and the Micron logo are trademarks and/or service marks of Micron Technology, Inc. All other trademarks are the property of their respective owners.

More Related