condor overview n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Condor Overview PowerPoint Presentation
Download Presentation
Condor Overview

Loading in 2 Seconds...

play fullscreen
1 / 28

Condor Overview - PowerPoint PPT Presentation


  • 118 Views
  • Uploaded on

Condor Overview. Bill Hoagland. Condor. Workload management system for compute-intensive jobs Harnesses collection of dedicated or non-dedicated hardware under distributed ownership. Condor History. Developed by University of Wisconsin-Madison Computer Science Department

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Condor Overview' - chanel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
condor overview

Condor Overview

Bill Hoagland

condor
Condor
  • Workload management system for compute-intensive jobs
  • Harnesses collection of dedicated or non-dedicated hardware under distributed ownership
condor history
Condor History
  • Developed by University of Wisconsin-Madison Computer Science Department
  • First put into production use 15 years ago
    • Mature and stable
condor availability
Condor Availability
  • Freely available under a BSD style license
  • Not open source, code is not distributed publicly
supported systems
Supported Systems
  • Solaris 8, 9, & 10 (Sparc)
  • Red Hat & Fedora Core (x86)
  • MS Windows 2000, XP & 2003 Server (x86)
  • Mac OS 10.3 & 10.4 (PPC)
  • Other Unixes (SuSE, AIX, HPUX,Yellow Dog, Debian)
condor design
Condor Design
  • Originally developed for “cycle stealing” from idle machines
  • Retains robustness to failures and changing availability from this legacy
condor goal
Condor Goal

• “High throughput” vs “High performance”

  • High performance - fast machines (ie. Cray)
  • High throughput - many machines, fault tolerant infrastructure (ie. SETI@Home)
condor components
Condor Components
  • Job queueing
  • Scheduling policy
  • Priority mechanism
  • Resource monitoring
  • Resource management
condor highlights
Condor Highlights
  • Checkpointing
    • Checkpointing saves complete running process and I/O state to disk
checkpointing
Checkpointing
  • Allows recovery from failures
    • Roll back to the last saved state
  • Allows process migration
    • Move saved state and restart
checkpointing continued
Checkpointing continued
  • Can compress checkpoint images
  • Checkpoint mechanism can be used outside of Condor
checkpointing continued1
Checkpointing continued
  • Some limitations
    • Single process space
    • Single kernel thread
    • Cannot save state of file open for both read and write
  • Not supported on all platforms
checkpointing continued2
Checkpointing continued
  • Must have object files
  • Usually requires no changes
  • Relink code to include condor library layer, e.g.

$ condor_compile gcc -o foo foo.c

condor highlights1
Condor Highlights
  • Remote system calls
    • Preserves user environment on remote machine
    • Users need not make files available or have access to remote machine
condor highlights2
Condor Highlights
  • Pools of Machines can be Hooked Together
    • Jobs submitted to one pool can migrate to a second
    • Subject to the policies of each pools owner
condor highlights3
Condor Highlights
  • Jobs can be Ordered
    • Jobs can be ordered because of dependencies easily
    • Dependencies are described in a directed acyclic graph
condor highlights4
Condor Highlights
  • Condor Enables Grid Computing
    • Condor has been designed with grid support hooks
    • Globus controlled resources
condor highlights5
Condor Highlights
  • Sensitive to the Desires of Machine Owners
    • Machine owners may set almost any usage policy
condor highlights6
Condor Highlights
  • Powerful priority policy mechanism
    • Requirements and preferences are associated with jobs and machines
    • A negotiation process matches job requirements then ranks on preferences
condor security
Condor Security
  • Condors purpose is to allow users to run arbitrary code on large numbers of machines
  • Assumes users are trustworthy
condor security continued
Condor Security continued
  • Cannot protect against users that can elevate their privileges
  • Does not run user jobs in sandboxes
condor security continued1
Condor Security continued
  • Can prevent unauthorized access to Condor
  • Optional authentication e.g. Kerberos, Grid Security Infrastructure (GSI), others
condor security continued2
Condor Security continued
  • Can ensure that user data has not been examined or tampered with
  • Optional encryption and integrity checking of all network traffic
condor backfill
Condor Backfill
  • When machine completely idle…
    • Configure default job
    • Support for BOINC
condor configuration
Condor Configuration
  • Controlled by hierarchical config files
    • Well commented
    • Human readable
    • In some cases, more clear than the manual
condor adminstration
Condor Adminstration
  • CondorView
    • Web based statistics
    • Machine and user data
condor website
Condor Website
  • http://www.cs.wisc.edu/condor