Deploying condor using virtualbox on windows hosts
Download
1 / 19

Deploying Condor Using VirtualBox on Windows Hosts - PowerPoint PPT Presentation


  • 92 Views
  • Uploaded on

Deploying Condor Using VirtualBox ® on Windows® Hosts. David Herzfeld 1 , Craig A. Struble, Ph.D 2 , and Lars E. Olson, Ph.D 1 Marquette University 1 Department of Biomedical Engineering 2 Department of Mathematics, Statistics, and Computer Science

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Deploying Condor Using VirtualBox on Windows Hosts' - lev


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Deploying condor using virtualbox on windows hosts

Deploying Condor Using VirtualBox® on Windows® Hosts

David Herzfeld1, Craig A. Struble, Ph.D2, and Lars E. Olson, Ph.D1

Marquette University

1Department of Biomedical Engineering

2Department of Mathematics, Statistics, and Computer Science

Southeast Wisconsin High Performance (SeWHiP) Computing Consortium (www.sewhip.org)


Introduction
Introduction

  • Most systems on Marquette’s campus run Windows XP® or Windows Vista®

  • Most researchers at Marquette require Linux based environments

  • Most sys-admins have primarily Windows® based experience


Project goals
Project Goals

  • Allow researchers to use underutilized Marquette systems

    • Requires Linux environment

  • Allow researchers full use of the Condor system

    • check pointing, migration, etc.

  • Do not change Windows® users’ experience

  • Easy installation and maintenance for Windows® sys-admins


Possible solutions
Possible Solutions

  • Use Condor for Windows®

    • Very few researchers using Windows® based applications

    • Most require a GUI

  • Windows® – Linux dual boot systems

    • Management is difficult

    • Windows® users will be affected

  • Condor Virtual Universe

    • VMware® = $

    • Xen does not support Windows® hosts

    • Configuration Overhead

  • Virtualization


Initial solution colinux
Initial Solution: CoLinux

  • Based on previous work by Neeman, Serverini (U. of Oklahoma), and Sumanth (Nebraska-Lincoln), 2006

  • Runs natively in Windows®

    • Near native performance

  • Small hardware dependencies

  • Open source

  • Relatively unsupported – small community

  • Only supports 32 bit environments

  • No current SMP support

  • Few recent developments


Current solution sun virtualbox
Current Solution: Sun VirtualBox®

  • Free & “Open Source”

  • Capable or running under 32 & 64 bit Windows®

  • Guest Additions allow for near native speeds

    • Some disk intensive operations may see a speed up

  • Supported & continually developed

  • Large community support

  • No current SMP support

    • planned for future release


Virtualbox windows
VirtualBox® & Windows®


Condormac comon
CondorMAC & CoMon

  • CondorMAC:

    • Allows for distribution via images

    • Replaces the VirtualBox® MAC address with a modification of the host MAC address

  • CoMon:

    • Determine sec. since last key stroke

    • Determine total processor utilization

    • Determine VirtualBox® processor utilization

    • Write data to file in host-guest shared folder

    • Visual Basic monitor service


Hclust service
hclust_service

  • Fork headless VirtualBox® machine at Windows® startup.

  • Remove any VirtualBox® “remnant” machines

  • “Nicely” shut down guest system when requested (via ACPI)



Virtual machine basics
Virtual Machine Basics

  • Guest hostname is based on assigned IP address

  • Central Manager name & IP are determined

    • Personalized text file in shared folder

    • Variables replaced in condor_config.local

  • For remote administration

    • ssh key in shared folder

  • Condor launches after guest startup

    • init.d script


Benefits of immutable hard drives
Benefits of Immutable Hard Drives

  • Defines an upper limit on the amount of space Condor users can utilize (100 GB).

  • Removal of the snapshots – by hclust_service – returns the VM to an original state.

  • Returns unused hard drive space back to the Windows® host.

    • This space would not be returned if it were simply a dynamically expanding .vdi image



Vm additions
VM Additions

  • Additions included in immutable hard drive

  • Software packages utilized by researchers

    • Geant, OpenFOAM, AutoDock, etc.

  • Sets up execution path for use in Condor jobs

  • Package names and versions added to Condor ClassAds


Installation
Installation

  • Distributed as a zip file, batch installer

  • Installs VirtualBox®

  • Creates a new virtual machine

    • MAC Address set by CondorMAC

    • Memory computed as fraction of total system memory

    • Bridged networking set up

  • Adds CentOS image to virtual machine

  • Sets services to run at system startup

  • Starts the new VM

  • Installation time is minutes


Virtualization issues
Virtualization Issues

  • Networking Problems

    • Current bridged connection requires an additional IP address for each worker

    • Doubles the on campus IP addresses!


Future work
Future Work

  • SMP Support

  • Support for different networking modes

    • using NAT, OpenVPN, etc.

  • Specific VM updates via repositories

  • Tailored distributions

    • Bioinformatics

    • Fluid Dynamics

    • Neural Simulations


Acknowleldgments
Acknowleldgments

  • Information Technology Services; Marquette University

    • Kathy Lang; CIO, Clifford Brown, Patrick Blume, Aaron Gember, Chad Gorectke

  • Lingtao Zeng; Marquette MSCS System Administrator

  • Brad Bonczkiewicz; Marquette Engineering System Administrator

  • Biomedical Engineering Department

  • Mathematics, Statistics and Computer Science Department



ad