Welcome to CW 2007!!!
Download
1 / 29

Welcome to CW 2007!!! - PowerPoint PPT Presentation


  • 73 Views
  • Uploaded on

Welcome to CW 2007!!!. The Condor Project (Established ‘85). Distributed Computing research performed by a team of ~40 faculty, full time staff and students who face software/middleware engineering challenges in a UNIX/Linux/Windows/OS X environment,

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Welcome to CW 2007!!!' - rosemarie


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

The condor project established 85 l.jpg
The Condor Project(Established ‘85)

Distributed Computing research performed by a team of ~40 faculty, full time staff and students who

  • face software/middleware engineering challenges in a UNIX/Linux/Windows/OS X environment,

  • involved in national and international collaborations,

  • interact with users in academia and industry,

  • maintain and support a distributed production environment (more than 4000 CPUs at UW),

  • and educate and train students.


Slide3 l.jpg

“ … Since the early days of mankind the primary motivation for the establishment of communitieshas been the idea that by being part of an organized group the capabilities of an individual are improved. The great progress in the area of inter-computer communication led to the development of means by which stand-alone processing sub-systems can be integrated into multi-computer ‘communities’. … “

Miron Livny, “Study of Load Balancing Algorithms for Decentralized Distributed Processing Systems.”,

Ph.D thesis, July 1983.


A good year for the principals and concepts we pioneered and the technologies that implement them l.jpg
A “good year” for the motivation for the establishment of principals and conceptswe pioneered and thetechnologies that implement them


Slide5 l.jpg

In August 2006 the motivation for the establishment of UW Academic PlanningCommittee approved theCenter for High Throughput Computing (CHTC). The L&S College created to staff positions for the center


Main threads of activities l.jpg
Main Threads of Activities motivation for the establishment of

  • Distributed Computing Research – develop and evaluate new concepts, frameworks and technologies

  • Keep Condor “flight worthy” and support our users

  • The Open Science Grid (OSG) – build and operate a national High Throughput Computing infrastructure

  • The Grid Laboratory Of Wisconsin (GLOW) – build, maintain and operate a distributed computing and storage infrastructure on the UW campus The NSF Middleware Initiative

  • Develop, build and operate a national Build and Test facility powered by Metronome


Slide7 l.jpg
Later today motivation for the establishment of Incorporating VM technologies(Condor VMs are now called slots)and improving supportfor parallel applications


Slide8 l.jpg

Downloads per month motivation for the establishment of


Slide9 l.jpg

Downloads per month motivation for the establishment of


Software development for cyberinfrastructure nsf 07 503 posted october 11 2006 l.jpg
Software Development for Cyberinfrastructure motivation for the establishment of (NSF 07-503) Posted October 11, 2006

All awards are required to use NMI Build and Test services, or an NSF designated alternative, to support their software development and testing.  Details of the NMI Build and Test facility can be found at http://nmi.cs.wisc.edu/.


Later today working with redhat on integrating condor into linux l.jpg
Later today motivation for the establishment of Working with RedHaton integrating Condor into Linux

Miron Livny and Michael Litzkow, "Making Workstations a Friendly Environment for Batch Jobs", Third IEEE Workshop on Workstation Operating Systems, April 1992, Key Biscayne, Florida.

http://www.cs.wisc.edu/condor/publications/doc/friendly-wos3.pdf


Slide12 l.jpg

06/27/97 motivation for the establishment of This month, NCSA's (National Center for Supercomputing Applications) Advanced Computing Group (ACG) will begin testing Condor, a software system developed at the University of Wisconsin that promises to expand computing capabilities through efficient capture of cycles on idle machines. The software, operating within an HTC (High Throughput Computing) rather than a traditional HPC (High Performance Computing) paradigm, organizes machines into clusters, called pools, or collections of clusters called flocks, that can exchange resources. Condor then hunts for idle workstations to run jobs. When the owner resumes computing, Condor migrates the job to another machine. To learn more about recent Condor developments, HPCwire interviewed Miron Livny, professor of Computer Science, University of Wisconsin at Madison and principal investigator for the Condor Project.


Why htc l.jpg
Why HTC? motivation for the establishment of

For many experimental scientists, scientific progress and quality of research are strongly linked to computing throughput. In other words, they are less concerned about instantaneous computing power. Instead, what matters to them is the amount of computing they can harness over a month or a year --- they measure computing power in units of scenarios per day, wind patterns per week, instructions sets per month, or crystal configurations per year.


High throughput computing is a 24 7 365 activity l.jpg
High Throughput Computing motivation for the establishment of is a24-7-365activity

FLOPY (60*60*24*7*52)*FLOPS


Slide15 l.jpg

The Grid: Blueprint for a New Computing Infrastructure motivation for the establishment of

Edited by Ian Foster and Carl Kesselman

July 1998, 701 pages.

The grid promises to fundamentally change the way we think about and use computing. This infrastructure will connect multiple regional and national computational grids, creating a universal source of pervasive and dependable computing power that supports dramatically new classes of applications. The Grid provides a clear vision of what computational gridsare, why we need them, who will use them, and how they will be programmed.


Slide16 l.jpg

“ … motivation for the establishment of We claim that these mechanisms, although originally developed in the context of a cluster of workstations, are also applicable to computational grids. In addition to the required flexibility of services in these grids, a very important concern is that the system be robust enough to run in “production mode” continuously even in the face of component failures. … “

Miron Livny & Rajesh Raman, "High Throughput Resource Management", in “The Grid: Blueprint for

a New Computing Infrastructure”.


Later today working with ibm on supporting htc on the blue gene l.jpg
Later today motivation for the establishment of Working with IBMon supporting HTC on the Blue Gene


Taking htc to the national level l.jpg
Taking HTC motivation for the establishment of to theNational Level


The open science grid osg taking htc to the national level l.jpg

The Open Science Grid (OSG) motivation for the establishment of Taking HTC to theNational Level

Miron Livny

OSG PI and Facility Coordinator

University of Wisconsin-Madison


The osg vision l.jpg
The OSG vision motivation for the establishment of

Transform processing and data intensive science through a cross-domain self-managed national distributed cyber-infrastructure that brings together campus and community infrastructure and facilitating the needs of Virtual Organizations at all scales


Osg principles l.jpg
OSG Principles motivation for the establishment of

  • Characteristics -

    • Provide guaranteed and opportunistic access to shared resources.

    • Operate a heterogeneous environment both in services available at any site and for any VO, and multiple implementations behind common interfaces.

    • Interface to Campus and Regional Grids.

    • Federate with other national/international Grids.

    • Support multiple software releases at any one time.

  • Drivers -

    • Delivery to the schedule, capacity and capability of LHC and LIGO:

      • Contributions to/from and collaboration with the US ATLAS, US CMS, LIGO software and computing programs.

    • Support for/collaboration with other physics/non-physics communities.

    • Partnerships with other Grids - especially EGEE and TeraGrid.

    • Evolution by deployment of externally developed new services and technologies:.


Tomorrow building campus grids with condor l.jpg
Tomorrow motivation for the establishment of Building Campus Gridswith Condor


Grid of grids from local to global l.jpg
Grid of Grids - from Local to Global motivation for the establishment of

National

Campus

Community


Who are you l.jpg
Who are you? motivation for the establishment of

  • A resource can be accessed by a user via the campus, community or national grid.

  • A user can access a resource with a campus, community or national grid identity.


Tomorrow just in time scheduling with condor glide ins scheduling overlays l.jpg
Tomorrow motivation for the establishment of Just in time schedulingwith Condor “glide-ins”(scheduling overlays)


Osg challenges l.jpg
OSG challenges motivation for the establishment of

  • Develop the organizational and management structure of a consortium that drives such a Cyber Infrastructure

  • Develop the organizational and management structure for the project that builds, operates and evolves such Cyber Infrastructure

  • Maintain and evolve a software stack capable of offering powerful and dependable capabilities that meet the science objectives of the NSF and DOE scientific communities

  • Operate and evolve a dependable and well managed distributed facility


The osg project l.jpg
The OSG Project motivation for the establishment of

  • Co-funded by DOE and NSF at an annual rate of ~$6M for 5 years starting FY-07.

  • 15 institutions involved – 4 DOE Labs and 11 universities

  • Currently main stakeholders are from physics - US LHC experiments, LIGO, STAR  experiment, the Tevatron Run II and Astrophysics experiments

  • A mix of DOE-Lab and campus resources

  • Active “engagement” effort to add new domains and resource providers to the OSG consortium


Security workflows fire walls scalability scheduling l.jpg
Security motivation for the establishment of WorkflowsFire-wallsScalabilityScheduling…


Slide29 l.jpg

Thank you for building such motivation for the establishment of

a wonderful community