nsf middleware initiative update and overview of release 2 l.
Skip this Video
Loading SlideShow in 5 Seconds..
NSF Middleware Initiative: Update and Overview of Release 2 PowerPoint Presentation
Download Presentation
NSF Middleware Initiative: Update and Overview of Release 2

Loading in 2 Seconds...

play fullscreen
1 / 63

NSF Middleware Initiative: Update and Overview of Release 2 - PowerPoint PPT Presentation

  • Uploaded on

Fall 2002 Internet2 Member Meeting. NSF Middleware Initiative: Update and Overview of Release 2. Alan Blatecky , National Science Foundation John McGee , USC/ISI Ken Klingenstein, Internet2 & University of Colorado-Boulder Mary Fran Yafchak , Southeastern Universities Research Association

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

NSF Middleware Initiative: Update and Overview of Release 2

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
nsf middleware initiative update and overview of release 2

Fall 2002 Internet2 Member Meeting

NSF Middleware Initiative:Update and Overview of Release 2

Alan Blatecky, National Science Foundation

John McGee, USC/ISI

Ken Klingenstein, Internet2& University of Colorado-Boulder

Mary Fran Yafchak, Southeastern Universities Research Association

Ann West, EDUCAUSE & Internet2

session topics
Session Topics
  • NSF Middleware Initiative Overview
  • GRIDS Center
  • NMI Integration Testbed
  • NMI Outreach and Participation
nsf middleware initiative
NSF Middleware Initiative
  • Purpose
  • To design, develop, deploy and support a set of reusable, expandable set of middleware functions and services that benefit applications in a networked environment
nmi organization
NMI Organization
  • GRIDS Center
    • ISI, NCSA, U Chicago, UCSD & U Wisconsin
  • EDIT Team (Enterprise and Desktop Integration Technologies)
    • EDUCAUSE, Internet2 & SURA

Core NMI Team

Grants for R & D

  • Year 1 -- 9 grants
  • Year 2 -- 9 grants
a vision for middleware
A Vision for Middleware

To allow scientists and engineers the ability to transparently use and

share distributed resources, such as computers, data, and instruments

To develop effective collaboration and communications tools such as

Grid technologies, desktop video, and other advanced services to expedite research and education, and

To develop a working architecture and approach which can be extended

to Internet users around the world.

Middleware is the stuff that makes “transparently use” happen, providing

persistency, consistency, security, privacy and capability

nmi goals
NMI Goals
  • a) facilitate scientific productivity,
  • b) increase research collaboration through shared data, computing, code, facilities and applications,
  • c) support the education enterprise,
  • d) encourage the participation of industry, government labs and agencies for more extensive development and wider adoption and deployment,
  • e) establish a level of persistence and availability so that other applications developers and disciplines can take advantage of the middleware,
  • f) encourage and support the development of standards and open source approaches and,
  • g) enable scaling and sustainability to support the larger research and education communities.
nmi process

Experimental Software & research applications

Early Implementations

- GRID services, directories, authentication, etc

Research &


Early Adopters


- experimental, Beta, scaling & “hardening”


- disciplines

- communities

- industries

Dissemination & Support

Middleware deployment

NMI Process
first deliverables nmi release 1
First Deliverables: NMI Release 1
  • Software
    • (Globus, Condor, Network Weather Service, KX.509, CPM, Pubcookie)
  • Object Classes
    • (eduPerson, eduOrg, commObject)
  • White Papers (Shibboleth, video directories, etc)
  • Best Practices (Directories, LDAP)
  • Policies (campus certificates, account management)
  • Services (certificate profile registry)


nmi release 2
NMI Release 2
  • Release 2 shipped on Oct-25-2002
  • New versions
    • Globus Toolkit, Condor-G, Network Weather Service, Pubcookie, etc
  • New components and best practices
    • OpenSAML 1.0, Shibboleth 1.0, etc
    • GSI-OpenSSH, Gridconfig Tools
    • LDAP Analyzer, Metadirectory Practices for Enterprise Directories, etc
  • Two releases each year: April/October
  • Release being adopted by projects, agencies
  • International interest in releases
3rd year program of nmi
3rd Year Program of NMI
  • Program Announcement in process
  • March 3, 2003 Proposal deadline
  • $7M available for FY’03
  • http://www.nsf-middleware.org
GRIDS Center Overview
  • John McGee
  • USC, Information Sciences Institute
  • mcgee@isi.edu
grids center part of the nsf middleware initiative
GRIDS Center, Part of the NSF Middleware Initiative
  • One of two NMI teams, the GRIDS Center (Grid Research, Integration, Development & Support)
  • In late 2001, GRIDS created to:
    • Define, develop, deploy, and support an integrated national middleware infrastructure for 21st Century S&E
    • Create robust, tested, packaged, & documented middleware for S&E, including large NSF projects (e.g., NEES, GriPhyN, TeraGrid)
    • Work with middleware research community to evolve architecture & integrate other components
    • Provide dedicated operations capability for 24x7 support and monitoring of Grid infrastructure
grids center participants
GRIDS Center Participants
  • The Information Sciences Institute (ISI), University of Southern California (Carl Kesselman)
  • The University of Chicago (Ian Foster)
  • The National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign (Randy Butler)
  • The San Diego Supercomputer Center (SDSC) at the University of California at San Diego (Phil Papadoupolus)
  • The University of Wisconsin at Madison (Miron Livny) 
elements of grid computing
Elements of Grid Computing
  • Resource sharing as a fundamental pursuit
    • Computers, storage, sensors, networks
    • Sharing is always conditional, based on issues of security, trust, policy, negotiation, payment, etc.
  • Coordinated problem solving
    • Beyond client-server: distributed data analysis, computation, collaboration, etc.
  • Dynamic, multi-institutional “virtual organizations”
    • Community overlays on classic org structures
    • Large or small, static or dynamic
grid applications
Grid Applications
  • Science portals
    • Help scientists overcome steep learning curves of installing and using new software
  • Distributed computing
    • High-speed workstations and networks as aggregated computational resources
  • Large-scale data analysis
  • Computer-in-the-loop instrumentation
    • Grids permit quasi-real-time analysis of data from telescopes, synchrotrons, and electron microscopes
  • Collaborative work
    • Grids enable collaborative problem formulation, data analysis, and discussion
mathematicians solve nug30
Mathematicians Solve NUG30
  • Looking for the solution to the NUG30 quadratic assignment problem
  • An informal collaboration of mathematicians and computer scientists
  • Condor-G delivered 3.46E8 CPU seconds in 7 days (peak 1009 processors) in U.S. and Italy (8 sites)
  • 14,5,28,24,1,3,16,15,
  • 10,9,21,2,4,29,25,22,
  • 13,26,17,30,6,20,19,
  • 8,18,7,27,12,11,23

MetaNEOS: Argonne, Iowa, Northwestern, Wisconsin

home computers evaluate aids drugs
Home ComputersEvaluate AIDS Drugs
  • Community =
    • 1000s of home computer users
    • Philanthropic computing vendor (Entropia)
    • Research group (Scripps)
  • Common goal = advance AIDS research
sloan digital sky survey analysis

Galaxy cluster

size distribution

Chimera Virtual Data System

+ iVDGL Data Grid (many CPUs)

Sloan Digital Sky Survey Analysis

Size distribution of

galaxy clusters?

ivdgl international virtual data grid laboratory

Tier0/1 facility

Tier2 facility

Tier3 facility

10 Gbps link

2.5 Gbps link

622 Mbps link

Other link

iVDGL: International Virtual Data Grid Laboratory
network for earthquake engineering simulation
Network for EarthquakeEngineering Simulation
  • NEESgrid: US national infrastructure to couple earthquake engineers with experimental facilities, databases, computers, and each other
  • On-demand access to experiments, data streams, computing, archives, collaboration
  • NEESgrid is a partnership of Argonne, Michigan, NCSA, UIUC, USC
the 13 6 tf teragrid computing at 40 gb s
The 13.6 TF TeraGrid:Computing at 40 Gb/s

Site Resources

Site Resources






External Networks

External Networks





External Networks

External Networks


8 TF

240 TB


4.1 TF

225 TB

Site Resources

Site Resources



TeraGrid: NCSA, SDSC, Caltech, Argonne www.teragrid.org

grids and industry
Grids and Industry
  • Grid computing has much in common with major industrial thrusts to decentralize (e.g., B2B, P2P, ASP, etc.)
  • Sharing issues are not adequately addressed by existing technologies
  • Companies like IBM, Platform Computing and Microsoft are now substantively involved with the open-source Grid community (e.g., OGSA, which combines Web services and Grid services)
grids software for nmi
GRIDS Software for NMI
  • GRIDS Center Software Suite in the first release (NMI-R2) is a package of:
    • Globus Toolkit™
    • Condor-G
    • Network Weather Service
    • KX.509 & KCA
    • GSI-OpenSSH
    • Gridconfig Tools
    • Grid Packaging Tools
    • For RedHat 7.2/7.3 on IA32, Solaris 8 on 32-bit Sparc

Enterprise and Desktop Integration Technologies (EDIT) Consortium

  • Ken Klingenstein
  • Director, Internet2 Middleware Initiative
  • kjk@internet2.edu
nmi edit consortium
NMI-EDIT Consortium
  • Enterprise and Desktop Integration Technologies Consortium
    • Internet2 – primary on grant and research
    • EDUCAUSE – primary on outreach
    • Southeastern Universities Research Association (SURA) – primary on NMI Integration Testbed
  • Grant funding is ~$1.2 million a year:
    • about ½ to short-term partial hiring of campus IT staff to develop and document required standards, best practices, etc.
    • about ½ to testbeds, dissemination and training sessions
  • Almost all funding passed through to campuses for work
nmi edit goals
  • Much as at the network layer, create a ubiquitous common, persistent and robust core middleware infrastructure for the R&E community
  • In support of inter-institutional and inter-realm collaborations, provide tools and services (e.g. registries, bridge PKI components, root directories) as required
nmi edit objectives
NMI-EDIT: Objectives
  • Foster the development of campus enterprise middleware to leverage both the academic and administrative missions.
  • Coordinate a common substrate across higher ed middleware implementations that would permit inter-institutional efforts such as Grids, digital libraries, and collaboratories to scale and leverage
  • In some instances, build collaboration tools for particularly important inter-institutional and government interactions, such as web services, PKI and video.
  • Insure that distinctive higher-ed requirements, from privacy and academic freedom to multi-realm portals, are served in the marketplace.
nmi edit core middleware scope
NMI-EDIT: Core Middleware Scope
  • Identity and Identifiers – namespaces, identifier crosswalks, real world levels of assurance
  • Authentication – campus technologies and policies, inter-realm interoperability via PKI, Kerberos
  • Directories – enterprise directory services architectures and tools, standard object classes, inter-realm and registry services
  • Authorization – permissions and access controls, delegation, privacy management
  • Integration Activities – common management tools, use of virtual, federated and hierarchical organizations
nmi edit organization
NMI-EDIT: Organization
  • Overall technical direction set by MACE
    • Middleware Architecture Committee for Education (MACE)
    • Bob Morgan, University of Washington, Chair
    • Campus IT architects and representatives from Grids and International Communities
  • Directions set via
    • NSF and NMI management team
    • Internet2 Network Planning and Policy Advisory Council
    • PKI and Directory Technical Advisory Boards
    • Internet2 members
sample nmi edit process directories
Sample NMI-EDIT Process (Directories )
  • MACE-DIR Working Group prioritizes needed materials
  • Subgroups established:
    • revision of basic documents (LDAP Recipe)
    • new best practices in groups and metadirectories
    • standards development for eduPerson 1.5 and eduOrg 1.0
  • Subgroups work in enhanced IETF approach: scenarios, requirements, architectures, recommended standards stages
  • Working group deliverables announced; input and conference call review/feedback processes start; work groups reconvene as needed
  • Process takes around 4-6 months, depending on product
  • 6-8 people drive the process with 15-50 schools participating
nmi edit participants
NMI-EDIT: Participants
  • Higher Ed – 15-20 leadership institutions, with 50 more campuses represented as members of working groups; readership around 2000 institutions
  • Corporate - (IBM/Metamerge, Microsoft, SUN, Liberty Alliance, DST, MitreTek, Radvision, Polycom, EBSCO, Elsevier, OCLC, Baltimore)
  • Government – NSF, NIST, NIH, Federal CIO Council
  • International – Terena, JISC, REDIRIS, AARnet, SWITCH
a few year one nmi edit milestones
A Few Year-One NMI-EDIT Milestones
  • Sept 1, 2001 – Grant awarded
  • Oct 2001– eduPerson 1.0 finalized; outreach begins with multiple workshops
  • Jan 2002 – HEBCA tested; first CAMP workshop held
  • Feb 2002 – PKI Lite CP/CPS; e-Gov and Management and Leadership Best Practice Awards
  • April 2002 – Shibboleth alpha ships; NMI testbed selected; NIST/NIH PKI workshop
  • May 2002 – NMI release, with eduPerson 1.5, pubcookie, KX.509, groups and metadirectories, video white papers
  • June 2002 – affiliated directories begins; Base CAMP; testbed kickoff
  • July 2002 – Shibboleth alpha v 2 ships; Advanced CAMP
  • August 2002 – LDAP Analyzer testing begins; Shibboleth pilot-sites selected; Work with content providers begins
  • September 2002 – Grant renewed; supplemental grant awarded for outreach; Shibboleth beta ships
nmi edit release 1 deliverables
NMI-EDIT: Release 1 Deliverables
  • Software

KX.509 and KCA, Certificate Profile Maker, Pubcookie

  • Object Classes

eduPerson 1.0, eduPerson 1.5, eduOrg 1.0, commObject 1.0

  • Service

Certificate Profile Registry

nmi edit release 1 deliverables37
NMI-EDIT: Release 1 Deliverables
  • Conventions and Practices
    • Practices in Directory Groups 1.0, LDAP Recipe 2.0
    • Metadirectory Practices for the Enterprise Directory in Higher Education 1.0
  • White Papers
    • Shibboleth Architecture v5
  • Policies
    • Campus Certificate Policy for use at the Higher Education Bridge Certificate Authority (HEBCA)
    • Lightweight Campus Certificate Policy and Practice Statement (PKI-Lite)
    • Sample Campus Account Management Policy
nmi edit release 1 deliverables38
NMI-EDIT: Release 1 Deliverables
  • Works in Progress
    • Role of Directories in Video-on-Demand
    • Resource Discovery for Videoconferencing
    • Directory Services Architecture for Video and Voice Conferencing over IP (commObject)
nmi edit release 2 new revised deliverables
NMI-EDIT: Release 2 New/Revised Deliverables
  • Software 

Programs and Libraries

      • OpenSAML 1.0               
      • Shibboleth 1.0               
      • Pubcookie 3.0

Directory Schemas

      • eduPerson            
      • eduOrg
nmi edit release 2 new revised deliverables40
NMI-EDIT: Release 2 New/Revised Deliverables
  • Conventions and Practices
    • LDAP Recipe       
    • Metadirectory Practices for Enterprise Directories    
    • Practices in Directory Groups
  • Architectures
    • Inter-domain Data Exchange (Draft)       
  • Services
    • LDAP Analyzer
the pieces fit together
The pieces fit together…

Campus infrastructure

Name space, identifiers, directories

Enterprise authentication and authorization

Inter-realm infrastructure

edu object classes

Exchange of attributes

Inter-realm Upperware


Digital libraries



NMI IntegrationTestbed

  • Mary Fran Yafchak
  • Testbed Manager,Southeastern Universities Research Association
  • maryfran@sura.org
nmi integration testbed
NMI Integration Testbed
  • Focus on the integration of released middleware components with real life use and conditions
  • Elements: Sites, Manager, Workshop
  • Integration is the point - could think of it as…
    • Where “EDIT” meets “GRIDS”
    • Where enterprise needs meet research needs
    • Where NMI components meet reality
nmi integration testbed44
NMI Integration Testbed
  • Planning and management by SURA
  • Participating Sites:
    • University of Alabama at Birmingham
    • University of Alabama in Huntsville
    • University of Florida
    • Florida State University
    • Georgia State University
    • Texas Advanced Computing Center (U Texas/Austin)
    • University of Virginia
    • University of Michigan
nmi integration testbed45




NMI Participation

















Target Communities

NMI Integration Testbed

NMI Integration Testbed

NMI Integration Testbed
nmi integration testbed recent activities
NMI Integration Testbed - Recent Activities
  • Testbed Kickoff June 10 - 12, 2002 at GSU
  • Site Integration Plans completed in July 2002
  • Testing of Release 1 completed
  • Press release & Web site announced 9/4/02
    • See http://www.nsf-middleware.org/testbed
  • Open Testbed BoF here at I2 Members’ Meeting
    • Wednesday, October 30, 11:45AM-1:15PM
nmi integration testbed some highlights from the sites
NMI Integration Testbed - Some Highlights from the Sites
  • Twenty-six very real institutional projects and applications “on board” for NMI integration - with more to come...
  • Ten projects targeting increased access to their existing or planning scientific grids (including emerging TeraGrid) through NMI Globus
  • Five sites actively implementing enterprise scale directories, with centralized authentication and integrated applications
  • Active PKI efforts, from PKI Lite to PKI “heavy” (maintaining HIPPA/FERPA compliance)
  • New collaborative tools also represented, such as click-to-dial desktop video conferencing and shared calendaring
nmi integration testbed from r1 to r2
NMI Integration Testbed - From R1 to R2
  • Summarizing evaluation results from R1 - to be made available on the Testbed Web site
  • Working with Outreach to disseminate lessons learned thus far
  • R2-specific Component Testing Guidelines under development
  • Testbed Sites actively refreshing site plans and project sets with respect to R2
  • R2 evaluation soon to be underway...
nmi integration testbed potential for expansion
NMI Integration Testbed - Potential for Expansion
  • Already on our minds...
    • Increase opportunities for both sponsored and unsponsored participation
    • Define a role and means of involvement for international participants
    • Define a role and means of involvement for corporate participants
    • Develop “hot topic” or application-specific testbeds
      • E.g., Digital Libraries, Digital Video, Medical middleware, Discipline-specific grids

NMI Outreach and Participation

NMI Participation and Outreach

  • Ann West
  • NMI-EDIT Outreach,EDUCAUSE/Internet2/Michigan Tech
  • awest@educause.edu
nmi outreach and participation
NMI Outreach and Participation
  • Targeted Communities
  • Outreach Strategy
  • Results from PHASE I – Building Awareness
  • Activities and Plans for PHASE II – Delivering on Promise
nmi outreach targeted communities
NMI Outreach – Targeted Communities

TERTIARY CIRCLE - General Interest

(Targeted NEW User Communities – Press/PR)


Participating/User Communities

(NEES, GriPhyN, TeraGrid, Campuses, etc.)

Corporate Partners (vendors, other industry)


NMI Project Team


GRIDS Center

NMI AdvisoryCouncil

nmi outreach strategy
NMI Outreach Strategy

Sep 2001

We are HERE in time

  • Phase I
  • Building Awareness (Months 1-9)
  • Creating effective intellectual capital building blocks for the general program
  • Broad information dissemination
  • Aggressive outreach/marketing to higher-ed, industry, research communities
  • Phase II
  • Delivering on promise (Months 6-20)
  • Interactive communication with primary user communities
  • Building and disseminating “success stories”
  • Development of detailed documentation, targeted events/activities and specific intellectual capital
  • Phase III
  • Extending the reach (Months 12-24)
  • Refining engagement strategy – independent and sustainable
  • Implementing “reference library” – evidence of program credibility.
  • Establish regular schedule of events/activities
phase i building awareness
Phase I – Building Awareness
  • GOAL: Creating effective intellectual capital building blocks regarding the general program –
    • Websites
      • www.nsf-middleware.org
      • www.grids-center.org
      • www.nmi-edit.org
    • Logo(s) created
    • Initial NSF Award Announcement Press Release issued (24 September 2001)
    • Three Campus Architecture and Middleware Planning workshops
phase i building awareness55
Phase I – Building Awareness
  • GOAL: Broad information dissemination –
    • E-lists created encouraging “virtual” involvement:
      • nmi-developer - Discuss NMI releases and development activities with the open source community.
      • nmi-supporter - Find out about Initiative-supporting opportunities and how you can incorporate components from and NMI release into your products.
      • nmi-user - Find out about new project and participation opportunities with the NMI.
      • news - Receive NMI and Initiative-related press information
      • nmi-announce - Keep informed about new developments in the NMI.
    • Internet2
      • mw-discuss – Discuss topics in enterprise middleware.
      • mw-announce – Receive information on upcoming educational and participation opportunities
phase i building awareness56
Phase I – Building Awareness
  • GOAL: Aggressive outreach/marketing to higher-ed, industry and the research community
    • Selected speaking opportunities – Internet2, EDUCAUSE, Coalition for Networked Information, GGF-3 and GGF-4, SC2001
    • Defined and implemented a strategy for involving selected vendor/industry partners – NMI Participation Model

NMI Participation




- Develop NMI-related or derived components- Support NMI components

Targeted User



  • Repackage NMI components and distribute under own label

- campuses

- GriPhyN, NEES, etc

Other Interested


- campuses

- industry

- government

NMI Testbed


- determined by Call

for Participation

phase ii delivering on the promise
Phase II – Delivering on the Promise
  • GOAL: Interactive communication with primary user communities –
    • NMI Documentation Team formed – working on plans for appropriate Technical and other documentation to support NMI Releases
    • Coordinated with Training and Support Team to ensure consistency and accuracy of messages to the users.
phase ii delivering on the promise59
Phase II – Delivering on the Promise
  • GOAL: Building and disseminating “success stories” and other achievements –
    • NMI Release 1 successfully delivered (7 May 2002)
    • Announcements of NMI R1 distributed via information distribution channels established – press lists, e-lists, website.
    • NMI Testbed Participants engaged in Outreach to share their stores
      • Internet2 tutorials and upcoming EDUCAUSE regional meetings
      • NMI SC2002 theme – “Sharing NMI User Experiences” (related activities and documentation)
phase ii delivering on the promise60
Phase II – Delivering on the Promise
  • Development of detailed documentation, targeted events/activities, and specific, in-depth intellectual capital
    • Articles
      • “Middleware: The New Frontier” – EDUCAUSE Review, July/August 2002
      • “Middleware: Addressing the Top IT Issues on Campus” – EDUCAUSE Quarterly, Vol. 25, No. 4 2002
    • Presentations planned – GGF5, I2 Fall Meeting, SC2002, EDUCAUSE Regionals, Coalition for Networked Information, and others
    • Tutorials/workshops planned – GRIDS Center, EDIT
      • SC2002, EDUCAUSE, Internet2 meetings, and others
    • Schedule for Year 2 deliverables in progress
      • Base CAMP February 5-7, 2003 - Tempe, Arizona
interested in nmi
Interested in NMI?
  • Visit the NMI web – http://www.nsf-middleware.org
  • Subscribe to NMI discussion and awareness e-lists
  • Potential User?
    • Visit the NMI Release site and “test drive” NMI packaged releases.
  • Potential Contributor?
    • Send email to nmi-supporter@nsf-middleware.org indicating interest in contributing components to the next release.
  • Press?
    • Contact outreach@nsf-middleware.org about newsworthy activities and achievements.
  • Alan Blatecky – ablatecky@nsf.gov
  • Ken Klingenstein – kjk@internet2.edu
  • John McGee – mcgee@isi.edu
  • Mary Fran Yafchak – maryfran@sura.org
  • Ann West –awest@educause.edu