1 / 27

The Gateway System

The Gateway System. This project is a collaborative effort between Northeast Parallel Architectures Center (NPAC) Ohio Supercomputer Center (OSC) Aeronautical Systems Center (ASC) MSRC. Goals.

ratana
Download Presentation

The Gateway System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Gateway System This project is a collaborative effort between Northeast Parallel Architectures Center (NPAC) Ohio Supercomputer Center (OSC) Aeronautical Systems Center (ASC) MSRC T. Haupt

  2. Goals • To provide a problem-oriented interface (a Web portal) to more effectively utilize HPC resources from the desktop via the Web browser. • This “point & click” view hides the underlying complexities and details of the HPC resources and creates a seamless interface between the user’s problem description on his/herdesktop system and theheterogeneous computing resources • TheseHPC resources include supercomputers, mass storage systems, databases, workstation clusters, collaborative tools, and visualization servers. T. Haupt

  3. Design Issues • Support for a seamless access (security) • Support for distributed, heterogeneous Back-End services (HPCC, DBMS, Internet, ...) managed independently from Gateway • Variable pool of resources: support for discovery and dynamical incorporation into the system • Scalable, extensible, low-maintenance Middle Tier • Web-based, extensible, customizable, self-adjusting to varying capacities and capabilities of clients (humans, software and hardware) front end T. Haupt

  4. Towards a complete solution ... Problem description:I need to model the surface damage due to the impact of laser to harden the material bulk. I need access to models including material bulk properties and interaction with intense electromagnetic fields. Task description: I need 64 nodes of SP-2 at Argonne to run my MPI-based executable “a.out” you can find in “/tmp/users/haupt” onmarylin.npac.syr.edu. In addition, I need any idle workstationwith jdk1.1 installed. Make sure that the output of my a.out is transferred to that workstation Middle-Tier: map the user’s task description onto the resource specification; this may include resource discovery, and other services Resource Specification Resource Allocation: run, transfer data, run

  5. Front-End Services User Modules Back-End Resources Three-Tier Architecture • Tier 1 is a high-level, browser-based Front End • for visual programming (including selection of applications, • generation of input data sets, specification of resources, • post-processing and visualizations) • Distributed, object-based, • scalable, and reusable • Web server, Object broker • and Resource Manager • Middleware forms Tier 2 • Back-End services • comprise Tier 3. T. Haupt

  6. Abstract Task Specification Middle-Tier Resource Specification Target Architecture Problem Solving Environment CTA specificknowledgedatabases Visual Authoring Tools User and Group Profiles ResourceIdentificationand Access Visualizations Collaboration WebFlow Back-End Resources

  7. Portal Page • Provides initial access to the Gateway. • After mutual authentication of the user and the Gateway server, creates a user context, and returns a (signed) control applet. T. Haupt

  8. User and Group Profile • Controls the user/group environment • file access • job monitoring • ... • Allows for customization • preferences • users with disabilities • ... • History of actions • Scientific notebook T. Haupt

  9. Control Applet • The control applet is responsible for maintaining the session, and direct communication with the middle-tier. • Direct communication is the most efficient, but since it is buried into an applet, this mechanism is not readily customizable. • The generic services, such as file service (upload, download, edit, copy, move, delete) and job services (show current jobs/show queues/kill jobs) will be supported this way. [combination of the user context and a query] • The Gateway will also support a non-direct communication with the middle-tier through servelts. T. Haupt

  10. Screen Dump of the Control Applet T. Haupt

  11. Application Domain DataBasesPSE Example: CCM 1. Enter the Gateway system 2. Define your problem 3. Identify resources (software and hardware) 4. Create input file 5. Run your application 6. Analyze results Ken Flurchick, http://www.osc.edu/~kenf/theGateway T. Haupt

  12. T. Haupt

  13. Selecting a Predefined Task T. Haupt

  14. Visual Authoring Tools • Allows for composition of the computational task from components (reusable modules) • Different tools to support various programming models such as data parallel, task parallel, data flow, object oriented • No assumption on granularity • Metadata about components and support for archiving and mining the components • Support for instrumentation and steering T. Haupt

  15. Example: Data Flow T. Haupt

  16. Example: DARP T. Haupt

  17. Navigate and choose an existing applicationto solve the problem at hand.Import all necessary data. Example of a portal Select host Select model Set parameters Run Retrieve data Pre/post-processing Run simulations

  18. Processing Task Specification Front-End Applet XML A visual representationis converted into a XML document parse Middle-Tier Web Server save XML service ApplContext Publishes IOR Generates Java code to add modules to ApplContext

  19. Other toolboxes • Visualizations • Collaboration • Scientific notebook • ... T. Haupt

  20. Mesh of WebFlow Servers implemented as CORBA objects that manage and coordinate distributed computation. Front End CORBA Based Middle-Tier Gatekeeper Authentication Authorization T. Haupt

  21. WebFlow Server WebFlow server is given by a hierarchy of containers and components WebFlow server hosts users and services Each user maintainsa number of applicationscomposed of custom modules and common services User 1 User 2 Application 1 App 1 App 2 Application 2 WebFlow Services

  22. WebFlow Context Hierarchy Master Server (Gatekeeper) Slave Server Proxy Slave Server Application Context Module Slave Server User Context T. Haupt

  23. Module functionality • May act as a client for a back-end service such as Globus GRAM or a database • May invoke other Gateway Middle Tier services such as File Services or Resource Discovery • May implement the desired functionality internally (say, in Java) [thus not a proxy] • May interact with other modules and the Front End through events T. Haupt

  24. Browser based Front-End BrowserbasedFront-End Middle-Tier modulesserve as proxies ofBack-End Services User Space Definition and Task Specification Services User Modules Metacomputing Services T. Haupt Back-End Resources

  25. Back End Services • Access to HPCC (via Globus) • Access to distributed databases (via JDBC) • Access to mass storage • Access to the Internet resources • Access to desktop application and local data • Access to code repositories T. Haupt

  26. In order to run WebFlow over Globus there must be at least one WebFlow node capable of executing Globus commands, such as globusrun Jobs that require computational power of massively parallel computers are directed to the Globus domain, while other jobs can be launched on much more modest platforms, such as the user’s desktop or even a laptop running Windows NT. Bridge between WebFlow and Globus WebFlow over Globus T. Haupt

  27. Security Model (Keberos) Front End Applet SECIOP Layer 1: secure Web delegation Layer 2: secure CORBA Gatekeeper SECIOP Layer 3: Secure access to resources authentication & authorization GSSAPI GSSAPI HPCC resources Policies defined by resource owners

More Related