1 / 33

Holding slide prior to starting show

Holding slide prior to starting show. Portal Interfaces for Grid-based Scientific Computing. David Walker Cardiff University david@cs.cf.ac.uk. The GECEM Project. GECEM = Grid-Enabled Computational ElectroMagnetics. Two-year project funded by the UK Department of Trade and Industry (DTI).

karah
Download Presentation

Holding slide prior to starting show

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Holding slide prior to starting show

  2. Portal Interfaces for Grid-based Scientific Computing David Walker Cardiff University david@cs.cf.ac.uk

  3. The GECEM Project • GECEM = Grid-Enabled Computational ElectroMagnetics. • Two-year project funded by the UK Department of Trade and Industry (DTI). • Collaborators: • Cardiff University • University of Wales, Swansea • BAE SYSTEMS • Hewlett-Packard • Singapore Institute of High Performance Computing • Welsh e-Science Centre CCGSC2006, Flat Rock, North Carolina

  4. Main Project Objective • To apply grid technologies to enable large-scale scientific and engineering research across a globally-distributed extended enterprise. • The partners in the extended enterprise only partially trust one another. • CEM was chosen as the application area, but our approach could also be applied to CFD and structural mechanics (for example). CCGSC2006, Flat Rock, North Carolina

  5. Subsidiary Project Objectives • Explore secure sharing of code and simulation capability. • Grid-enable legacy code by exposing software components as services. • Develop a GECEM portal as an integrated user interface to the underlying GECEM services and tools. CCGSC2006, Flat Rock, North Carolina

  6. Aims of Talk • Brief overview of GECEM portal. • Lessons learned from the portal. • End-user tools for workflow composition and portal development. CCGSC2006, Flat Rock, North Carolina

  7. Geometry Create surface mesh Create volume mesh Perform CEM simulation Visualize The GECEM Workflow • GT3.2 services are used to create the surface and volume meshes, and to initiate the CEM simulation. • The CEM simulation is migrated to a remote machine for execution. • Simulation results are collaboratively visualised using the RAVE system. CCGSC2006, Flat Rock, North Carolina

  8. UWS BAE SYSTEMS Geometry data Generate surface and volume meshes Create geometry Migrate code and meshes Collaborative visualisation WeSC or IHPC Output RAVE system Perform CEM simulation GECEM Typical Use Case CCGSC2006, Flat Rock, North Carolina

  9. GECEM Portal • Based on GridSphere 2.0.4 portal framework and GridPortlets 1.1.1. • Uses Tomcat 5.0.30. • Accesses GT3.2.1 services for mesh generation and CEM simulation migration. • Accesses “vanilla” Web services for collaborative visualisation via RAVE. • Uses JSR-168 compliant portlets to construct the portal. CCGSC2006, Flat Rock, North Carolina

  10. Portlets • Portlets = pluggable user interface components • Portlets are designed to run in the context of a portal. • They conform to the Portlet API. • Users’ view: a portlet is a window in a portal that provides a specific service. • Developers’ view: a portlet is a software component written in Java, managed by a portlet container, that handles user requests and generates dynamic content. CCGSC2006, Flat Rock, North Carolina

  11. What is a Portlet? • A portlet is a Java-based Web component that processes requests and generates dynamic content. • The content generated by a portlet is called a fragment, a piece of markup (e.g., HTML, XHTML, or WML (Wireless Markup Language)) adhering to certain rules. • A fragment can be aggregated with other fragments to form a complete document. • A portlet's content normally aggregates with the content of other portlets to form the portal page. • A portlet container manages a portlet's life cycle. CCGSC2006, Flat Rock, North Carolina

  12. GECEM Portal Design MyProxy Server UDDI registry Surface Mesh Service Volume Mesh Service GECEM Portal GridPortlets CEM Migration Service GECEM Portlets RAVE Services CCGSC2006, Flat Rock, North Carolina

  13. GridPortlets • Resource Browser Portlet • Resource Registry • Credential Retrieval Portlet • Credential Management Service uses credential repository to store user credentials. • File Browser Portlet • Easy-to-use portlet for invoking Reliable File Transfer Service CCGSC2006, Flat Rock, North Carolina

  14. Main GECEM Portlets • File Selection Portlet: user selects input data for mesh services and CEM migration service, and location of output. Based on GridPortlets File Browser Portlet. • UDDI Portlet: user selects from a list of services. • Service Invocation Portlet: invokes the service selected by UDDI Portlet using files from File Selection Portlet. • RAVE Portlet: for collaborative visualisation. CCGSC2006, Flat Rock, North Carolina

  15. Other GECEM Portlets • Job Status Portlet: view status of submitted jobs and input/output files. User can logout of portal and login again later to check progress. • Progress Portlet: displays estimate of percentage of job completed. • Progress Log Portlet: displays information on events happening as a job progresses. • Conversion Portlet: converts ADF solver output file to RAVE-compatible format. CCGSC2006, Flat Rock, North Carolina

  16. RAVE Portlet • RAVE supports remote rendering and collaborative visualisation, navigation and manipulation of data sets. • Participants in a collaborative session can either explore data independently or in “follow-the-leader” mode. • The RAVE Portlet allows a user to select machines to host the RAVE Data Service and perform the rendering, and to initiate or join a collaborative visualisation session. CCGSC2006, Flat Rock, North Carolina

  17. Security • Makes use of certificates and Grid Security Infrastructure (GSI) from Globus project. • Provides for single sign-on and credential delegation. • Uses MyProxy Upload Tool embedded in portal to upload proxy certificate to online MyProxy repository. • GridSphere’s Logon Portlet retrieves proxy certificate and stores it in the GridPortlets credential repository. • Services authenticate users by interacting with the GridPortlets credential repository. CCGSC2006, Flat Rock, North Carolina

  18. GECEM Screenshots • Login: • Welcome: • File selection: • Service discovery: • Job submission and status: • Progress: • RAVE: CCGSC2006, Flat Rock, North Carolina

  19. Positive Lessons Learned • Portal provides a high-level user interface for scientists to access distributed resources easily. • Portal shields users from the complexities of using the Grid. • Users can access the portal from any web browser. • Easy to integrate software from different sources into portal. CCGSC2006, Flat Rock, North Carolina

  20. Negative Lessons Learned • Debugging distributed applications is hard! • Portals built from portlets appear to be limited in their support for dynamic workflow composition. • In the GECEM portal we have a static linear workflow in which each node is a placeholder for a service selected by the user from a list. CCGSC2006, Flat Rock, North Carolina

  21. Workflow Composition • General-purpose service-based workflow composition environments are stand-alone, eg., Taverna, Triana, Kepler, rather than being web- or portlet-based. • Scientific users appear to want more generality than the static workflows typified in GECEM. • But they may not want the generality of a fully dynamic workflow composition environment. CCGSC2006, Flat Rock, North Carolina

  22. Portal Workflow Composition and Use • Design workflow structure. • Label workflow. • “Compile” workflow. • Assign specific service implementations to abstract services, and specific files to workflow inputs and outputs. • Execute workflow. • Monitor workflow execution. • Analyse/visualize workflow output. CCGSC2006, Flat Rock, North Carolina

  23. Workflow Design • Need a tool to create workflow structure by connecting together programming patterns or templates. • For example, these 2 patterns: can be used to build this workflow: CCGSC2006, Flat Rock, North Carolina

  24. Workflow Labelling • Associates an abstract service with each node in the workflow structure. • Associates an abstract file with input/output of a node. • Where do abstract service and abstract file labels come from? CCGSC2006, Flat Rock, North Carolina

  25. Workflow Compilation • Automatically generates the portlets needed to specify specific service implementation and input/output files in portal. • Configures portal to include these portlets. CCGSC2006, Flat Rock, North Carolina

  26. Labels for Abstract Services and Files • All equivalent services in the VO should be given the same name  abstract service. • All equivalent files in the VO should be given the same name  abstract file. • Need to determine which abstract file name is used to label each input/output of an abstract service. CCGSC2006, Flat Rock, North Carolina

  27. Assigning Unique Labels • By decree: VO authority stipulates how services and files are named. • From metadata: services and files are annotated with sufficient metadata to determine which services are equivalent, and which files can be associated with service input/outputs. • Metadata may reference ontologies. CCGSC2006, Flat Rock, North Carolina

  28. Service and File Registries Service registries Service Meta-registry Semantic Classification and Naming Virtualized File Store File stores CCGSC2006, Flat Rock, North Carolina

  29. Portal Service Meta-registry Service/file selection Workflow invocation Design Tool Labelling Tool Compilation Tool Execution monitoring Virtual File Store Analysis and visualization Workflow Composition and Portal CCGSC2006, Flat Rock, North Carolina

  30. Conclusions • Portals based on JSR-168 compliant portlets provide a good way of building user interfaces to distributed scientific applications. • But tools are needed to allow new scientific workflows to be integrated into a portal. CCGSC2006, Flat Rock, North Carolina

  31. Web Sites GECEM: http://www.wesc.ac.uk/projectsite/gecem/ RAVE: http://www.wesc.ac.uk/projectsite/rave/ CCGSC2006, Flat Rock, North Carolina

  32. Workflow Optimization Workshop • Workshop on “Workflow Optimization in Distributed Environments”. • e-Science Institute, Edinburgh, 19-20 October 2006. • Topics include: • Predictive scheduling • Performance-based middleware • Semantic support for service selection • Web service discovery and optimisation • Workflow optimisation • Workflow restructuring • http://www.nesc.ac.uk/esi/events/702/ CCGSC2006, Flat Rock, North Carolina

  33. CCGSC2006, Flat Rock, North Carolina

More Related