Facilitating the process of enabling applications within grid portals

1. Facilitating the process of enabling applications within grid portals Michal Kosiedowski kat@man.poznan.pl

2. PSNC was established in 1993 and is a R&D center in: New Generation Networks POZMAN and PIONIER networks 6-NET, SEQUIN, ATRIUM projects HPC and Grids GridLab, Crossgrid, SGIGrid,PROGRESS, Clusterix projects Portals and Content Management Tools Polish Educational Portal "Interkl@sa", Multimedia City Guide, Digital Library Framework,Interactive TV Poznan Supercomputing and Networking Center

3. Budget: ~4,0 MEuro Project Partners Sun Microsystems Poland PSNC IBCh Poznań Cyfronet AMM, Krakow Technical University Lodz Co-funded by the State Committee for Scientific Research (KBN) and Sun Microsystems Poland Goal: construct an access environment to grid resources and services PROGRESS

4. PROGRESS: architecture

5. Job Submission Service computing job building, submitting jobs to the grid for execution and monitoring the execution grid job descriptions in the XRSL language are transferred to the GRBfor the execution of the job possibility to prepare workflows Application Management Service manages the PROGRESS application repository an application descriptor contains a reference to the application’s executable and a set of its available, required or optional arguments, required environment variables as well as input and output files one executable may be referenced by many applications different application configurations are recognized as independent applications Grid Service Provider

6. A distributed system enabling the management of grid data files Storing files in distributed storage modules of various types: generic filesystems, archivers, relational databases Uses metadata to describe files Allows access to data banks like a mirror of Sequence Retrieval System Exposes its functionality within the Data Broker Service Data Management System

7. PROGRESS HPC Portal

8. “My computing jobs”: creation, configuration and execution of grid jobs on the grid downloading and visualizing the results utilizes the functionality of the GSP Job Submission and Application Management services and the DMS Data Broker. “Applications”: management of grid-enabled applications within the GSP application repository co-operates with the GSP Application Management Service and the DMS Data Broker. “My data”: management of the files stored within the DMS uploading and downloading files from the DMS communicates with the DMS DataBroker. Core portlets

9. Install the application on a computing server or prepare an executable and upload it to the Data Management System Use the “Applications” portlet to add the application to the application repository managed by the Application Management Service Use the “My computing jobs”portlet to build a grid job utilizing the newly added application Enabling an application within the PROGRESS HPC Portal

10. Exposes the whole functionality of the Job Submission Service Enables submission of any job utilizing any available application Great for experienced users allowing them to fulfill their requirements Too complicatedfor use by non-advanced users Solution: specialized application portlets utilizing the same services as core portlets and familiar with the corresponding grid-enabled application structure “My computing jobs”portlet

11. “Gaussian” portlet (1)

12. “Gaussian” portlet (2)

13. “Gaussian” portlet (3)

14. “Gaussian” portlet (4)

15. Provide users with interfaces that are easy to use and require no grid knowledge Grid jobs may be configured very quickly with an intuitive wizard Easy and quick access to results: application portlets are familiar with corresponding applications Problem: How to quickly build such specialized portlets? Application portlets

16. The Framework allows preparing a high-level interface to Web Services with special beans to store the data obtained from WS responses and classes used to translate HTTP requests into the invocation of proper WS methods. This allows to quickly build new portlets on top of those classes to utilize the already existing functionality that they provide. PROGRESS Portlet Framework

17. Web Service Proxy Request Handler Web Service Proxy Request Handler Request Handler Request Handler Request Handler Request Handler HTTP Provider Provider Provider Provider Provider ... Portlets Content Generator Content Generator Content Generator Content Generator ... Content Generator ... Web Service Proxy Web Service Proxy Web Service Proxy ... Web Services Invocation Framework SOAP (to GSP, DMS)

18. Web Services Invocation Framework is used for communication with Web Services Object executeMethod( String methodName, ArgumentTable methodArguments) AXIS toolkit is used to generate the binding classes for complex class types mapping Each Web Service is described in an XML configuration file Communication with Web Services (1)

19. <?xml version="1.0" encoding="UTF-8"?> <web-services> <web-service id="appmgmt"> <name>Application Management</name> <wsdl-location>ApplicationManagementWS.wsdl</wsdl-location> <port-type-name> http://viola.man.poznan.pl:5021/axis/services/ApplicationManagement </port-type-name> <port-type-namespace>ApplicationCtrl</port-type-namespace> <description>GSP Application Management Service</description> </web-service> <web-service id="jobs"> <name>Job Submission</name> <wsdl-location>JobSubmission.wsdl</wsdl-location> <port-type-name> http://viola.man.poznan.pl:5021/axis/services/JobSubmission </port-type-name> <port-type-namespace>JobSubmissionCtrl</port-type-namespace> <description>GSP Job Submission Service</description> </web-service> <web-services> Communication with Web Services (2)

20. The content is generated based on two HTTP request parameters (page and action). page e.g. ApplicationList, JobList action e.g. removeApplication, submitJob Actions are defined in actions.xml, pages in pages.xml (the content of those files is read by appropriate classes, ContentGenerator reads Action and Page objects from those classes). Actions are executed before page content is generated; pages usually contain default actions to generate their content. Actions are usually page independent, they can also replace the default page action An action is associated with one or more RequestHandler methods,or with methods of many RequestHandlers The Java reflection mechanism is used to call RequestHandler methods Content generation (1)

21. Page content is delivered by RequestHandlers in form of PageBean classes Each page can contain one or more classes implementing the PageBean interface (PageBean classes know how to deliver their content in form of a DOM) The page generates its content by converting its PageBeans into a DOM and then transforming it using an XSLT template Content generation (2)

22. DOM1 DOM2 DOM3 DOM4 Content Generator Page DOM PageBean1 RequestHandler1 HTML PageBean2 PageBean3 RequestHandler2 PageBean4 RequestHandler3

23. Pages are defined in an XML config file: <pages> <page name="ApplicationList"> <display-name>Application List</display-name> <description>List of PROGRESS applications</description> <xsl-file>application-list.xsl</xsl-file> <action name="ApplicationList"> <request-handlers> <request-handler class="ApplicationGetRequestHandler"> <method name="getApplicationList"/> </request-handler> </request-handlers> </action> </page> ... </pages> Content generation (3)

24. Invocation of manyRequestHandlers <page name="EditApplicationExecutable"> <display-name>Edit Application Executable</display-name> <description> Edit application executable file - choose/change the file. The executable can also be uploaded to SZD. </description> <xsl-file>edit-application-executable.xsl</xsl-file> <action name="EditApplicationExecutable"> <request-handlers> <request-handler class="ApplicationGetRequestHandler"> <method name="getApplication"/> </request-handler> <request-handler class="SzdGetRequestHandler"> <method name="getUserDirectoryListing"/> </request-handler> </request-handlers> </action> </page> Content generation (4)

25. RequestHandler “workflow” <page name="JobList"> <display-name>Job List</display-name> <description>List of jobs for a given user</description> <xsl-file>job-list.xsl</xsl-file> <scripts> <script language="JavaScript" src="jscript/job-submission/job_list.js" /> </scripts> <action name="JobList"> <request-handlers> <request-handler class="JobGetRequestHandler"> <method name="getJobsOfUser" output-name="userJobs"/> <method name="filterJobs" input-name="userJobs" output-name="filteredJobs"/> <method name="sortJobs" input-name="filteredJobs"/> </request-handler> </request-handlers> </action> </page> Content generation (5)

26. Page error redirection – defined in anerror-redirections.xmlfile: <?xml version="1.0" encoding="UTF-8"?> <redirections> <error-redirection-group id="EditJob:saveJob"> <error-redirection type="validation" redirect-to-page="EditJob" /> </error-redirection-group> <error-redirection-group id="EditTaskInfo:saveTaskInfo"> <error-redirection type="validation" redirect-to-page="EditTaskInfo" /> </error-redirection-group> </redirections> Content generation (6)

27. It’s easy to reuse the portlets in different portal frameworks and on various portals and websites The top-level Provider class can be easily adjusted to implement any portal framework API The content generated by a portlet can be easily changed by playing with configurations files The same RequestHandlers can be used by mutliple portlets thus enabling easy creation of new portlets communicating with the same WS services specialized application portlets, e.g.: “Gaussian” portlet, “DNA Assembly” portlet Framework advantages

28. PROGRESS grid access environment features the Application Management Service which enables to easily manage applications available in the grid, the “Applications” portlet which allows to utilize the functionality of this service and the “My computing jobs” portlet which can be used to create a computing job based on any application available in the repository. Our experience shows it’s important to provide users with interfaces to grid-enabled applications that are easy to use and do not require deep knowledge about the grid and/or the application. PROGRESS Portlet Framework facilitate the construction of specialized application portlets that deliver the functionality of the corresponding grid-enabled applications within a grid portal. Conclusions

29. http://progress.psnc.pl/ http://progress.psnc.pl/portal/ progress@psnc.pl PROGRESS