End-User Tools for Grid Computing

1. End-User Tools for Grid Computing Francisco Hernández Purushotham Bangalore Kevin Reilly Department of Computer and Information Sciences University of Alabama at Birmingham, Birmingham, AL International Conference on Software Engineering, 2005 Proceedings of the first workshop on End-user software engineering, pages 1-5 Poster by: Luai Mahmud All material from above paper unless specified at bottom of slide

2. End-User tools for Grid Computing Summary • Grid computing is a valuable resource to access extensive programming capabilities but is too complicated for most to implement because of the various technologies required. • Abstract the complexities of Grid computing in order to make development and deployment achievable without the need for extensive support requirements or exclusive solutions. • Current directions in simplifying the Grid either lack flexibility or require expertise of multidisciplinary support teams.

3. End-User tools for Grid Computing Summary • Approach this problem by constructing an end-user tool to provide a high-level abstract layer for the construction of Grid applications composed of visual models of specific application domains constructed using the concepts of domain-specific modeling. • Programs that manage the application execution are generated from the corresponding visual models. • Users don’t need to learn to use the specific Grid technologies in order to develop Grid-enabled applications.

4. End-User tools for Grid Computing Concepts • Definition: • A distributed computing approach which offers computing specialists and other scientists a valuable resource to access extensive processing capabilities through the creation of virtual organizations that permit seamless aggregation of heterogeneous resources that can belong to different administrative domains. • Globus Toolkit is the de facto standard. The approach is to act a middleware to provide virtualization of resources. http://www.globus.org/

5. End-User tools for Grid ComputingConcepts • Evolution of Grid computing • Web services has altered the approach of traditional Grid computing and both have converged into Grid Services. • Grid Services are Web services at the base providing interoperability, heterogeneity, and platform independence, but Grid services also enable working on a Grid environment with features such as support for life cycle management and notifications.

6. End-User tools for Grid Computing The Problem • Complexity of Developing a Grid service • Implement the functionality of the service which might require the use of scientific libraries and constraints on efficiency expected for the service • Write a functional interface for the service so that clients know how to invoke it. • Define the deployment parameters to indicate the specifics of the service. • Deploy the service to a Grid service container and register it so that it can be found by other services. Most of the previous are done manually so few apps can exploit the full Grid potential

7. End-User tools for Grid Computing The Problem • How to convert legacy apps designed for dedicated supercomputers or parallel computers with no attention to distributed computing • Complex apps consist of more than one service so special effort to compose (creating complex apps of smaller services) and orchestrate (sequencing these services for desired result) services. This requires specific knowledge of Grid technology.

8. End-User tools for Grid Computing The Problem • Composing services – the below steps must be done for each service. • Discover the service from a registry which in response indicates the location of the service. • Obtain the service description which indicates how to use the service • Generate code needed to invoke the service according to its description, then use this code to connect the service to the application or service being implemented.

9. End-User tools for Grid Computing Current Directions • Problem Solving Environments (Portals) • Attempts to simplify the use of the Grid by creating a Problems Solving Environment (PSE) or portal. • Supplies a repository of ready-to-use applications that were previously created and can be reused utilizing different inputs • Portals only hide complexity by appearing to expedite simple task like job submission and checking job status. • Lack flexibility required to create complex apps composed of different services.

10. End-User tools for Grid Computing Current Directions • Workflow Systems • Facilitate construction of applications by creating a workflow of services composing the application. • Terminology is borrowed from workflow languages like BPEL which have been successful in composing and orchestrating business related Web services.… • As in the case of PSE or portals, the workflow system requires services to be independently developed and stored in a repository for later use. • Process is complex and often requires the expertise of multidisciplinary support team.

11. End-User tools for Grid Computing Current Directions • Component Frameworks • Development of individual services is equally important as is the simplification of composition of services. • Attacks the problem by constructing frameworks that ease the implementation of individual components • Component frameworks are engineered to facilitate and accelerate the development of applcations by focusing on the reuse of individual components.

12. End-User tools for Grid ComputingCurrent Directions • Component Frameworks – Three problems • Most component frameworks were developed before Gird “era”, thus must be adapted • Different component frameworks are not standardized • Current procedures from composing components are focused at the code level where the complexities of programming languages and the component frameworks impeding there use. • Thus multidisciplinary support teams are required for component framework use in Grid computing

13. End-User tools for Grid Computing Methodology • Problem that we commonly see in current direction is the reliance on multidisciplinary support teams. • Trend in other areas of CS is to create tools that empower users to create complex applications without the need for such teams. • There seems to be a need for support tools that function in domains more familiar to end-users. • Tools should not be composed of ad-hoc methods but instead rely on modern software engineering practices that can increase reliability and improve development of such tools.

14. End-User tools for Grid Computing Methodology • Two issues of creating Grid applications: • Creation of individual application components or services • Deployment of those components in the distributed resources.

15. End-User tools for Grid Computing Methodology • In order to increase the use of Grid computing then we must hide the complexities of the use from end-users and embed this knowledge into a code generator that can generate complex configurations. • Use technology such as Domain-specific modeling (DSM).

16. End-User tools for Grid Computing Methodology • Domain-specific Modeling • Domain-Specific Modeling raises the level of abstraction beyond programming by specifying the solution directly using domain concepts. The final products are generated from these high-level specifications. This automation is possible because both the language and generators need fit the requirements of only one company and domain. Your expert defines them, your developers use them. http://www.dsmforum.org/

17. End-User tools for Grid Computing Methodology • Process for creating the domain models: • Analyze the domain in order to extract concepts relevant to the domain as well as knowledge on how to build applications in that domain (domain engineering). • Build a meta-model with the knowledge extracted during the domain engineering steps, creating in the process a graphical domain-specific language to specify applications for the domain. • Create a model interpreter to generate the appropriate low-level configurations. The model interpreter provides the semantics for the visual models.

18. End-User tools for Grid Computing Methodology • Users then interact with graphical models which represent concepts familiar to them and the corresponding grid applications are automatically generated by the tool. • The particular code that enables use of the distributed resources can be reused because the underlying Grid technologies are the same on every application domain.

19. End-User tools for Grid Computing Methodology • Benefits to proposed methodology • Code can be optimized and developed by Grid experts and then used by end-users working on different application domains. • Quality of applications is improved since the rules of the programs that can be created are embedded in the meta-models. • Valid models are only accepted and bugs are also minimized.

20. End-User tools for Grid Computing Example • Development of modeling tools is facilitated by use of MetaCASE MetaCASE tools which are beneficial for this endeavor since they provide support for meta-model editing as well as the creation of new notation • The following is an screenshot of a MetaCASE tool: MetaEdit. http://www.metacase.com/

21. End-User tools for Grid Computing Example • Initial exploration for creating a tool that automates work with the Grid. • Example involved the creation of a workflow system that abstracts and simplifies the development of Grid applications by hiding the low-level implementation details of the Grid middleware. • Facilitates the deployment part of Grid application construction process

22. End-User tools for Grid Computing Example • Tool provides the inexperienced Grid user by providing an environment in which they can graphically specify the workflow for their application and automatically generate the code that manages the execution of the application. • Applications can be created by specifying jobs on distributed resources and by specifying file transfers between those resources. Usually jobs require one or more input files and also produce one or more output files.

23. End-User tools for Grid Computing Example • Application using Hidden Markov Models (HMM) to illustrate use of tool • HMM was constructed to compare the differences between English and Spanish Language patterns. • Input: intermingled file (parts in En and Sp) that only indicates if a letter is a vowel or consonant. The output file consists of the language prediction.

24. End-User tools for Grid Computing Example

25. End-User tools for Grid Computing Example • Above illustrates the manner in which a job is specified by users. • After all tasks are defined, the application can be constructed by specifying the required sequence of tasks. (see next slide) • After the model is specified, the model interpreter transverses the internal representation of the model and generates the control code that manages the application execution.

26. End-User tools for Grid Computing Example

27. End-User tools for Grid Computing Example

28. End-User tools for Grid Computing Example • End-users need only to know the particulars of their applications and are not required to learn the manner in which the Grid technologies operate. • End-users are able to specify complex applications and execute them in a distributed resources.

29. End-User tools for Grid Computing Conclusion • Improve the development of tools to automate the creation of Grid applications from particular domains. • Tools using this approach permit the graphical definition of models and the automatic generation of code that controls the execution of the Grid applications.

30. End-User tools for Grid Computing Conclusion • Research is based on benefits of domain-specific modeling techniques. • Domain modeling focus on higher levels of abstraction at the problem space rather than solution space (such as Grid Middleware). • Modeling tools and their code generators facilitate the rapid ability to change the application details…domain knowledge is embedded in the rules. • Quality of the systems is improved since the high level models only permit the specification of correct models and low-level implementations are coded by Grid experts. • Use of MetaCASE tools facilitate the rapid development of Grid tools for different application domains.

31. End-User tools for Grid Computing Conclusion • What was accomplished... • A tool was created t abstract the Grid domain and permitted the specification of applications that were able to run in distributed resources. • Users were able to submit their applications by graphically specifying the details of their application. • End-users are better able to understand models that are expressed in there day-to-day language rather that in cumbersome and extraneous programming laguages. • Gain access to the processing capabilities of the Grid without the need of expertise of support teams.

32. End-User tools for Grid ComputingCritique – Strengths of Paper Spends a good amount of effort to point out the problems with difficulty of using Grid computing for the ‘end-user’. Lists current approaches to dealing with the complexity of Grid computing and shows gives the pros and cons of each leading to justification for the authors’ own path of research. The proposed methodology on how to tackle the current complexity of Grid computing are spelled out quite well with a clear direction about how the authors plan to continue there future work.

33. End-User tools for Grid ComputingCritique – Weaknesses of Paper • The title of the paper emphasized end-user tools but the paper’s example section which focuses on the tool created by the authors, is given very little attention. • A very trivial example is pursued with no real comparison to another solution. • Most of the paper is mostly a regurgitation of one Francisco Hernandez’s technical Report : UABCIS-TR-2004-0504-1. (written in the previous year). • What where the problems to the current methodology and or toolset? How an end-user perceives a tool ease of use, regardless about how effective the tool actually is, could be important.