1 / 18

Commodity Workflows for the Grid: An Evaluation of Microsoft Windows Workflow Foundation

2. Perceived Problem Grid Usability. If a grid interface or the working environment is (apparently) too difficult to use, it will not be adopted by many of the appropriate usersMost scientists, engineers, and researchers will be (service) end usersCurrently grids are serving power users and do not yet benefit non-computer-technical researchers.

tyme
Download Presentation

Commodity Workflows for the Grid: An Evaluation of Microsoft Windows Workflow Foundation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Commodity Workflows for the Grid: An Evaluation of Microsoft Windows Workflow Foundation Quinn S. Lewis Grid Computing (COSC6376) 14 December 2006

    2. 2 Perceived Problem – Grid Usability If a grid interface or the working environment is (apparently) too difficult to use, it will not be adopted by many of the appropriate users Most scientists, engineers, and researchers will be (service) end users Currently grids are serving power users and do not yet benefit non-computer-technical researchers http://wiki.oucs.ox.ac.uk/esp-grid/Final_recommendations_for_e-Science http://wiki.oucs.ox.ac.uk/esp-grid/JISC_Proposal_for_Phase_2?action=AttachFile&do=get&target=ESPGRID_miniExtensionProposalAug06v1.pdfhttp://wiki.oucs.ox.ac.uk/esp-grid/Final_recommendations_for_e-Science http://wiki.oucs.ox.ac.uk/esp-grid/JISC_Proposal_for_Phase_2?action=AttachFile&do=get&target=ESPGRID_miniExtensionProposalAug06v1.pdf

    3. 3 One Approach Improve the usability of the tools used to interact with the grid Should be natural and easy-to-use Take advantage of other familiar commodity tools and technologies Windows Workflow Foundation Free Platform level component Used by other (Microsoft) tools

    4. 4 Workflows Defined Workflow is a set of activities that describe the implementation of a process Workflow can be short or long running and can involve people and systems Business Processes and Web Services: Strategies and Standards (John Evdemon) Keynote.ppt Images taken from http://msdn2.microsoft.com/en-us/netframework/aa973808.aspx.Business Processes and Web Services: Strategies and Standards (John Evdemon) Keynote.ppt Images taken from http://msdn2.microsoft.com/en-us/netframework/aa973808.aspx.

    5. 5 Workflows in Technical Computing Accelerating Discovery in Science and Engineering (Fabrizio Gagliardi) Taken from Santander-Fab.pptAccelerating Discovery in Science and Engineering (Fabrizio Gagliardi) Taken from Santander-Fab.ppt

    6. 6 Pillars for Connected System Business Processes and Web Services: Strategies and Standards (John Evdemon) Keynote.ppt In some MS presentations I’ve seen this as “Services & Messaging, Interaction, Workflow, Identity & Accessed, Shared Data”. Business Processes and Web Services: Strategies and Standards (John Evdemon) Keynote.ppt In some MS presentations I’ve seen this as “Services & Messaging, Interaction, Workflow, Identity & Accessed, Shared Data”.

    7. 7 Connected Systems Architecture Model Business Processes and Web Services: Strategies and Standards (John Evdemon) Keynote.ppt In some MS presentations I’ve seen this as “Services & Messaging, Interaction, Workflow, Identity & Accessed, Shared Data”. Business Processes and Web Services: Strategies and Standards (John Evdemon) Keynote.ppt In some MS presentations I’ve seen this as “Services & Messaging, Interaction, Workflow, Identity & Accessed, Shared Data”.

    8. 8 Similarity to OGSA Image taken from https://forge.gridforum.org/sf/go/doc13070;jsessionid=3C9F6C4276EC4D99F021178AA9EB4F71?nav=1Image taken from https://forge.gridforum.org/sf/go/doc13070;jsessionid=3C9F6C4276EC4D99F021178AA9EB4F71?nav=1

    9. 9 Workflow Foundation Architecture Reference http://www.informit.com/articles/article.asp?p=418010&seqNum=4&rl=1 Although not used, may be helpful to consider http://www.codeproject.com/dotnet/UnderstandWWF.asp.Reference http://www.informit.com/articles/article.asp?p=418010&seqNum=4&rl=1 Although not used, may be helpful to consider http://www.codeproject.com/dotnet/UnderstandWWF.asp.

    10. 10 Work Performed Attempt to use Windows Workflow to connect to grid middleware WSRF.NET/OGSI.NET Globus Condor Explore re-hosting of Workflow Designer outside of VisualStudio Windows Workflow Foundation: Everything About Re-Hosting the Workflow Designer http://msdn2.microsoft.com/en-us/library/aa480213.aspx See also Windows Workflow Foundation Samples http://msdn2.microsoft.com/en-us/library/ms741723.aspxWindows Workflow Foundation: Everything About Re-Hosting the Workflow Designer http://msdn2.microsoft.com/en-us/library/aa480213.aspx See also Windows Workflow Foundation Samples http://msdn2.microsoft.com/en-us/library/ms741723.aspx

    11. 11 WSRF.NET/OGSI.NET From the University of Virginia Use Microsoft .NET to implement WSRF and OGSI Takes advantage of .NET attributes Uses VisualStudio Add-In to generate stubs (client/server) to handle state Code generation tool didn’t seem to work for web services on ports other than 80 Requires WSE 3.0

    12. 12 Condor Exposes job scheduling capability as web services to enable execution of complex computational workflows consisting of a number of inter-related jobs Ported SOAPScheddApiHelper to C# Reference: http://www.cs.wisc.edu/condor/doc/birdbath.pdf Reference: http://www.cs.wisc.edu/condor/doc/birdbath.pdf

    13. 13 Condor Text to Graphics In code… TransactionAndStatus tx = schedd.beginTransaction(60); IntAndStatus clust = schedd.newCluster(tx.transaction); IntAndStatus job = schedd.newJob(tx.transaction, clust.integer); ClassAdStructAndStatus cas = schedd.createJobTemplate( clust.integer, job.integer, owner, type, cmd, args, reqs); Status st = schedd.commitTransaction(tx.transaction); RequirementsAndStatus reqs_s = schedd.submit( tx.transaction, clust.integer, job.integer, cas.classAd);

    14. 14 Unsecure Condor Setup Modify condor_config and run condor_restart ENABLE_SOAP=TRUE ALLOW_SOAP=* HOSTALLOW_WRITE=* //or may get “Permission denied” from schedd.beginTransaction(30); QUEUE_ALL_USERS_TRUSTED=TRUE //see next slide Collector web services runs on port 9618 http://<machine>:9618 (couldn’t use localhost) Check CollectorLog for more info Schedd runs on random port Check …\log\.schedd_address for port Start explicitly in condor_config or via collector web service (SCHEDD_ARGS=-p 1134) [Reference] Manage grid resources with Condor Web services https://www6.software.ibm.com/developerworks/education/gr-wscondor/index.html Developer APIs to Condor + A Tutorial on Condor’s Web Service Interface http://www.cs.wisc.edu/condor/CondorWeek2006/presentations/farrellee_tannenba_APIs.ppt Where to begin? https://lists.cs.wisc.edu/archive/condor-users/2006-August/msg00322.shtml Other “magic” condor_config vars https://lists.cs.wisc.edu/archive/condor-users/2006-September/msg00223.shtmlManage grid resources with Condor Web services https://www6.software.ibm.com/developerworks/education/gr-wscondor/index.html Developer APIs to Condor + A Tutorial on Condor’s Web Service Interface http://www.cs.wisc.edu/condor/CondorWeek2006/presentations/farrellee_tannenba_APIs.ppt Where to begin? https://lists.cs.wisc.edu/archive/condor-users/2006-August/msg00322.shtml Other “magic” condor_config vars https://lists.cs.wisc.edu/archive/condor-users/2006-September/msg00223.shtml

    15. 15 Condor Problem Get "Could not create new cluster" when trying to “IntAndStatus cluster = schedd.newCluster(xact.transaction);” SchedLog says “QMGT command failed: anonymous user not permitted” Undocumented answer is to add “QUEUE_ALL_USERS_TRUSTED=TRUE” to condor_config since no security is being used

    16. 16 Another Condor Problem Nothing seems to execute but it is submitted and get strange timer errors ShadowLog 12/9 16:28:47 (24.0) (8000): ERROR: Could not locate valid credential for user 'Quinn@COPERNICUS' 12/9 16:28:47 (24.0) (8000): init_user_ids() failed! 12/9 16:28:47 (24.0) (8000): ERROR: Could not locate valid credential for user 'Quinn@COPERNICUS' 12/9 16:28:47 (24.0) (8000): init_user_ids() failed! 12/9 16:28:47 (24.0) (8000): ERROR "set_user_priv() failed!" at line 517 in file ..\src\condor_c++_util\uids.C SchedLog 12/9 16:25:43 (pid:4044) my_popen: CreateProcess failed 12/9 16:25:43 (pid:4044) Failed to execute C:\condor/bin/condor_shadow.pvm.exe, ignoring 12/9 16:25:43 (pid:4044) my_popen: CreateProcess failed 12/9 16:25:43 (pid:4044) Failed to execute C:\condor/bin/condor_shadow.std.exe, ignoring RESOLUTION…. Use condor_store_cred! (Reference) Don’t use condor_credd if you want something easy Will need to set a password even if you don’t normally use one (net user Quinn <password>) http://www.kellys-korner-xp.com/win_xp_passwords.htm

    17. 17 Globus Can’t consume GT4 web services from .NET Tried manipulating WSDL Discovered Microsoft and Globus didn’t use some WS-Addressing mechanism Thought to use Axis and found out a C++ version exists Complicated install and language mixing issues

    18. 18 Closer to home… UH Work Computational Data Grid for Scientific and Biomedical Applications Marc Garbey and Victoria Hilford Rapid application development (use of the Microsoft Visual Studio .NET technology) Visual modeling interfaces (forms driven Graphical User Interfaces) Database Connectivity (interface with Microsoft SQL Server 2005) Query support (clients can store, update, delete, retrieve database metadata) Context-sensitive, role-based access (Microsoft Windows Server 2003, ASP.NET) Robust security (HIPPA compliance through Microsoft’s Authentication and Authorization from IIS and ASP.NET) Connectivity to other biomedical resources (PACS, DICOM, XML) http://research.microsoft.com/workshops/escience2005/ http://research.microsoft.com/workshops/escience2005/videos/12797/lecture.htmhttp://research.microsoft.com/workshops/escience2005/ http://research.microsoft.com/workshops/escience2005/videos/12797/lecture.htm

    19. 19 References http://ws.apache.org/axis/cpp/index.html http://www.windowsclusters.org/condor_guide.htm Condor WS not possible w/ 6.8.0 and helper? https://lists.cs.wisc.edu/archive/condor-users/2006-August/msg00350.shtml

More Related