Condor Gotchas III

1. Condor Gotchas III • John Kewley • STFC Daresbury Laboratory • john.kewley@stfc.ac.uk Euro Condor Week 2008 Barcelona

2. Outline • In this talk we will look at a few aspects of condor that frequently cause problems for new users. • Custom attributes / ClassAds • Arguments and Environment variables • Connectivity (especially Firewalls) Euro Condor Week 2008 Barcelona

3. What is a "gotcha" • From Wikipedia: "A 'gotcha' is a feature of a system, a program or a programming language that works in the way it is documented, but is counter-intuitive and almost invites mistakes because it is both enticingly easy to invoke and completely unexpected and/or unreasonable in its outcome." Euro Condor Week 2008 Barcelona

4. Custom attributes • How do I declare my own attributes so my jobs can distinguish between Red Hat 9 and other Linuxes? • On Red Hat 9 machines add the following to its configuration: OPSYS_FLAVOUR = RH9 OPSYS_FLAVOUR = "RH9" STARTD_EXPRS = OPSYS_FLAVOUR STARTD_ATTRS = OPSYS_FLAVOUR, $(STARTD_ATTRS) • It can then be matched with a REQUIREMENTSstatement: REQUIREMENTS = (OPSYS_FLAVOUR = RH9) REQUIREMENTS = (OPSYS_FLAVOUR == RH9) REQUIREMENTS = (OPSYS_FLAVOUR == "RH9") REQUIREMENTS = (OPSYS_FLAVOUR =?= "RH9") √ Euro Condor Week 2008 Barcelona

5. Custom Attributes • Declare the attribute and match against it • Always quote non-numerics • "Export" the attribute by adding to STARTD_ATTRS, rather than STARTD_EXPRS • Remember the use of the meta operators: =?= and =!= Euro Condor Week 2008 Barcelona

6. Environment variables ENVIRONMENT = ROOT=C:\ ARGUMENTS = first_param ENVIRONMENT = ROOT=C:\\ ARGUMENTS = first_param • How do I passROOT=C:\to Windows? ROOT=C:ARGUMENTS = … ROOT=C:\ARGUMENTS = … ENVIRONMENT = ROOT=C:\\\ ARGUMENTS = first_param ENVIRONMENT = ROOT=C:\□ ARGUMENTS = first_param ROOT=C:\\ARGUMENTS = … ROOT=C:ARGUMENTS = … √ ENVIRONMENT = ROOT=C:\\ ARGUMENTS = first_param ENVIRONMENT = ROOT=C:\|DUMMY="" ARGUMENTS = first_param √ ENVIRONMENT = ROOT="C:\" ARGUMENTS = first_param ? Euro Condor Week 2008 Barcelona

7. Environment variables • How about passing more than one environment variable on Windows and/or Linux? • E1=AB E1=AB • □E2=DE □E2=DE • E1=AB E1=AB • E2=DE E2=DE ENVIRONMENT = E1=AB|□E2=DE ENVIRONMENT = E1=AB□|□E2=DE ENVIRONMENT = E1=AB□|E2=DE ENVIRONMENT = E1=AB|E2=DE √ √ Euro Condor Week 2008 Barcelona

8. Environment Variables • What about using newlines for better formatting when passing lots of environment variables? • E1=AB • □E2=DE • E1=AB • E2=DE √ ENVIRONMENT = E1=AB;□\ E2=DE ENVIRONMENT = E1=AB□;\ E2=DE ENVIRONMENT = E1=AB\ ;E2=DE ENVIRONMENT = E1=AB;\ E2=DE Euro Condor Week 2008 Barcelona

9. Environment variables • Remember a trailing backslash will escape the newline, even if you add trailing whitespace or escape it with another backslash, or both. • Be careful about blanks and tabs in the middle of the environment line; they may render your declarations impotent. Probably better to avoid whitespace. OR, even better … Euro Condor Week 2008 Barcelona

10. The "new" syntax • [Paraphrased from condor_submit man pages] • Put double quotes around the entire string. • To insert a double quote, repeat it. • Use whitespace to separate arguments. To include whitespace, surround by single quotes. • To insert a single quote, repeat it inside a single-quoted section. Euro Condor Week 2008 Barcelona

11. How are previous gotchas handled? • The following all work as expected for passing environment variables: • ENVIRONMENT = "ROOT=C:\" • ENVIRONMENT = "E1=AB E2=AC" • ENVIRONMENT = "E1=AB \ • E2=AC" Euro Condor Week 2008 Barcelona

12. What? No Gotchas? • How do I setup an environment variables for a single quote or a double quote? • DQUOTE="\"" DQUOTE='"' • SQUOTE="'" • # in old Syntax • ENVIRONMENT = DQUOTE="\"" ENVIRONMENT = DQUOTE='"' • ENVIRONMENT = SQUOTE="'" Euro Condor Week 2008 Barcelona

13. DQUOTE • # New Syntax • # Trying to produce DQUOTE='"' • ENVIRONMENT = "DQUOTE=''''""''''" • # Trying to produce DQUOTE="\"" • ENVIRONMENT = "DQUOTE=""\""""" Euro Condor Week 2008 Barcelona

14. SQUOTE • # New Syntax • # Trying to produce SQUOTE="'" • ENVIRONMENT = "SQUOTE=""''''""" Euro Condor Week 2008 Barcelona

15. So … • Use the "new syntax" Euro Condor Week 2008 Barcelona

16. Condor + Connectivity • Firewalls are becoming more commonplace; as well as site firewalls, many machines will have their own firewalls. • Many grid resources or local clusters will have their worker nodes on private networks which are not directly addressable. • Is this a problem? Euro Condor Week 2008 Barcelona

17. Central Manager Negotiator Collector Submit Machine Execute Machine Schedd Startd Starter Job Shadow Submit Euro Condor Week 2008 Barcelona Slide based on one from the University of Wisconsin-Madison

18. Condor + Firewalls • All nodes in the pool must be able to "see" the central node(s) • Every submit node must be able to "see" all execute nodes on which its jobs are expected to run and vice versa • ["see" means accessible for the relevant condor fixed ports as well as the ephemeral ports (defined using HIGHPORT and LOWPORT) for TCP and UDP ] Euro Condor Week 2008 Barcelona

19.  Condor + Grids • How is Flocking affected? • Your jobs will not run on remote execute nodes which you cannot "see" or can't "see" you.  • How about Glidein? • Your jobs will not run on remote execute nodes which you cannot "see" or can't "see" you.  • Grid Universe (incl. Condor-C) • Your job is effectively handed over to that grid so firewalls are less of a problem.  Euro Condor Week 2008 Barcelona

20. Firewall solutions • Run the whole pool inside a single (site?) firewall so no problems. • Use a restricted set of submit nodes (possibly with portal front-ends) and open resources for these. • Mirror the firewall settings for each machine in its ClassAds (exercise for the user – but see me for hints) • Use the Grid universe • Use the Generic Connection Broker – see Todd! Euro Condor Week 2008 Barcelona

21. In Summary • If you declare your own attributes – avoid the above Gotchas • Use the "new" syntax for environment and argument values in submit files • Be aware/wary of Firewalls and design your pool accodingly • For further info on Firewall mirroring: • http://tardis.dl.ac.uk/Condor Euro Condor Week 2008 Barcelona