Condor Gotchas II

1. Condor Gotchas II (this time it's cynical!) • John Kewley • Daresbury Laboratory e-Science Centre Condor Week 2007, Madison WI

2. What is a "gotcha" • This presentation was originally called "Hints and Tips", but I think the term "Gotchas" (aka pitfalls) is more appropriate. • From Wikipedia: "A 'gotcha' is a feature of a system, a program or a programming language that works in the way it is documented, but is counter-intuitive and almost invites mistakes because it is both enticingly easy to invoke and completely unexpected and/or unreasonable in its outcome." Condor Week 2007, Madison WI

3. Outline • Some "gotchas", mostly related to submit files • Custom attributes • Environment variables (and whitespace) • The new syntax – is it a panacea? • If time, also a tip for formatting with condor_status Condor Week 2007, Madison WI

4. Custom attributes • How do I declare my own attributes so my jobs can distinguish between Red Hat 9 and other Linuxes? • On Red Hat 9 machines add the following to its configuration: OPSYS_FLAVOUR = RH9 OPSYS_FLAVOUR = "RH9" STARTD_EXPRS = OPSYS_FLAVOUR STARTD_ATTRS = OPSYS_FLAVOUR • It can then be matched with a REQUIREMENTSstatement: REQUIREMENTS = (OPSYS_FLAVOUR = RH9) REQUIREMENTS = (OPSYS_FLAVOUR == RH9) REQUIREMENTS = (OPSYS_FLAVOUR == "RH9") REQUIREMENTS = (OPSYS_FLAVOUR =?= "RH9") √ Condor Week 2007, Madison WI

5. Custom Attributes • Declare the attribute and match against it • Always quote non-numerics • “Export” the attribute by adding to STARTD_ATTRS, rather than STARTD_EXPRS • Remember the use of the meta operators: =?= and =!= Condor Week 2007, Madison WI

6. Environment variables ENVIRONMENT = ROOT=C:\ ARGUMENTS = first_param ENVIRONMENT = ROOT=C:\\ ARGUMENTS = first_param • How do I passROOT=C:\to Windows? ROOT=C:ARGUMENTS = … ROOT=C:\ARGUMENTS = … ENVIRONMENT = ROOT=C:\\\ ARGUMENTS = first_param ENVIRONMENT = ROOT=C:\□ ARGUMENTS = first_param ROOT=C:\\ARGUMENTS = … ROOT=C:ARGUMENTS = … √ ENVIRONMENT = ROOT=C:\\ ARGUMENTS = first_param ENVIRONMENT = ROOT=C:\|DUMMY="" ARGUMENTS = first_param √ ENVIRONMENT = ROOT="C:\" ARGUMENTS = first_param ? Condor Week 2007, Madison WI

7. Environment variables CondorVersion 6.6 • How about passing more than one environment variable on Windows and/or Linux? • E1=AB E1=AB • □E2=DE □E2=DE • E1=AB E1=AB • E2=DE E2=DE ENVIRONMENT = E1=AB|□E2=DE ENVIRONMENT = E1=AB□|□E2=DE ENVIRONMENT = E1=AB□|E2=DE ENVIRONMENT = E1=AB|E2=DE √ √ Condor Week 2007, Madison WI

8. Environment Variables • What about using newlines for better formatting when passing lots of environment variables? • E1=AB • □E2=DE • E1=AB • E2=DE √ ENVIRONMENT = E1=AB;□\ E2=DE ENVIRONMENT = E1=AB□;\ E2=DE ENVIRONMENT = E1=AB\ ;E2=DE ENVIRONMENT = E1=AB;\ E2=DE Condor Week 2007, Madison WI

9. Environment variables • Remember a trailing backslash will escape the newline, even if you add a trailing space or escape it with another backslash, or both. You must escape it and have a blank line afterwards. • Be careful about blanks and tabs in the middle of the environment line, they may render your declarations impotent. Probably better to avoid whitespace. Condor Week 2007, Madison WI

10. The "new" syntax • Put double quotes around the entire string. • Use whitespace to separate arguments. To include whitespace, surround by single quotes. • To insert a double quote, repeat it. • To insert a single quote, repeat it inside a single-quoted section. Condor Week 2007, Madison WI

11. How are previous gotchas handled? • The following all work as expected for passing environment variables: • ENVIRONMENT = "ROOT=C:\" • ENVIRONMENT = "E1=AB E2=AC" • ENVIRONMENT = "E1=AB \ • E2=AC"  Condor Week 2007, Madison WI

12. What? No Gotchas? • How do I setup an environment variables for a single quote or a double quote? • SQUOTE="'" • DQUOTE="\"" DQUOTE='"' • # in old Syntax • ENVIRONMENT = SQUOTE="'" • ENVIRONMENT = DQUOTE="\"" ENVIRONMENT = DQUOTE='"' Condor Week 2007, Madison WI

13. SQUOTE • # New Syntax • # Trying to produce SQUOTE="'" • ENVIRONMENT = "SQUOTE=""''''""" Condor Week 2007, Madison WI

14. DQUOTE • # New Syntax • # Trying to produce DQUOTE='"' • ENVIRONMENT = "DQUOTE=''''""''''" • # Trying to produce DQUOTE="\"" • ENVIRONMENT = "DQUOTE=""\""""" Condor Week 2007, Madison WI

15. In Summary • Good writing is all about spelling, grammar and punctuation. The same is true with condor configuration and submit files. • Use the new arguments/environment syntax! Condor Week 2007, Madison WI

16. Tip for condor_status • $ condor_status -f "%-6s" Arch -f "%-7s" OpSys \ • -f " %-12s\n" OPSYS_FLAVOUR | sort | uniq -c • 1 • 1 INTEL LINUX Gentoo • 1 INTEL LINUX Mandrake10 • 2 INTEL LINUX RH80 • 3 INTEL LINUX RH9 • 2 INTEL LINUX SUSE80 • 6 INTEL LINUX SUSE90 • 5 INTEL LINUX WBL • 1 INTEL WINNT40INTEL WINNT50INTEL WINNT51 Condor Week 2007, Madison WI

17. Tip for condor_status • $ condor_status -f "%-6s" Arch -f "%-7s" OpSys \ • -f " %-12s" OPSYS_FLAVOUR -f "\n" OpSys | sort | uniq -c • 1 • 1 INTEL LINUX Gentoo • 1 INTEL LINUX Mandrake10 • 2 INTEL LINUX RH80 • 3 INTEL LINUX RH9 • 2 INTEL LINUX SUSE80 • 6 INTEL LINUX SUSE90 • 5 INTEL LINUX WBL • 1 INTEL WINNT40 • 1 INTEL WINNT50 • 1 INTEL WINNT51 Condor Week 2007, Madison WI