Tutorial: Condor on Windows

1. Tutorial:Condor on Windows

2. Overview • Simple job submission • Accessing network shares • Handling DLL dependencies • Checkpointing in Vanilla universe • Submitting interpreted scripts

3. Overview • Simple job submission • Accessing network shares • Handling DLL dependencies • Checkpointing in Vanilla universe • Submitting interpreted scripts

4. Simple Job Submission

5. condor_store_cred • Contacts local SchedD and asks it to securely store a user’s password • Password is placed encrypted in a registry location C:\>condor_store_cred add Account: gquinn@CROW Enter password: Operation succeeded. myp4sswd y0urs

6. Job Execution: Submit Side SchedD Secure Password Store Submit myp4sswd y0urs Submit Shadow

7. Job Execution: Execute Side Starter Jobs run using a Condor-specific account with minimal privileges. Job condor-reuse-slot1

8. Overview • Simple job submission • Accessing network shares • Handling DLL dependencies • Checkpointing in Vanilla universe • Submitting interpreted scripts

9. Accessing Files on a Network Share • The condor-reuse-slotX accounts are local accounts without network privileges • To access files on a network share, the job needs to run using an account that has access to the shared network resources • Using a CredD, its possible to have jobs run with the credentials of the submitting user

10. The Windows CredD • A centralized repository for user passwords C:\>condor_store_cred add Account: gquinn@CROW Enter password: Operation succeeded. myp4sswd y0urs “store password” CredD <password>

11. The Windows CredD SchedD myp4sswd “fetch password” y0urs <password> Shadow Submit machines can use the CredD to impersonate the user in the shadow

12. The Windows CredD Starter “fetch password” myp4sswd y0urs <password> Job Execute machines can use the CredD to run jobs as the desired user

13. Securing the CredD • NTSSPI can be used to authenticate to CredD and send the password encrypted over the network C:\>condor_store_cred add Account: gquinn@CROW Enter password: Operation succeeded. myp4sswd y0urs CredD “store password”

14. NTSSPI Authentication • Uses Windows built-in authentication capability • Used for user-to-daemon authentication • Always works when client and server are on the same machine • Can also be used over the network if the user account exists on both the client and server and the passwords match

15. Securing the CredD Starter myp4sswd “fetch password” y0urs Job Condor normally runs as SYSTEM, and therefore can’t use NTSSPI

16. Securing the CredD • Options for securing password fetch operations • Kerberos • SSL • Password

17. Password Authentication • Daemon-to-daemon communication is secured using a shared secret (the “pool password”) • Pool password must be distributed to all machines in the pool before enabling • Pool password is stored using the “-c” option to condor_store_cred • Password can be set remotely using “-n” option (make sure connection is authenticated!)

18. Setting the Pool Password • Enable CONFIG access from the machine that you will run condor_store_cred: • If setting the password remotely, ensure the user account that runs the condor_store_cred command exists on every machine and has a common password • Set the pool password on every machine: HOSTALLOW_CONFIG = $(HOSTALLOW_ADMINISTRATOR) C:\> condor_store_cred add –c –n node01.cs.wisc.edu

19. Configuring the CredD Host • Once pool password has been distributed, the CredD can be configured: # Set security settings so that full security to the credd is required CREDD.SEC_DEFAULT_AUTHENTICATION = REQUIRED CREDD.SEC_DEFAULT_ENCRYPTION = REQUIRED CREDD.SEC_DEFAULT_INTEGRITY = REQUIRED CREDD.SEC_DEFAULT_NEGOTIATION = REQUIRED # Require NTSSPI auth for password storing CREDD.SEC_WRITE_AUTHENTICATION_METHODS = NTSSPI # Require PASSWORD auth for password fetching CREDD.SEC_DAEMON_AUTHENTICATION_METHODS = PASSWORD

20. Configuring the CredD Host • Use the condor_config.local.credd example file: • Include this file as one of the local configuration files: C:\Condor> copy etc\condor_config.local.credd .\ LOCAL_CONFIG_FILE = $(LOCAL_DIR)/condor_config.local \ $(LOCAL_DIR)/condor_config.local.credd

21. Running Jobs as Submitting User • Submit and execute machines need to be configured to use the CredD: CREDD_HOST = vault.cs.wisc.edu STARTER_ALLOW_RUNAS_OWNER = True SEC_CLIENT_AUTHENTICATION_METHODS = \ NTSSPI, PASSWORD CREDD_CACHE_LOCALLY = True

22. Running Jobs as Submitting User • Example submit file: universe = vanilla executable = whoami.exe log = whoami.log output = whoami.out run_as_owner = true queue

23. Overview • Simple job submission • Accessing network shares • Handling DLL dependencies • Checkpointing in Vanilla universe • Submitting interpreted scripts

24. Jobs with DLL Dependencies

25. Jobs with DLL Dependencies • Two ways to deal with DLL dependencies: • Ship the DLLs to the execute machine with the job • Make sure the job runs on an execute machines with the needed DLLs, and knows where to find them

26. Shipping DLLs with the Job universe = vanilla initial_dir = c:\cygwin executable = sleep.exe log = sleep.log transfer_input_files = cygwin1.dll,cygintl-8.dll,cygiconv-2.dll queue

27. Using DLLs on the Execute Side • Windows searches directories in the PATH environment variable for needed DLLs C:\> set PATH=%PATH%;c:\cygwin\bin universe = vanilla executable = c:\cygwin\sleep.exe log = sleep.log getenv = true queue

28. Overview • Simple job submission • Accessing network shares • Handling DLL dependencies • Checkpointing in Vanilla universe • Submitting interpreted scripts

29. Checkpointing in Vanilla Universe • No Standard universe on Windows • Some jobs can do their own checkpointing • Condor will “warn” your job before eviction by sending the WM_CLOSE window message • For console applications, this translates to a CTRL_CLOSE_EVENT

30. Checkpointing in Vanilla Universe • Catching a “eviction notice” in a C-based console app: • Need to let Condor know that partial results are valuable in the submit file: SetConsoleCtrlHandler(ckpt_function, TRUE) when_to_transfer_output = ON_EXIT_OR_EVICT

31. Overview • Simple job submission • Accessing network shares • Handling DLL dependencies • Checkpointing in Vanilla universe • Submitting interpreted scripts

32. Submitting Scripts • Can just submit the interpreter as the executable and the script as an input file executable = c:\perl\bin\perl.exe arguments = script.pl args for script transfer_executable = false

33. Submitting Scripts • If UNIX-style “#!” behavior is desired, use a USER_JOB_WRAPPER • Useful for using a single submit file that works on all platforms executable = script.pl arguments = args for script

34. USER_JOB_WRAPPER # open the actual executable so we can look for interpreters # if (open(EXECUTABLE, $ARGV[0])) { # see if the first two bytes are #! # my $two_bytes; if ((read(EXECUTABLE, $two_bytes, 2) == 2) && ($two_bytes eq "#!")) { # look for the /usr/bin/perl interpreter # if (<EXECUTABLE> =~ /\/usr\/bin\/perl/) { unshift(@ARGV, "c:\\cygwin\\bin\\perl.exe"); } } } system(@ARGV);

35. Questions? • Windows BOF: • Thursday, 11:00 – 12:00 • Room 221