80 likes | 178 Views
Weekly Report LSF API Timing Tests. Summary of the week. Meeting with Tim Bell (IT) about accounting systems, allocation requests
E N D
Weekly ReportLSF API Timing Tests P. Nilsson
Summary of the week • Meeting with Tim Bell (IT) about accounting systems, allocation requests • Possible to use (e.g.) LSF, Condor, Torque-Maui for PROOF accounting without using the full machinery of the batch systems. The PROOF master can then query the “batch system” master for resource information • Bell recommended us to start with LSF. IT can help us set up LSF on the PROOF cluster (1-2 days of work, cluster already under LSF license) • LSF API • How to use it for allocation requests (see next few slides) P. Nilsson
LSF API Example • How to request a number of hosts for a task • Example: Ask for two “suitable” machines with atlas user environment interactively The equivalent API function is ls_placereq() > lsplace -R atlas -n 2 lxb5509 lxb5510 char resreq[MAXLINELEN]; // resource requirement expression int num = 2; sprintf(resreq,"%s","atlas"); char **host = ls_placereq(resreq, &num, 0, NULL); P. Nilsson
Resource Names (Used to build the resource requirement expression) RESOURCE_NAME TYPE ORDER DESCRIPTION r15s Numeric Inc 15-second CPU run queue length r1m Numeric Inc 1-minute CPU run queue length (alias: cpu) r15m Numeric Inc 15-minute CPU run queue length ut Numeric Inc 1-minute CPU utilization (0.0 to 1.0) pg Numeric Inc Paging rate (pages/second) io Numeric Inc Disk IO rate (Kbytes/second) ls Numeric Inc Number of login sessions (alias: login) it Numeric Dec Idle time (minutes) (alias: idle) tmp Numeric Dec Disk space in /tmp (Mbytes) swp Numeric Dec Available swap space (Mbytes) (alias: swap) mem Numeric Dec Available memory (Mbytes) lftm Numeric Dec Minutes until next shutdown pool Numeric Dec Free MB in work directories maxpool Numeric Dec Max size of pool disk cms Boolean N/A CMS DC04 tests atlas Boolean N/A ATLAS CD alice Boolean N/A ALICE MDC … … … … + many more (full list with the lsinfo command) See separate word document P. Nilsson
Timing of request calls • Timing performed with external time command Requesting one host [lxplus006] > time lsplace -R atlas -n 1 lxb5510 0.000u 0.010s 0:00.02 50.0% 0+0k 0+0io 197pf+0w Requesting 20 hosts [lxplus006] > time lsplace -R atlas -n 20 lxb5510 lxb5510 lxb5504 lxb5504 lxb5501 lxb5501 lxb5507 lxb5507 lxb5506 lxb5506 lxb5508 lxb5508 lxb5502 lxb5502 lxb5503 lxb5503 lxb5505 lxb5505 lxb5509 lxb5509 0.000u 0.000s 0:00.02 0.0% 0+0k 0+0io 197pf+0w P. Nilsson
Timing of request calls • Timing performed with Gerri’s TMeasure class Requesting 20 hosts with same resource requirement expression (“atlas”) using the API [lxplus006] > hostreq lxb5510 lxb5510 lxb5504 lxb5504 lxb5503 lxb5503 lxb5507 lxb5507 lxb5502 lxb5502 lxb5506 lxb5506 lxb5509 lxb5509 lxb5501 lxb5501 lxb5505 lxb5505 lxb5508 lxb5508 #-------------------- TMeasure ----------------------# # place: ResourceQuery # # results (seconds): # 0.01 Done requesting #----------------------------------------------------# P. Nilsson
Timing of request calls • Varying the number of requests (same simple requirement resource expression) No of requests Time (s) 1 0.01 100 1.9 500 10.0 1000 18.1 P. Nilsson
Timing of request calls • More complex requirement resource expression resreq = "atlas:type=local:cpu=.1:ls=2:mem=1024“ resreq = atlas && (type == local) && (cpu <= .1) && (ls <= 2) && (mem >= 1024) No of requests Time (s) 1 0.02 100 2.5 500 13.1 1000 25.0 P. Nilsson