Nikhef Multicore Observations

1. Nikhef Multicore Observations Jeff Templon Multicore TF #2 2014.01.28

2. Data set • Jobs running on 26 and 27 january Jeff Templon, Multicore TF #2

3. Waiting time Jeff Templon, Multicore TF #2

4. Short pilots • Submission to short queue (4 hrs) helped multicore throughput a lot Jeff Templon, Multicore TF #2

5. Production pilots • Still could win some ground here with walltime limits ATLAS prod submits normally to ‘atlas’ or ‘medium’ queues: 52 and 38 hours resp. Jeff Templon, Multicore TF #2

6. Multicore memory Jeff Templon, Multicore TF #2

7. Memory and maui • Maui checks mem; recall we use pvmem • Maui assumes each proc needs ‘pvmem’ • Submit filter assumes “multicore” : 8 cores = 8 times mem … • Hence ATLAS was “asking for” 256 GB per job!!! • Changed by hand to 32 GB per job; seems ok (jobs consume 16) Jeff Templon, Multicore TF #2

8. Maui & Memory II • My preferred solution: • Ask for nr cores: assumes independent cores • Hence pvmem = 4096mb • Need true multicore? Ask for mem too • But then Maui will over-ask because it does not understand multicore • “Not a problem” at moment? All ATLAS jobs use the indep. Core model Jeff Templon, Multicore TF #2

9. Holding onto cores Jeff Templon, Multicore TF #2

10. Holding onto cores • ATLAS doesn’t • Normal fair share fluctuations -> lose cores • Probably need some dynamic partitioning to “multicore” segment. Dynamic means others can use it for single-core work when ATLAS workload disappears. Jeff Templon, Multicore TF #2