Moving out of SI2K

1. Moving out of SI2K How INFN is moving out of SI2K as a benchmark for Worker Nodes performance evaluation Michele Michelotto at pd.infn.it

2. INFN Problem • INFN needs to procure computing power for the CNAF Tier1 and for several Tier2 • Which benchmark should we use to specify our requirements? • Since at least Hepix Fall 2006 is clear the SI2000 is close to meaningless • INFN started a working group in charge of proposing a solution michele michelotto - INFN

3. What it is SI2K • SI2K is actually a nickname for SPECCPUInt2000 benchmark • The geometric mean of a dozen of benchmark which stress the integer performance of a cpu • Delivered by SPEC in 2000, it came after Spec89, Spec Int 92 and Spec Int 95 • Declared obsolete by SPEC in 2006 • Replaced by CPUInt2006 michele michelotto - INFN

4. Why is it important for us? • SI2K is the benchmark used up to now to measure the computing power of all the HEP experiments • Computing power requested by experiment, e.g. in the Computing TDR • Computing power provided by a Lab or a Tier-x • Computing power a funding agency is committed to pay for • Well, you have seen it yourself this week. Several site reported the Kilo or Mega SI2K available at their sites (other talked only of cores available) michele michelotto - INFN

5. Transition phase • Impossible to find SPEC Int 2000 pubblished results for the new processors • (e.g. the not so new intel Clovertown 4core) • Impossible to find pubblished SPEC Int 2006 for old processor (before 2006) • E.g. Old P4 Xeon, P4, AMD 2xx • There is no conversion constant from SI2000 to SI2006 • the ratio for x86 architectures is in the 137 – 172 range michele michelotto - INFN

6. SI2K/SI2K6 ratio michele michelotto - INFN

7. The SI2K inflaction • The main problem with SI2000 in our community is that it not (any more) proportional to HEP codes performance • You can buy new processors with huge SI2K number but you gain only a smaller increase in real performances • The face value is not the real value michele michelotto - INFN

8. Nominal SI vs real SI • SI2K results for the last generation processor affected by inflation • So CERN (and FZK) started to use a new currency: SI2K measured with “gcc”, the gnu C compiler and using two flavour of optimization • High tuning: gcc –O3 –funroll-loops–march=$ARCH • Low tuning: gcc –O2 –fPIC –pthread michele michelotto - INFN

9. Nominal SI vs real SI • CERN Proposal: Use as site rating the “Real SI” obtained by SI measured with gcc-low and increased by 50% • This make sense but only for a short period of time and for the last generation of processor (2006-2007) • Actually is has been proposed as a short term solution • Run n copies in parallel • Where n is the number of cores in the worker node • To take in account the drop in performance of a multicore machine when fully loaded. • Preferred to the use of Spec INT 2000 Rate which had syncronized access to memory, unlike our batch farm worker nodes michele michelotto - INFN

10. Too many SI2K • An example: how to evalutate a worker node with two Intel Woodcrest dual core 5160 at 3.06 GHz ? • SI2K (1 core): 2929 – 3089 (min – max) • SI2K on 4 cores: 11716 - 12536 • SI2K on 4 cores gcc-low : 5523 • SI2K on 4 cores gcc-high: 7034 • SI2K on 4 cores gcc-low + 50%: 8284 • SI2K on 4 cores gcc-high + 50%: 10551 michele michelotto - INFN

11. Even more • Actually all the gcc results in the previous slide are on i386 (32bit) • if you would like to know how your code is running on 64 bit machine, you can measure Spec INT 2000 with gcc on x86_64 • So the worker node with two Intel Woodcrest dual core 5160 at 3.06 GHz • SI2K nominal: 2929 – 3089 (min – max) • SI2K on 4 cores: 11716 - 12536 • SI2K on 4 cores gcc-low: 6021 • SI2K on 4 cores gcc-high: 6409 • SI2K pn 4 cores gcc-low + 50%: 9031 michele michelotto - INFN

12. A scale factor • All these numbers would be only annoying in a world with a unique architecture, in which only clock improves in time • You would be able to find a fixed ratio between all those number. • But in the real world, the ratio depends on both the CPU producer (intel vs AMD) and processor generation (old xeon vs new “core” Xeon ) michele michelotto - INFN

13. The nominal SI2K Big Differences between Intel and AMD when SI2K/GHz are plotted michele michelotto - INFN

14. The nominal SI2Krate SI2K RATE differentiate the dual core (TOP) from the 4 core (bottom) Smaller differences between Intel and AMD using SI2K Rate /GHz / core michele michelotto - INFN

15. The “CERN” SI • The intel-amd gap is now smaller • Intel Xeon 5xxx scores -50% wrt nominal • Amd Opteron scores only -29% • Big differences between 32 and 64 bit on same core • SI2K/core/Ghz derived by M.Alef page on the HEPIX site at Caspur michele michelotto - INFN

16. SI2K6rate michele michelotto - INFN

17. Which is the better? • At INFN we started to measure performances of HEP codes on several machines • Our goal was to find a “commercial” mantained benchmark to replace SI2K • We compared HEP code with • SI2K pubblished results • SI2K CERN (gcc low) • SI2006 and SI2006 rate pubblished results • SI2006 and SI2006 with gcc4 (32 and 64 bit) michele michelotto - INFN

18. Babar TierA Results • If you normalize by core and clock all new processors have the same performance • Doubling the older generation cpu • SI2006 matches this pattern (pubblished and gcc ratio constant) • SI2000-cern better than SI2K nominal • SI2000 clearly does’nt work michele michelotto - INFN

19. CMS MC SIM and Pythia • CMS Montecarlo simulation (32bit) and Pythia (64bit) show the same performance once normalized • Both Specint 2006 pubblished and Specint 2006 with gcc show the same behaviour • SI2K pubbished does not match HEP sw • SI2K cern better but not as good as SI2006 michele michelotto - INFN

20. Atlas • Here 100% is Xeon5160 • Few results for SI2006+gcc but no diff from CMS and babar • Few results also from SI2006 pubblished because of several old architectures • SI2K+gcc not bad • SI2K pubblished heavily overstimate new Xeon • Atlas simulation normalized performs the same on the new intel “core” or amd “opteron” (confirm the behaviour seen with CMS, Pithya, Babar) michele michelotto - INFN

21. What INFN should be do? • The Working Group proposed to infn: • Stop using SI2000 pubblished because it gives meaningless number • SI2k CERN is good as a temporary solution but it is still using a 7 years old benchmark that is not mantained by SPEC (v1.3) • You can use it in a tender this year • We should move to a benchmark of the SPEC CPU 2006 suite • Best long term solution michele michelotto - INFN

22. SI2006 Rate Best candidate is Spec INT 2006RATEmeasured withgcc • Spec INT 2006 because it’s new, you can find pubblished results to compare, will have bug fixes, and will be portable to new platform and new compiler • RATE because you can take in account of scalability issues with the whole machine architecture • Measured with GCC: to keep the environment as close as possible as the experiments. michele michelotto - INFN

23. What is missing • All the official number are still expressed in terms of SI2K (nominal) • So funding agencies are still bounded to use SI2K for all formal agreements • A general agreements on the next benchmark is still missing but work is in progress • Example: use gcc4.1 with CERN “low” tuning and 32bit • NB SI2006 rate with gcc 64 bit too big for my machines. It needs more than 1 GB per core michele michelotto - INFN

24. Thanks • The working group: A. Crescente, L. Dell’Agnello, D. Salomoni, F. Rosso, A. Sartirana, R.Stroili, D.Salomoni, M.Morandin, A.De Salvo, T.Zani • Thanks to CCR: the INFN Computing Committee • Thanks for input from LCG MB michele michelotto - INFN