1 / 31

Performance 9 ways to fool the public

Performance 9 ways to fool the public. Old Chapter 4 New Chapter 1.4. #1 – Reporting Results. Reporting “ peak ” performance (Capable of 4 Instructions Per Cycle). #1 – Reporting Results. Reporting “ peak ” performance (Capable of 4 Instructions Per Cycle)

georgevance
Download Presentation

Performance 9 ways to fool the public

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Performance9 ways to fool the public Old Chapter 4 New Chapter 1.4

  2. #1 – Reporting Results • Reporting “peak” performance (Capable of 4 Instructions Per Cycle)

  3. #1 – Reporting Results • Reporting “peak” performance (Capable of 4 Instructions Per Cycle) • Solution: Report actual performance on actual programs

  4. #2 - Choosing Applications • Choose programs that run quickly but do not represent real world • That you know run faster on your machine than your competitor’s

  5. #2 - Choosing Applications • Choose programs that run quickly but do not represent real world • Solution: Form a committee that creates benchmarks • Benchmarks should: • Representative of real world • Expose performance aspects of machine

  6. #3 - Choosing Applications • Stuff the benchmark body with company members and do not invite your competitor

  7. #3 - Choosing Applications • Stuff the benchmark body with company members and do not invite your competitor • Intel did this to AMD • Solution: Make sure lots of companies are represented on the committee

  8. Benchmark suites • SPEC-fp – scientific codes • SPEC-int – integer codes (irregular, small codes) • TPC – database benchmarks • EEMBC – embedded proc benchmarks • Many, many others (Java, Networks, media, …)

  9. #4 - Running Applications • Include flags in your compiler that hand-optimize specific benchmarks • Solution:

  10. #4 - Running Applications • Include flags in your compiler that hand-optimize specific benchmarks • Solution: Specify how much optimization is allowed and/or have independent bodies perform tests • http://www.tomshardware.com • http://www.anandtech.com

  11. #5 – Reporting Results • Selectively reporting results • Solution:

  12. #5 – Reporting Results • Selectively reporting results • Apple vs Intel • Solution: Report suite performance, not just individual benchmarks • Speedup of X over Y =

  13. #5 – Reporting Results • Selectively reporting results • Apple vs Intel • Solution: Report suite performance, not just individual benchmarks • Speedup of X over Y = ExecutionTimeY / ExecutionTimeX

  14. Speedup • On P1, A is ____ x’s as fast as B • On P1, B is ____ x’s as fast as C

  15. Speedup • On P1, A is 10 x’s as fast as B • On P1, B is 2 x’s as fast as C

  16. #6 – Floating 0 on graph

  17. #6 – Normalized speedup, 0 to 1… All execution times are divided by North’s execution time to obtain speedup

  18. #7 – Reporting Results • Manipulating average results • Solution:

  19. Speedup

  20. #7 – Reporting Results • Manipulating average results • Solution: Use geometric mean or weighted means rather than arithmetic means (average)

  21. #8 – Reporting Results • Reporting speedup of a small portion of code • Solution:

  22. #8 – Reporting Results • Reporting speedup of a small portion of code • Solution: Use Amdahl’s Law

  23. Amdahl’s Law • Speedup =

  24. Amdahl’s Law • Helps target design effort • Optimize the common case • Allow rare cases to suffer

  25. Amdahl’s Law Example Suppose multiplication operations constitute 80% of the execution of a program. How much improvement do we need in the multiply to get a 3x speedup in the program? What is the most possible speedup by improving only the multiply?

  26. #9 – Misusing alternate metrics • IPC = Instructions per Cycle • CPI = Cycles per Instruction • MIPS = Millions of Instructions per Second

  27. CPU time = • #Insts * Clock cycle time * CPI • #Inst * CPI / Clock rate • (go to board)

  28. Using Execution Time • Our old 1GHz machine runs our program in 20 seconds. We are designing a new machine, and the target execution time is 10 seconds. Through various architectural innovations, we can increase clock rate. Unfortunately, this increase comes at a price – we can pump the clock rate up to 2GHz, but the new execution takes 1.2 times as many cycles. Is this enough? What is the clock rate necessary for our performance target?

  29. #9 – Misusing alternate metrics • IPC = Instructions per Cycle • CPI = Cycles per Instruction • MIPS = Millions of Instructions per Second • Clock Rate = Cycles / Second • Solution: Only use alternate metrics when other elements of execute time equation are equal.

  30. Using CPI We have two code sequences involving recalculation of the same values. The compiler writer can choose between having a code sequence of 7 instructions to recalculate three values, or temporarily storing them in the stack. What are the total cycles of the code sequences? Which should the compiler do?

  31. Machine performance characteristics: Program characterstics

More Related