1 / 12

MisSPECulation

MisSPECulation. Partial and Misleading use of SPEC CPU2000 in Computer Architecture Conferences Dave Kroondyk. Abstract. 173 papers surveyed 115 used benchmarks from SPEC CINT 23 used entire suite Results are speculative Violation of Amdahl’s Law Unwarranted disregard for CFP2000

chipo
Download Presentation

MisSPECulation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MisSPECulation Partial and Misleading use of SPEC CPU2000 in Computer Architecture Conferences Dave Kroondyk

  2. Abstract • 173 papers surveyed • 115 used benchmarks from SPEC CINT • 23 used entire suite • Results are speculative • Violation of Amdahl’s Law • Unwarranted disregard for CFP2000 • Higher data cache miss rate

  3. SPEC CPU2000 • 2 suites • CINT2000 for integer applications (12) • CFP2000 for floating point applications (14)

  4. ISCA 2002 • 27 papers published • 23 used SPEC • 4 of 16 used all of CINT • 2 of 14 used all of CFP

  5. CINT2000 benchmark usage • Median = 7

  6. Why?

  7. Why? (cont.) • Chosen benchmarks stress problem paper solves • Violation of Amdahl’s Law • Several benchmarks couldn’t be adapted to the environment used • Understandable unless more than half can’t be run

  8. Amdahl’s Law • “The performance improvement to be gained from using some faster mode of execution is limited by the fraction of the time the faster mode can be used”

  9. Applying Amdahl’s Law

  10. CFP2000 • Used less and less benchmarks are used per paper • 36% of suite vs. 58% for CINT • Why? • FP applications are built around highly predictable and parallelizable loops • Less branches are performed • Higher prediction rate • Instruction cache miss rate is lower

  11. However… • Data cache miss rate is much higher • Suggests that memory-hierarchy oriented research should put emphasis on improving CFP2000 benchmarks • But they don’t • 35% use CFP2000 • 61% use CINT2000

  12. Summary • Misuse of tools leads to speculative and misleading results • Applying Amdahl’s law “steals their thunder” • CFP2000 has higher data cache miss rates • However, CINT2000 is used nearly twice as much as CPF2000 for memory-hierarchy oriented papers

More Related