1 / 23

Benchmarking, Performance Evaluation, Modeling and Prediction

Benchmarking, Performance Evaluation, Modeling and Prediction. Erich Strohmaier. ICL Benchmarking Activities. Linpack benchmark ParkBench LLCbench The Performance Database Server TOP500 Performance Analysis Tool for NAS. Linpack Benchmark. 3 main ‘flavors 100x100 1000x1000 NxN

sydney
Download Presentation

Benchmarking, Performance Evaluation, Modeling and Prediction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Benchmarking, Performance Evaluation, Modeling and Prediction Erich Strohmaier

  2. ICL Benchmarking Activities • Linpack benchmark • ParkBench • LLCbench • The Performance Database Server • TOP500 • Performance Analysis Tool for NAS

  3. Linpack Benchmark • 3 main ‘flavors • 100x100 • 1000x1000 • NxN • R max from NxN used in the TOP500

  4. ParkBench • Collection of • LowLevel benchmarks • sequential • MPI • PVM • Kernels • Linear Algebra • NAS PB Kernels • Applications • NAS PB (simulated applications) • PSTSWM

  5. LLCbench • Low-Level Tests for various hardware aspects: • MPBench • Core MPI contructs • BLASBench • BLAS routines • CacheBench • In and out of cache data movement

  6. Performance Database Server • Searchable Interface to DB of benchmark results such as: • Linpack • SPEC • (Dhrystone etc)

  7. Basis for analysing the HCP market - Quantification of observations - Detection of trends • market, • architecture, • technology

  8. TOP500 Procedure • Listing of the 500 most powerful • Computers in the World • Yardstick: Rmax/LINPACK • Updating halfyearly

  9. TOP500 list - Data shown • Manufacturer Manufacturer or vendor • Computer Type indicated by manufacturer or vendor • Inst. Site Customer • Location Location and country • Year Year of installation/last major update • Field of Appl. Academic,Research,Industry,Vendor,Class. • # Proc. Number of processors • Rmax Maxmimal LINPACK performance achieved • Rpeak Theoretical peak performance • Nmax Problemsize for achieving Rmax • N1/2 Problemsize for achieving half of Rmax • Nworld Position within the TOP500 ranking

  10. TOP10

  11. Performance Development

  12. Manufacturer

  13. Architectures

  14. Performance Development

  15. Performance Development

  16. Performance Development

  17. www.top500.org.

  18. NAS Performance Analysis Tool • To provide a tool to NAS users which allows • Access to performance data of NAS reference codes • Performing a variety of predefined performance tests on target systems in an easy and comparable ways • Analyzing user performance measurements using advanced statistical methods • Analyze scalability of his/her application • Compare its efficiency to other codes • Get a first prediction of attainable performance ranges on other NAS systems

  19. Necessary Functionality • Broad variety of predefined performance tests • Simple interface do define custom performance tests • Reference measurement of NAS applications • Generation and maintenance of performance database

  20. Necessary Functionality • Black Box performance analysis for individual codes • Cross-code and cross-platform performance analysis. • Tool for performance prediction for NAS applications • Detection and analysis of system and code performance signatures

  21. System Architecture Test definition database - All predefined performance test with performance models GUI: - Test definition - Test suite setup Test suite setup file Benchmark Skeleton codes Create script - User data entry Test benchmarks Execute script - Result display - Result analysis using various methods Test results - Result commit to include in DB Performance DB Connected by same keys

  22. Functions available for non-linar regression GUI - Analysis Component Input of measured data as: #proz time Data are NPB MG Class A on Steger Statistical output of analysis - best model Data and regression display

More Related