1 / 14

PAPI for Blue Gene/Q: The 5 BGPM Components

PAPI for Blue Gene/Q: The 5 BGPM Components. Heike Jagode and Shirley Moore Innovative Computing Laboratory University of Tennessee- Knoxville http:// icl.eecs.utk.edu/papi /. ESP Code for “Q” Workshop Argonne National Laboratory March 19-21, 2012. Overview. Introduction

aliza
Download Presentation

PAPI for Blue Gene/Q: The 5 BGPM Components

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PAPI for Blue Gene/Q:The 5 BGPM Components Heike Jagode and Shirley Moore Innovative Computing Laboratory University of Tennessee-Knoxville http://icl.eecs.utk.edu/papi/ ESP Code for “Q” Workshop Argonne National Laboratory March 19-21, 2012

  2. Overview • Introduction • Processor Unit (PUnit) Component • L2 Unit Component • I/O Unit Component • Network Component • Compute Node Kernel Unit (CNKUnit) Component

  3. Introduction • Very little effort was put into hardware performance monitoring tools for the BG/Q predecessor BG/P. • HPC community was left behind with rather poor and incomplete methods. • To eliminate this limitation, for BG/Q we planned carefully and we collaborate closely with IBM’s Performance Group. Result: • Added 5 new components to PAPI to support hardware performance monitoring for the BG/Q network, the I/O system, and the Compute Node Kernel in addition to the processing cores

  4. PUnit Component • Each of the 18 A2 CPU cores has a local UPC module. • Each of these modules provides 24 counters (14-bit) to sample A2 events, L1 cache related events, floating point operations, etc. • Local UPC module is broken down into 5 internal sub-modules: FU, XU, IU, LSU and MMU. • The sub-modules are transparently identifiable from the PUnit event names (see next slide for examples). • The BGPM PUnitinterfaces with these modules. • PAPI uses the BGPM interface.

  5. PUnit Events (Native | Presets) • Currently, there are 269 native PUnitevents available: • Out of 107 possible PAPI predefined events, there are currently 41 events available of which 12 are derived events:

  6. L2 Unit Component • Shared L2 cache is split into 16 separate slices • Each of the 16 L2 memory slices (per chip) has a L2 UPC module that provides 6 counters (node-wide)

  7. L2 Unit Native Events • Currently, there are 32 L2 Unit events available: • BG/Q processor has two DDR3 memory controllers, each interfacing with eight slices of the L2 cache to handle their cache misses (one controller for each half of the 16 cores on the chip). • The counting hardware can either keep the counts from each slice separate, or combine the counts from each slice into single values (which is the default).

  8. I/O Unit Component • The Message, PCIe, and DevBusmodules – which are collectively referred to as I/O modules – provide together 43 counters (node-wide)

  9. I/O Unit Native Events • Currently, there are 44 I/O Unit events available. • The three I/O sub-modules are transparently identifiable from the I/O Unit event names.

  10. Network Unit Component • The 5D-Torus network provides a local UPC network module with 66 counters - each of the 11 links has six 64-bitcounters. • As of right now, a PAPI user cannot select which network link to which to attach. • Currently, all network links are attached and this is hard-coded in the PAPI NWUnitcomponent. The BGPM NWUnit interfaces with the network modules.  PAPI Network Unit Component interfaces with BGPM.

  11. Network Unit Native Events • Currently, there are 31 Network Unit events available

  12. CNK Unit Component • CNK is the lightweight Compute Node Kernel that runs on all the 16 compute cores. • BGPM offers a “virtual” CNK Unit that has software counters collected by the kernel(kernel counter values are read via a system call). • Currently, there are 29 CNK Unit events available.

  13. Overflow and Multiplexing Overflow: • Only the local UPC module, L2 and I/O UPC hardware support performance monitor interrupts when a programmed counter overflows • For that reason, only the PUnit, L2Unit, and I/OUnit provide overflow support in BGPM and PAPI. Multiplexing: • PAPI supports multiplexing for the BG/Q platform. • The BGPM PUnitdoes not directly implement multiplexing of event sets, but it does indirectly support multiplexing by supporting a multiplexed event set type.

  14. PAPI on VEAS • Installed in /soft/libraries/papi-4.1.3.0-bgq-beta-4/ • Utilities in bin directory • papi_avail, papi_native_avail, etc. • Run on compute node using qsub – e.g., qsub-n 1 --mode c1 -t 10 papi_avail • Man pages in share directory • MANPATH=“/soft/libraries/papi-4.1.3.0-pgq-beta-4/share/man:$MANPATH” • Examples in /home/shirley/papi_beta4 • cd /home/shirley • cp –R papi_beta4 ~ • cd papi_beta4/src/ctests • Run on compute node using qsub – e.g., qsub –n 1 –mode c1 –t 10 ./matrix-hl

More Related