460 likes | 573 Views
Compiling for VLIWs and ILP. Profiling Region formation Acyclic scheduling Cyclic scheduling. Profiling. Many crucial ILP optimizations require good profile information ILP optimizations try to maximize performance/price by increasing the IPC
E N D
Compiling for VLIWs and ILP • Profiling • Region formation • Acyclic scheduling • Cyclic scheduling
Profiling • Many crucial ILP optimizations require good profile information • ILP optimizations try to maximize performance/price by increasing the IPC • Compiler techniques are needed to expose and enhance ILP • Two types of profiles: point profiles and path profiles
Point Profiles • “Point profiles” collect statistics about points in call graphs and control flow graphs • gprof produces call graph profiles, statistics on how many times a function was called, who called it, and (sometimes) how much time was spent in that function • Control flow graph profiles give statistics on nodes (node profiles) and edges (edge profiles)
Path Profiles B1 • “Path profiles” measure the execution frequency of a sequence of basic blocks on a path a CFG • A “hot path” is a path that is (very) frequently executed • Types include forward paths (no backedges), bounded-length paths (start/stop points), and whole-program paths (interprocedural) • The choice is a tradeoff between accuracy and efficiency to collect the profile B2 B3 B4 B5 B6 B7 Path 1 {B1, B2, B3, B5, B7} count = 7Path 2 {B1, B2, B3, B6, B7} count = 9Path 3 {B1, B2, B4, B6, B7} count = 123
Profile Collection • Data collected through code instrumentation is very detailed, but instrumentation overhead affects execution • Hardware counters have very low overhead but information is not exhaustive • Interrupt-based sampling examines machine state in intervals • Collecting path profiles requires enumerating the set of paths encountered during runtime Instrumentation insertsinstructions to recordedge profiling events
Profile Bookkeeping • Problem: compiler optimization modifies (instrumented) code in ways that change the use and applicability of profile information for later compilation stages • Apply profiling right before profile data is needed • Axiom of profile uniformity: “When one copies a chunk of a program, one should equally divide the profile frequency of the original chunk among the copies.” • Use this axiom for point profiles as a simple heuristic • Path profiles correlate branches and thus path-based compiler optimizations preserve these profiles
Instruction Scheduling • Instruction scheduling is the most fundamental ILP-oriented compilation phase • Responsible for identifying and grouping operations that can be executed in parallel • Two approaches: • Cyclic schedulers operate on loops to exploit ILP in (tight) loop nests usually without control flow • Acyclic schedulers consider loop-free regions Region shape Acyclic Cyclic Basicblock Super-block Trace DAG
Acyclic Scheduling of Basic Block Region Shapes • Region is restricted to single basic block • Local scheduling of instructions in a single basic block is simple • ILP is exposed by bundling operations into VLIW instructions (instruction formation or instruction compaction) add $r13 = $r3, $r0shl $r13 = $r13, 3ld.w $r14 = 0[$r4] sub $r16 = $r6, 3shr $r15 = $r15, 9 add $r13 = $r3, $r0sub $r16 = $r6, 3;; ## end of 1st instr. shl $r13 = $r13, 3shr $r15 = $r15, 9ld.w $r14 = 0[$r4];; ## end of 2nd instr. bundle bundle
Intermezzo: VLIW Encoding • A VLIW schedule can be encoded compactly using horizontal and vertical nops • Start bits, stop bits, or instruction templates are used to compress the VLIW instructions into variable-width instruction bundles add $r13 = $r3, $r0sub $r16 = $r6, 3;; ## end of 1st instr. shl $r13 = $r13, 3shr $r15 = $r15, 9ld.w $r14 = 0[$r4];; ## end of 2nd instr.
Intermezzo: VLIW Execution Model Subtleties • Horizontal issues within an instruction: • A read sees the original value of a register • A read sees the value written by the write • Read and write to same register is illegal • Also exception issues • Vertical issues across pipelined instructions: • EQ model • LEQ model mov $r1 = 2;;mov $r0 = $r1mov $r1 = 3;; ld.w $r0 = 0[$r1];;add $r0 = $r1, $r2;;sub $r3 = $r0, $r4…# load completed: add $r3 = $r3, $r0 EQ model allows $r0 to be reusedbetween issue of 1st instruction andits completion when latency expires
Acyclic Region Scheduling for Loops • To fulfill the need to enlarge the region size of a loop body to expose more ILP, apply: • Loop fusion • Loop peeling • Loop unrolling DO I = 1, N A(I) = C*A(I)ENDDODO I = 1, N D(I) = A(I)*B(I)ENDDO DO I = 1, N A(I) = C*A(I) D(I) = A(I)*B(I)ENDDO DO I = 1, N, 2 A(I) = C*A(I) D(I) = A(I)*B(I) A(I+1) = C*A(I+1) D(I+1) = A(I+1)*B(I+1)ENDDO (Assuming 2 divides N)
Region Scheduling Across Basic Blocks • Region scheduling schedules operations across basic blocks, usually on hot paths • Fulfill the need to increase the region size by merging operations from block to expose more ILP • But problem with conditional flow: how to move operations from one block to another for instruction scheduling? Move operationfrom here to there B3 B6 B4 But operation is nowmissing on this path
Region Scheduling Across Basic Blocks • Problem: how to move operations from one block to another for instruction scheduling? • Affected branches need to be compensated Move operationfrom here to there B3 B6 B4 But operation is nowinserted on this path
Trace Scheduling • Earliest region scheduling approach has restrictions • A trace consists of a the operations from a list of basic blocks B0, B1, …, Bn • Each Bi is a predecessor (falls through or branches to) the next Bi+1on the list • For any i and k there is no path BiBkBi except for i=0, i.e. the code is cycle free except that the entire region can be part of a loop 10 10 B1 B1 70 30 70 30 B2 B5 B2 B5 70 30 70 30 B3 B3 80 80 B6 B6 20 20 80 80 B4 B4 90 90 10 10
Superblocks • Superblocks are single-entry multiple-exit traces • Superblock formation uses tail duplication to to eliminate side entrances • Each Bi is a predecessor of the next Bi+1on the list (fall through) • For any i and k there is no path BiBkBi except for i=0 • There are no branches into a block in the region (no side entrances), except to B0 10 10 B1 B1 30 70 30 70 B2 B5 B5 B2 70 30 70 30 B3 B3 B3’ 14 20 24 6 B6 80 56 B6 20 20 B4 B4 B4’ 90 10 5.6 4.4 50.4 39.6
Hyperblocks • Hyperblocks are single-entry multiple-exit traces with internal control flow effectuated via instruction predication • If-conversion folds flow into single block using instruction predication 10 10 B1 B1 70 30 B2 B5 B2,B5 70 30 B3 B3 20 20 B6 80 80 B6 20 20 B4 B4 B4’ 90 10 8 2 72 18 20
Intermezzo: Predication cmpgt $b1 = $r5, 0 ;; br $b1, L1 ;; mpy $r3 = $r1, $r2 ;;L1: stw 0[$r10] = $r3 ;; • If-conversion translates control dependences into data dependences by instruction predication to conditionally execute them • Predication requires hardware support • Full predication adds a boolean operand to (all or selected) instructions • Partial predication executes all instructions, but selects the final result based on a condition Original cmpgt $p1 = $r5, 0;;($p1) mpy $r3 = $r1, $r2;;stw 0[$r10] = $r3;; After fullpredication mpy $r4 = $r1, $r2;;cmpgt $b1 = $r5, 0;;slct $r3 = $b1, $r4, $r3;;stw 0[$r10] = $r3;; After partialprediction
Treegions • Treegions are regions containing a trees of blocks such that no block in a treegion has side entrances • Any path through a treegion is a superblock Treegion 1 Treegion 2 Treegion 3
Region Formation • The scheduler constructs schedules for a single region at a time • Need to select which region to optimize (within limits of regions shape), i.e. group traces of frequently executed blocks into regions • May need to enlarge regions to expose enough ILP for scheduler Regionselection Regionenlargement Scheduleconstruction
Region Selection by Trace Growing • Trace growing uses the mutual most likely heuristic: • Suppose A is last block in trace • Add block B to trace if B is most likely successor of A and A is B’s most likely predecessor • Also works to grow backward • Requires edge profiling, but result can be poor because edge profiling does not correlate branch probabilities 55 A 5 40 10 5 40 B
Region Selection by Path Profiling • Treat trace as a path and consider its execution frequency by path profiling • Correlations are preserved in the region formation process B1 B1 B2 B5 B5 B2 B3 B3 B3’ B6 B6 path 1: {B1, B2, B3, B4} count = 44path 2: {B1, B2, B3, B6, B4} count = 0path 3: {B1, B5, B3, B4} count = 16path 4: {B1, B5, B3, B6, B4} count = 12 B4 B4 B4’
Superblock Enlargement by Target Expansion • Target expansion is useful when the branch at the end of a superblock has a high probability but the superblock cannot be enlarged due to a side entrance • Duplicate sequence of target blocks to a create larger superblock 80 80 B1 B1 B2 B2 20 20 70 10 10 B3’ B3 B3 B4’ B4 B4 70 90 20
Superblock Enlargement by Loop Peeling • Peel a number of iterations of a small loop body to create a larger superblock that branches into the loop • Useful when profiled loop iterations is bounded to a small constant (two iterations in the example) 10 B1 10 B2 B1 0 10 B1’ B2 B2’ 10 10 0 10 B1” B2” 0 0
Superblock Enlargement by Loop Unrolling • Loops with a superblock body and a backedge with high probability are called superblock loops • When a superblock loop is small we can unroll the loop 10 B1 10 B2 B1 3.3 10 B1’ B2 B2’ 90 10 3.3 B1” B2” 30 3.3
Exposing ILP After Loop Unrolling • Loop unrolling exposes limited amount of ILP • Cross-iteration dependences on the loop counter’s updates prevent parallel execution of the copies of the loop body • Cannot generally move instructions across split points • Note: can use speculative execution to hoist instructions above split points B1 B2 Split point 10 B1’
Schedule Construction • The schedule constructor (scheduler) uses compaction techniques to produce a schedule for a region after region formation • The goal is to minimize an objective cost function while maintaining program correctness and obeying resource limitations: • Increase speed by reducing completion time • Reduce code size • Increase energy efficiency Regionselection Regionenlargement Scheduleconstruction
Schedule Construction and Explicitly Parallel Architectures • A scheduler for an explicitly parallel architecture such as VLIW and EPIC uses the exposed ILP to statically schedule instructions in parallel • Instruction compaction must obey data dependences (RAW, WAR, and WAW) and control dependences to ensure correctness add $r13 = $r3, $r0shl $r13 = $r13, 3ld.w $r14 = 0[$r4] sub $r16 = $r6, 3shr $r15 = $r15, 9 add $r13 = $r3, $r0sub $r16 = $r6, 3;; ## end of 1st instr. shl $r13 = $r13, 3shr $r15 = $r15, 9ld.w $r14 = 0[$r4];; ## end of 2nd instr. bundle bundle
Schedule Construction and Instruction Latencies • Instruction latencies must be taken into account by the scheduler, but they’re not always fixed or the same for all ops • A scheduler can assume average or worst-case instruction latencies • Hide instruction latencies by ensuring that there is sufficient height between instruction issue and when result is needed to avoid pipeline stalls • Also recall the difference between the EQ versus the LEQ model Takes 2 cyclesto complete Takes 1 cycleto complete mul $r3 = $r3, $r1 add $r13 = $r2, $r3ld.w $r14 = 0[$r5]add $r13 = $r13, $r14ld.w $r15 = 0[$r6] Takes >3 cycles(4 cycles ave.) RAW hazards
Linear Scheduling Techniques cycle • Instruction compaction using linear-time scans over region: • As-soon-as-possible (ASAP) scheduling places ops in the earliest possible cycle using top-down scan • As-late-as-possible (ALAP) scheduling places ops in the latest possible cycle using bottom-up scan • Critical-path (CP) scheduling uses ASAP followed by ALAP • Resource hazard detection is local mul $r3 = $r3, $r1add $r13 = $r2, $r3ld.w $r14 = 0[$r5]add $r13 = $r13, $r14ld.w $r15 = 0[$r6] 02031 mul $r3 = $r3, $r1ld.w $r14 = 0[$r5];; ld.w $r15 = 0[$r6];; add $r13 = $r2, $r3;; add $r13 = $r13, $r14;; At most oneload per inst.
List Scheduling • List scheduling schedules operations from the global region based on a data dependence graph (DDG) or program dependence graph (PDG) which both have O(n2) complexity • Repeatedly selects an operation from a data-ready queue (DRQ), where an operation is ready when all if its DDG predecessors have been scheduled for each root r in the PDG sorted by priority do enqueue(r) while DRQ is non-empty doh = dequeue() schedule(h)for each DAG successor s of hdoif all predecessors of s have been scheduled then enqueue(s)
Data Dependence Graph • The data dependence graph (DDG) • Nodes are operations • Edges are RAW, WAR, and WAW dependences
Compensation Code • Compensation code is needed when operations are scheduled across basic blocks in a region • Compensation code corrects scheduling changes by duplicating code on entries and exits from a scheduled region X SchedulerinterchangesA with B A entry B exit C Y Entry and/or exitmust be compensated
No Compensation • No compensation code is needed when block B does not have an entry and exit X X A B B A C C Y Y
Join Compensation • Join compensation is applied when block B has an entry • Duplicate block B X X A B Z Z B A B’ C C Y Y
Split Compensation • Split compensation is applied when block B has an exit • Duplicate block A X X A B B A A’ W C C W Y Y
Join-Split Compensation • Join-split compensation is applied when block B has an entry and an exit • Duplicate block A and B X X Z A B Z B’ B A W A’ W C C W Y Y
Resource Management with Reservation Tables • A resource reservation table records which resources are busy per cycle • Reservation tables allow easy scheduling of operations by matching the operation’s required resources to empty slots • Construction of reservation table at a join point in the CFG is constructed by merging busy slots from both branches
SoftwarePipelining DO i = 0, 6 A B C D E F G HENDDO prologue kernel epilogue Assuming that the initiationinterval (II) is 3 cycles
Software Pipelining Example > 3 cycles > 2 cycles >1 cycle
Modulo Scheduling DDG MRT
Constructing Kernel-Only Code by Predicate Register Rotation BRT branches to the top and rotates the predicate registers:p1 = p0, p2 = p1, p3 = p2, p0 = p3