Data Flow Analysis for Software Prefetching Linked Data Structures in Java

1. Data Flow Analysis for Software Prefetching Linked Data Structures in Java Brendon Cahoon Dept. of Computer Science University of Massachusetts Amherst, MA Kathryn S. McKinley Dept. of Computer Sciences University of Texas at Austin Austin, TX

2. Motivation • Object-oriented languages are mainstream • Key performance issues • Same old: processor-memory gap, parallelism • Combination of modern processors and languages results in poor memory performance

3. RSIM PerformanceCompiled Java (Vortex) – no GC

4. Prefetching Arrays vs. Objects • Most prior work concentrates on arrays • Compilers directly prefetch any element • Loop transformations enable effective scheduling • Successful results using both hardware and software • Cannot use same techniques on linked data structures • Objects are small and disjoint • Access patterns are less regular and predictable • Only know the address of directly connected objects

5. Software Data Prefetching for Java Hide memory latency from linked structure traversals • Introduced by Luk and Mowry for C programs: • We add data flow and interprocedural analysis • Identify pointer structures from declaration • Find pointer chasing in loops and self recursive calls • Challenges introduced by Java • Dynamically allocated objects make analysis difficult • Small methods obscure context

6. Outline • Data flow analysis for identifying linked structures • New intra and interprocedural analysis • Greedy prefetching • Jump-pointer prefetching • Experimental results

7. We define a data flow solution: Intraprocedural for loops Interprocedural for recursion Benefits: Independent of program representation Many compilers use data flow frameworks May be composed with other analyses Loop while (o != null) { t = o; … o = t.next; } Recursion method visit() { …. if (this.next != null) visit(this.next); } Identifying Linked Structure Traversals

8. Data Flow Analysis • Data flow information • Sets of tuples: <variable, field name, statement, status> • Status values: not recurrent, possibly, recurrent Not recurrent : initial value Possibly : first use of a field reference Recurrent: an object accessed in linked structure traversal • Intraproceedural: forward, flow-sensitive, may analysis • Interprocedural: bidirectional, context-sensitive

9. while (o != null) { s1: t = o.next; s2: o = t; } while (o != null) { s1: o = o.next; s2: o = bar(); } s1: o = o.next; s2: o = o.next; 1st Iteration s1: o is not recurrent, set t to possibly s2: t is possibly, set o to possibly Analysis Examples 2nd Iteration s1: o is possibly, set t to recurrent s2: t is recurrent, set o to recurrent 1st Iteration s1: set o to possibly s2: set o to not recurrent s1: set o to possibly s2: set o to possibly

10. Analysis Extensions for Common Idioms • Track objects in fields or arrays • Class based field assignments • Arrays are monolithic • Indirect recurrent objects • Unique objects referenced by linked structures while (e.f != null) { o = e.f; e.f = o.next; o.compute(); } while (e.hasMoreElements()) { o = (ObjType)e.nextElement(); o.compute(); }

11. Greedy Prefetching • Prefetch directly connected objects • Algorithm consists of two steps: • Detect accesses to linked structures • Schedule prefetches • When object is not null • Completely hiding latency is difficult

12. Greedy Prefetching Example int sum (Dlist l) { int s = 0; while (l != null) { s =+ l.data; l = l.next; } return s; } Doubly linked list

13. Greedy Prefetching Example int sum (Dlist l) { int s = 0; while (l != null) { prefetch(l.next); s += l.data; l = l.next; } return s; } Doubly linked list Greedy prefetching

14. Jump-Pointer Prefetching • Prefetch indirectly connected objects • Tolerates more latency than greedy prefetching • Algorithm contains three steps: • Find linked data structure traversal and creation sites • Create jump-pointers • When creating or traversing the linked structure • Schedule prefetches • Prefetch special jump-pointer field

15. Inserting Jump-Pointers at Creation Time Void add(ObjType o) { ListNode n = new ListNode(o); jumpObj = jumpQueue[i]; jumpObj.jmp = n; jumpQueue[i++%size] = n; if (head == null) { head = n; } else { tail.next = n; } tail = n; } 2 1 3 4 5 jumpObj n

16. Jump-Pointer Prefetching Example Doubly linked list Jump-pointer prefetching int sum (Dlist l) { int s = 0; while (l != null) { prefetch(l.jmp); s += l.data; l = l.next; } return s; }

17. Experimental Results • Object-oriented Olden benchmarks in Java • Simulation using RSIM • Out-of-order, superscalar processor • Compile programs using Vortex • Translate Java programs to Sparc assembly • Contains object-oriented, traditional optimizations • Linked structure analysis, greedy and jump-pointer prefetching

18. Prefetching Performance health mst perimtr treeadd bh bisort tsp voronoi em3d power

19. Prefetch Effectiveness health mst perimtr treeadd bh bisort tsp voronoi em3d power

20. Static Prefetch Statistics

21. Contributions and Future Work • New interprocedural data flow analysis for Java • Evaluation of prefetching on Java programs Prefetching hides latency, but Room for improvement Other uses for analysis (work in progress) • Garbage collection: prefetching, object traversal • Prefetching arrays of objects