Static Analysis of Communication Structures in Parallel Programs

1. Static Analysis of Communication Structures in Parallel Programs So-Yan Ho and Nai-Wei Lin Department of Computer Science and Information Engineering National Chung Cheng University

2. Motivations • Communication structures in message passing programs is the key to understand the behaviors of the programs • Staticanalysis of communication structures allows programmers to identify potential communication bugs at compile time

3. Message Passing Schemes • Direct-Addressing: CSP, Occam, Ada, MPI send(task-id, message); message = receive(task-id); • Indirect-Addressing: Fortran M, CCC send(channel-id, message); message = receive(channel-id);

4. Channels in CCC • Pipes: one-to-one communication • Spliters: one-to-many communication • Mergers: many-to-one communication • Multiplexers: many-to-many communication

5. Merits of Abundant Channels • Communication structures are morecomprehensive • The specification of communication structures is easier • The implementation of communication structures is more efficient • The static analysis of communication structures is more effective

6. Higher-Order Channels • A channel is first-order if it is used to transfer data • A channel is higher-order if it is used to transfer channels • The static analysis of higher-order channels is much more difficult than the static analysis of first-order channels

7. A Simple CCC Program task::main() { spliterint ch; /* one-to-many channel */ int i; ch = channel(); par { producer(ch, many); /* one sender */ parfor (i = 0; i < many; i++) consumer(ch); /* many receivers */ } }  

8. A Simple CCC Program task::producer(ch, many)spliterint ch; int many;{int i;for (i = 0; i < num_data; i++)send(ch, i); for (i = 0; i < many; i++) send(ch, end_data); /* signal end of data */}

9. A Simple CCC Program task::consumer(ch) spliterint ch; { int data; while (1) { data = receive(ch); if (data == end_data) break; process(data); } }

10. Overview of the Analysis • For each channel ch, infer the number of senders of ch, ch.s and the number of receivers of ch, ch.r • Intraprocedural analysis: within a function • infer them once and for all, and use them as the initial values of the interprocedural analysis • Interprocedural analysis: among functions • propagate the values among functions until they reach a fixed point

11. First-Order Channels • Use aliasing induced by parameter passing • For each channel ch, infer ch.s = ch.self.s + ch.others.s ch.r = ch.self.r + ch.others.rwhere, ch.self.s, ch.self.r: {0, 1} ch.others.s, ch.others.r: {0, 1, 2} • We call the four tuple {ch.self.s, ch.self.r, ch.others.s, ch.others.r} the mode of ch

12. Intraprocedural Analysis • For each channel ch, if ch is sent by current function then ch.self.s = 1 else ch.self.s = 0 if ch is received by current function then ch.self.r = 1 else ch.self.r = 0

13. Interprocedural Analysis • Compact control flow trees • call nodes, spawn nodes, block nodes, alternative nodes, repetition nodes • depth-first order (once) • The call graph and the strongly connected component graph • topological sort order (once) • Strongly connected components • worklist (iterations)

14. Compact Control Flow Trees task::main() { spliterint ch; int i; ch = channel(); par { producer(ch, many); parfor (i = 0; i < many; i++) consumer(ch); } }  

15. The Call Graph and Strongly Connected Component Graph task :: main() { . . . par { producer(ch,…); parfor ( . . . ) consumer(ch); } . . . } task ::producer(ch, …) { . . . } task :: consumer(ch) { . . . process(data); . . . }

16. Call Nodes • Let L be the set of channel parameters aliased with ch at the call node n • Then chn.self.s = x L x.self.s, chn.self.r = x L x.self.r, chn.others.s = x L x.others.s, chn.others.r = x L x.others.r

17. Spawn Nodes • Let L be the set of channel parameters aliased with ch at the spawn node n • Then chn.self.s = 0, chn.self.r = 0, chn.others.s = x L x.self.s + x L x.others.s, chn.others.r = x L x.self.r + x L x.others.r

18. Block Nodes • If the block node n is the initial nodechn.self.s=value from intraprocedural analysis, chn.self.r=value from intraprocedural analysis, chn.others.s=0, chn.others.r=0 • Otherwise chn.self.s=0, chn.self.r=0, chn.others.s=0, chn.others.r=0

19. Bolck Nodes • Let L be the sequence of nodes inside the block node n • Then chn.self.s = chn.self.s ||x L chx.self.s, chn.self.r = chn.self.r ||x L chx.self.r, chn.others.s = chn.others.s +x L chx.others.s, chn.others.r = chn.other.r +x L chx.others.r

20. Alternative Nodes • Let L be the sequence of nodes inside the alternative node n • Then chn.self.s = maxx L chx.self.s, chn.self.r = maxx L chx.self.r, chn.others.s = maxx L chx.others.s, chn.others.r = maxx L chx.others.r

21. Repetition Nodes • Let m be the node inside the repetition node n • Then chn.self.s = chm.self.s, chn.self.r = chm.self.r, chn.others.s = if chm.others.s == 0 then 0 else 2, chn.others.r = if chm.others.r == 0 then 0 else 2

22. Second-Order Channels • Use aliasing induced by both parameter passing and message passing • For each second-order channel ch, infer ch.sS = the set of channels sent to ch ch.rS = the set of channels received from ch

23. Intraprocedural Analysis • For each second-order channel ch, ch.sS =  ch.rS =  For each first-order channel x, if x is sent to ch ch.sS = ch.sS ∪ { x } if x is received from ch ch.rS = ch.rS ∪ { x }

24. Parameter Passing Aliasing task :: main() { merger pipe int ch; pipe int m; . . . par { parfor ( . . . ) client(ch, m); server(ch); } } task :: client(ch, c) { . . . send(ch, c); . . . } ch.sS : { c } task :: server(ch) { . . . s = receive(ch); . . . } ch.sS ::  ch.sS : { m }

25. Call and Spawn Nodes • Let S be the set of second-order channel parameters aliased with ch at the call node n • Let F be the set of first-order channel parameters at n, and for each pF, let (p) represent the corresponding channel argument of p at n • Then chn.sS = x S x.sS{pF ,p/(p)} , chn.rS = x S x.rS {pF ,p/(p)}

26. Message Passing Aliasing task :: main() { merger pipe int ch; pipe int m, n; . . . par { parfor ( . . . ) client(ch, m); server(ch, n); } } task :: client(ch, c) { . . . send(ch, c); . . . } ch.sS : { c } ch.rS:  task :: server(ch, s) { . . . s = receive(ch); . . . } ch.sS ::  ch.rS: { s } ch.sS : { m } ch.rS: { n }

27. Call and Spawn Nodes • For each x in ch.sS, x is aliased with every y in ch.rS • Then, for each c in {x}  ch.rS c.self.s = d {x}  ch.rS d.self.s, c.self.r = d {x}  ch.rS d.self.r, c.others.s = d {x}  ch.rS d.others.s, c.others.r = d {x}  ch.rS d.others.r

28. Block Nodes • If the block node n is the initial nodechn.sS=value from intraprocedural analysis, chn.rS=value from intraprocedural analysis • Otherwise chn.sS= , chn.rS=

29. Block Nodes • Let L be the sequence of nodes inside the block node n • Then chn.sS = chn.sS x L chx.sS, chn.rS = chn.rS x L chx.rS

30. Alternative Nodes • Let L be the sequence of nodes inside the alternative node n • Then chn.sS = x L chx.sS, chn.rS = x L chx.rS

31. Repetition Nodes • Let m be the node inside the repetition node n • Then chn.sS = chm.sS, chn.rS = chm.rS

32. Time Complexity intraprocedural analysis; for each SCC in SCCG begin while changed begin for each CCFT in the SCC begin for each node in the CCFT begin … endfor endfor endwhile endfor The time complexity: O(V+E)

33. Conclusions • Design a linear algorithm to infer the number of senders and receivers for each channel • Handle both first-order and higher-orderchannels • Use the information about aliasing induced by both parameter passing and message passing