A Polymorphic Type System for Bulk Synchronous Parallel ML

1. A Polymorphic Type System for Bulk Synchronous Parallel ML Frédéric Gava and Frédéric Loulergue Laboratory of Algorithms, Complexity and Logic Paris, France

2. Overview • Introduction • Pure functional bulk synchronous parallel programming • Problems with nesting • Overview of the system • Conclusion and future work PaCT 2003

3. Introduction

4. Bulk Synchronous Parallelism + Functional Programming = BSML • Bulk Synchronous Parallelism : • Scalability • Portability • Simple cost model • Functional Programming : • High level features (higher order functions, pattern matching, concrete types, etc.) • Programs proofs • Safety of the environment PaCT 2003

5. Bulk Synchronous Parallelism T(s) = (max0i<p wi) + hg + l PaCT 2003

6. The BSMLlib Library • Bulk Synchronous Parallel ML libraryfor the Objective Caml language • operations on a parallel data structureAbtract type:  par • access to BSP parameters: bsp_p: unit  int bsp_p() = number of processes PaCT 2003

7. (f 0) (f 1) … f (p-1) Creation of Parallel Vectors • mkpar: (int  )   par (mkpar f ) Cost: (max0i<p wi) PaCT 2003

8. f0 f1 … fp-1 v0 v1 … vp-1 f0 v0 f1 v1 … fp-1 vp-1 Pointwise Parallel Application • apply: () par  par  par apply = PaCT 2003

9. 0 1 2 3 0 1 2 3 None Some v2 None None None None None Some v1 None None Some v5 None Some v2 None Some v3 Some v4 None Some v3 None None None Some v5 None None Some v1 Some v4 None None None None None None Communication Operation: Put type  option = None | Some of  put: (int option) par(int option) par put = Cost: (max0i<p wi) + hg + l PaCT 2003

10. b0 b1 … true … fp-1 Global Conditional if vec at n then … else … if at n then e1 else e2 = e1 Cost: (p-1)g + l n PaCT 2003

11. Implementations of BSMLlib • BSMLlib v 0.1 : • O. Ballereau, G. Hains, F. Loulergue • Ocaml + BSPlib • End of 1999 • BSMLlib v 0.2 : • Frédéric Gava, Xavier Leroy, Frédéric Loulergue • Ocaml + MPI • Available (http://bsmllib.free.fr) PaCT 2003

12. Examples • let replicate x = mkpar(fun pid->x) • (* bcast: int->’a par->’a par *) let bcast n vec = let tosend=mkpar(fun i v dst -> if i=n then Some v else None) in let recv=put(apply tosend vec) in apply (replicate noSome) (apply recv (replicate n)) PaCT 2003

13. Problems with the nesting of parallel vectors

14. Example (1) let example1 = mkpar(fun pid->bcast pid vec) (* example1: t par par *) PaCT 2003

15. Example (2) let example2 = mkpar(fun pid-> let this=mkpar(fun i->i) in pid) (* example2: int par *) PaCT 2003

16. Example (3) (* fst: ’a *’b ->’a *) let fst = fun (a,b) -> a • two usual values: fst(1,2) • two parallel values: fst(mkpar(fun i->i),mkpar(fun i->i)) • parallel and usual: fst (mkpar(fun i->i),1) • usual and parallel: fst (1, mkpar(fun i->i)) PaCT 2003

17. Example (4) let v1 x=mkpar(fun pid -> pid) and v2 x=put(mkpar(fun i s-> 1+s))in let c1 x = ((v1 x),1) and c2 x = ((v2 x),2) in mkpar(fun pid -> if pid<(nproc/2) then snd (c1()) else snd (c2())) PaCT 2003

18. Overview of the type system

19. Types PaCT 2003

20. Constraints PaCT 2003

21. Types of some predefined expressions PaCT 2003

22. Typing judgments in the type environment E(which binds variables to types) the expression ehas the type [t/C]. PaCT 2003

23. Some Rules PaCT 2003

24. Conclusion and Future Work • Conclusion • A Polymorphic Type to avoid the nesting of parallel vectors of BSML • Implementation for a mini-BSML • Future Work • Sum types • Imperative features • Full implementation PaCT 2003