260 likes | 339 Views
Semantics of Minimally Synchronous Parallel ML. Myrto Arapini, Frédéric Loulergue, Frédéric Gava and Frédéric Dabrowski LACL, Paris, France. Outline. Introduction Pure functional minimally synchronous parallel programming High level semantics Distributed evaluation
E N D
Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frédéric Loulergue, Frédéric Gava and Frédéric Dabrowski LACL, Paris, France
Outline • Introduction • Pure functional minimally synchronous parallel programming • High level semantics • Distributed evaluation • Conclusions and future work SNPD 2003
Our previous work:Bulk Synchronous Parallelism + Functional Programming = BSML • Bulk Synchronous Parallelism : • Scalability • Portability • Simple cost model • Functional Programming : • High level features (higher order functions, pattern matching, concrete types, etc.) • Programs proofs • Safety of the environment SNPD 2003
What is MSPML ?Why MSPML ? • = Minimally Synchronous Parallel ML • BSML without synchronization barriers • Comparison of efficiency BSML/MSPML • Further extensions restricted or impossible in BSML (divide-and-conquer, nesting of parallel values) • Basis for an additional level to BSML for Grid computing: BSML on each nodes (clusters) + MSPML for coordination SNPD 2003
The MSPML Library • Parallel ML library for the Objective Caml language • operations on a parallel data structureAbtract type: par • access to the machine parameters: p: unit int p() = number of processes SNPD 2003
(f 0) (f 1) … f (p-1) Creation of Parallel Vectors • mkpar: (int ) par (mkpar f ) SNPD 2003
f0 f1 … fp-1 v0 v1 … vp-1 f0 v0 f1 v1 … fp-1 vp-1 Pointwise Parallel Application • apply: () par par par apply = SNPD 2003
5 5 … 5 Example (1) • let replicate x = mkpar(fun pid->x)replicate: ’a -> ’a par (replicate 5) SNPD 2003
(f v0 ) f v0 f v1 (f v1 ) … … … (f vp-1 ) f vp-1 apply = Example (2) • let parfun f v = apply (replicate f) vparfun: (’a->’b)->’a par->’b par parfun (fun x->x+1) parallel_vector f SNPD 2003
get v0 v1 … vp-1 i0(=1) i1 … ip-1 = vi0 vi1 … vip-1 Communication Operation: get get: parint par par SNPD 2003
v0 vroot v1 vroot … … vp-1 vroot (bcast_direct root ) = Example (3) let bcast_direct root parallel_vector =if not(within_bounds root)then raise Bcast_Error else get parallel_vector (replicate root) bcast_direct: int->’a par->’a par SNPD 2003
b0 b1 … true … fp-1 Global Conditional if parallel_vector at n then … else … if atnthen E1 else E2 = E1 n SNPD 2003
Implementation of MSPML • MSPML v 0.05 : • F. Loulergue • Library for Ocaml (uses threads and TCP/IP) • October 2003 • http://mspml.free.fr SNPD 2003
Terms • Functional semantics: can be evaluated sequentially or in parallel • Terms : SNPD 2003
Values and judgements • Values: • Judgement: e v Term « e » evaluates to value « v » SNPD 2003
Some rules SNPD 2003
Informal presentation (1) • MSPML programs seen as SPMD programs • The parallel machine runs p copies of the same MSPML program • Rules for local reduction + rule for communication • For example at processor i the expression (mkpar f) is evaluated to (f i) SNPD 2003
Informal presentation (2) Proc. 0 1 2 0,v’’ 0,v’ empty 0,v A BIT LATER Local computation get v 1 request 0 1 v’ Communication environment SNPD 2003
Informal presentation (3) Proc. 0 1 2 0,v’’ 1,w’ 2,w’’ 0,v’ 0,v’ 1,w empty 0,v Local computation Not Ready !! request 2 0 Communication environment SNPD 2003
Terms and judgments • New term: request i jthe processor asks the value stored at processor j during the ithstep • Step = each step is ended by a call to get • Judgment: At process i, the term ed with communication environment Ec is evaluate to e’d with new communication environment E’c SNPD 2003
Local computation rules SNPD 2003
Communication rule SNPD 2003
Conclusion and Future Work • Conclusion • Minimally Synchronous Parallel ML:Functional semantics & Deadlock free • Two semantics: • High level semantics (programmer’s view) • Distributed evaluation (implementation’s view) • Implementation • Future Work • MPI Implementation • Comparison with BSMLlib • Extensions: parallel composition, etc. SNPD 2003