1 / 26

Semantics of Minimally Synchronous Parallel ML

Semantics of Minimally Synchronous Parallel ML. Myrto Arapini, Frédéric Loulergue, Frédéric Gava and Frédéric Dabrowski LACL, Paris, France. Outline. Introduction Pure functional minimally synchronous parallel programming High level semantics Distributed evaluation

tracy
Download Presentation

Semantics of Minimally Synchronous Parallel ML

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Semantics of Minimally Synchronous Parallel ML Myrto Arapini, Frédéric Loulergue, Frédéric Gava and Frédéric Dabrowski LACL, Paris, France

  2. Outline • Introduction • Pure functional minimally synchronous parallel programming • High level semantics • Distributed evaluation • Conclusions and future work SNPD 2003

  3. Introduction

  4. Our previous work:Bulk Synchronous Parallelism + Functional Programming = BSML • Bulk Synchronous Parallelism : • Scalability • Portability • Simple cost model • Functional Programming : • High level features (higher order functions, pattern matching, concrete types, etc.) • Programs proofs • Safety of the environment SNPD 2003

  5. What is MSPML ?Why MSPML ? • = Minimally Synchronous Parallel ML •  BSML without synchronization barriers • Comparison of efficiency BSML/MSPML • Further extensions restricted or impossible in BSML (divide-and-conquer, nesting of parallel values) • Basis for an additional level to BSML for Grid computing: BSML on each nodes (clusters) + MSPML for coordination SNPD 2003

  6. The MSPML Library • Parallel ML library for the Objective Caml language • operations on a parallel data structureAbtract type:  par • access to the machine parameters: p: unit  int p() = number of processes SNPD 2003

  7. (f 0) (f 1) … f (p-1) Creation of Parallel Vectors • mkpar: (int  )   par (mkpar f ) SNPD 2003

  8. f0 f1 … fp-1 v0 v1 … vp-1 f0 v0 f1 v1 … fp-1 vp-1 Pointwise Parallel Application • apply: () par  par  par apply = SNPD 2003

  9. 5 5 … 5 Example (1) • let replicate x = mkpar(fun pid->x)replicate: ’a -> ’a par (replicate 5) SNPD 2003

  10. (f v0 ) f v0 f v1 (f v1 ) … … … (f vp-1 ) f vp-1 apply = Example (2) • let parfun f v = apply (replicate f) vparfun: (’a->’b)->’a par->’b par parfun (fun x->x+1) parallel_vector f SNPD 2003

  11. get v0 v1 … vp-1 i0(=1) i1 … ip-1 = vi0 vi1 … vip-1 Communication Operation: get get:  parint par  par SNPD 2003

  12. v0 vroot v1 vroot … … vp-1 vroot (bcast_direct root ) = Example (3) let bcast_direct root parallel_vector =if not(within_bounds root)then raise Bcast_Error else get parallel_vector (replicate root) bcast_direct: int->’a par->’a par SNPD 2003

  13. b0 b1 … true … fp-1 Global Conditional if parallel_vector at n then … else … if atnthen E1 else E2 = E1 n SNPD 2003

  14. Implementation of MSPML • MSPML v 0.05 : • F. Loulergue • Library for Ocaml (uses threads and TCP/IP) • October 2003 • http://mspml.free.fr SNPD 2003

  15. High level semantics

  16. Terms • Functional semantics: can be evaluated sequentially or in parallel • Terms : SNPD 2003

  17. Values and judgements • Values: • Judgement: e  v Term « e » evaluates to value « v » SNPD 2003

  18. Some rules SNPD 2003

  19. Distributed evaluation

  20. Informal presentation (1) • MSPML programs seen as SPMD programs • The parallel machine runs p copies of the same MSPML program • Rules for local reduction + rule for communication • For example at processor i the expression (mkpar f) is evaluated to (f i) SNPD 2003

  21. Informal presentation (2) Proc. 0 1 2 0,v’’ 0,v’ empty 0,v A BIT LATER Local computation get v 1 request 0 1 v’ Communication environment SNPD 2003

  22. Informal presentation (3) Proc. 0 1 2 0,v’’ 1,w’ 2,w’’ 0,v’ 0,v’ 1,w empty 0,v Local computation Not Ready !! request 2 0 Communication environment SNPD 2003

  23. Terms and judgments • New term: request i jthe processor asks the value stored at processor j during the ithstep • Step = each step is ended by a call to get • Judgment: At process i, the term ed with communication environment Ec is evaluate to e’d with new communication environment E’c SNPD 2003

  24. Local computation rules SNPD 2003

  25. Communication rule SNPD 2003

  26. Conclusion and Future Work • Conclusion • Minimally Synchronous Parallel ML:Functional semantics & Deadlock free • Two semantics: • High level semantics (programmer’s view) • Distributed evaluation (implementation’s view) • Implementation • Future Work • MPI Implementation • Comparison with BSMLlib • Extensions: parallel composition, etc. SNPD 2003

More Related