Parallel Specification and Implementation Language: A Pointer-Free Path to

1. ParaSail Parallel Specification and Implementation Language: A Pointer-Free Path to Secure Object-Oriented Parallel Programming S. Tucker Taft FOOL Workshop, Tuscon, AZ October 2012

2. Goals of ParaSail • A simple, safe, pervasively parallel, yet familiar, language for the multi/many-core world • Easier to write parallel algorithms than sequential ones • Parallel by default, yet race-free • Have to work harder to get sequential execution • Not an academic exercise • More of a human engineering exercise • Presumes advanced static analysis at compile-time

3. Why a new language? Computers have stopped getting faster Courtesy IEEE Computer, January 2011, page 33.

4. A Simplified Approach to Secure Parallel Programming • Simplify/Unify • Smaller number of concepts, uniformly applied, all features available to user-defined types -- generally a good thing • Simplify to make conceptualroom for parallelism and formalism • Simplify to make pervasiveparallelism easy to verify • Remove all the hard problems! • Parallelize • Parallel by default, every expression is parallelizable • Have to work harder to force sequential execution • Formalize • Assertions, Invariants, Preconditions, Postconditions integrated into the syntax • Compiler complains if it can’t prove the assertion • No run-time exceptions, no run-time exception handling

5. ParaSail Simplifications • No pointers (the “goto” of data structuring) • No global variables • No global, garbage-collected heap • No parameter aliasing • No special syntax reserved for built-in types • No run-time exception handling • No explicit threads, lock/unlock, wait/signal • No race conditions • No algebraic data types -- use OO polymorphism • No boxing/run-time-type overhead unless explicitly polymorphic

6. ParaSail Type and Value ModelOO/functional core Types: • T ::= Tid | Tid+ | Mid <T1, T2, ...> | optionalT | new T • FT ::= (Oid1 : T1; Oid2 : T2; ... func Fid1FT1; ...) -> T3 Values: • V ::= N | L | (V) | FN (V1, V2, ... FV1, FV2, ...) V ::= (ifV1thenV2elseV3) | Visnull | Vnotnull • N ::= Oid | N.Oid | Tid :: N// names • L ::= 42 | 3.14159 | “a string” | ‘c’ | #green // literals L ::= null | (Oid1 => V1, Oid2 => V2, ...) • FV ::= FN | lambdaFTis (V) // function values • FN ::= Fid | Tid :: FN// function names

7. ParaSail Declarations and ModulesOO/functional core Declarations: • D ::= type Tid isT; D ::= func Fid FT [is (V)]; D ::= const Oid [ : T ] [ := V ]; D ::= M Modules: • M ::= [abstract] interface Mid < Tid1is Mid1<>, ... > [extends Mid2] [implements Mid3<...>, ...] isD1; D2; ... end interface Mid; M ::= class Mid isD11; ... exportsD21; ... end class Mid;

8. ParaSail Syntactic Sugar Sugared Syntax Expanded Syntax

9. Example: Countable_Set interface Countable_Set <Element_Type is Countable<>> is op "[]"() -> Countable_Set; func Singleton(Elem : Element_Type) -> Countable_Set; op “in”(Element_Type; Countable_Set) -> Boolean; op “|”(Left, Right : Countable_Set) -> Countable_Set; ... end interface Countable_Set; class Countable_Set is type Elem_Interval is Closed_Interval<Element_Type>; const Items : optional AA_Tree<Elem_Interval>; exports op "[]"() -> Countable_Set is ((Items => [])); func Singleton(Elem : Element_Type) -> Countable_Set is ((Items => [(Low => Elem, High => Elem)] )); ... end class Countable_Set;

10. How to add Mutability?Why Pointer Free? Consider F(X) + G(Y) ... • We want to be able to safely evaluate F(X) and G(Y) in parallel without looking inside of F or G • Presume X and/or Y might be incoming var (in-out) parameters to the enclosing operation • No global variables is clearly pretty helpful • Otherwise F and G might be stepping on same object • No parameter aliasing is important, so we know X and Y do not refer to the same object; use hand-off semantics • What do we do if X and Y are pointers? • Without more information, we must presume that from X and Y you could reach a common object Z • Parameter modes (in-out vs. in, var vs. non-var) don’t help with objects accessible via pointers; need a more elaborate “permission” system

11. ParaSail types, values, declarationswith mutable objects Types and Values with mutable objects: • T ::= ... [as before] • FT ::= ( [locked | queued] [var] Oid1 : T1; ... func Fid1FT1; ...) [ -> [Oid3 : ]T3] • V ::= ... [as before] Declarations and Modules with mutable objects: • D ::= type Tid isT; D ::= func Fid FT [is (V) | is S]; D ::= (const | var) Oid [ : T ] [ := V ]; D ::= M • M ::= [abstract] [concurrent]interface ... [as before] M ::= [concurrent]class ... [as before]

12. ParaSail imperative statementspreserve value semantics • S ::= D// declarations interspersed S ::= N := V | N1 <== N2 | N1 <=> N2// copy, move, swap S ::= FN (V1, V2, ... FV1, FV2, ...) // call for effects on params S ::= ifVthenS1elseS2 end if S ::= block S end block S ::= [whileV] loop S end loop S ::= exit (loop | block) S ::= return [ V ] // or may assign to named output S ::= S1 ; S2// sequential, but OK to parallelize S ::= S1 || S2// parallel; error if data dependence S ::= S1thenS2// force sequential

13. Expandable Mutable Objects Instead of Pointers • All types have additional null value; objects can be declared optional (i.e.null is OK) and can grow and shrink • Eliminates many of the common uses for pointers, e.g. trees • Assignment (“:=“) is by copy • Move (“<==“) and swap (“<=>”) operators also provided • Generalized indexing into containers replaces pointers for cyclic structures • for each N in Directed_Graph[I].Successors loop ... • Region-Based Storage Mgmt can replace Global Heap • All objects are local with growth/shrinkage using local heap • null value carries indication of region to use on growth • Short-lived references to existing objects are permitted • Returned by user-defined indexing functions, for example

14. Mutable Pointer-Free Trees interface Tree_Node<Payload_Type is Assignable> is var Payload : Payload_Type; var Left : optional Tree_Node := null; // defaults to null var Right : optional Tree_Node := null; // defaults to null end interface Tree_Node; var Root : Tree_Node<Univ_String> := (Payload => “Top”, Left => null, Right => null); Root.Left := (Payload => “L”, Right => (Payload => “LR”)); Root.Right <== Root.Left.Right; // Root.Left.Right now null

15. Example: Mutable Countable_Set interface Countable_Set <Element_Type is Countable<>> is op "[]"() -> Countable_Set; op “|=“(var S : Countable_Set; Elem : Element_Type); ... end interface Countable_Set; class Countable_Set is type Elem_Interval is Closed_Interval<Element_Type>; var Items : optional AA_Tree<Elem_Interval>; exports op "[]"() -> Countable_Set is return (Items => []); endop "[]"; op "|="(var S : Countable_Set; Elem : Element_Type) is const IV : Elem_Interval := (Low => Elem, High => Elem); if not Overlapping(S.Items, IV) then S.Items |= IV; end if; end op "|="; ...

16. Region-Based Storage Management • Pioneered by Tofte and Talpin • Refined in Cyclone language • Region (i.e. local heap) in each scope • Space allocated from, and returned to, associated region when expanding and shrinking object • null value identifies region from which to allocate • No garbage accumulates; no asynchronous collector • Region reclaimed in full on exiting scope • effectively => stack-based heap management • Move (“<==“) and swap (“<=>”) very cheap when staying within region (analogous to “mv” in Unix) • Can declare object and give “hint” to where it will be moved: var X for Root : Payload_Type := ... Root.Left.Payload <== X;

17. Stack of Regions with Region Chunks Regions Region Chunks Object handles (no chunks)

18. Regions in a Parallel Programming Language • Global garbage-collected heap bad news in parallel language • Global lock on allocation is serious bottleneck • Concurrent threads allocate unrelated objects in neighboring locations • Individual object has storage spread all over memory • Worst of both worlds (well, all 3 really) • Region provides attractive alternative • Each region can have its own lock • Concurrent threads are allocating in different regions • Individual objects are concentrated in a single region • Better in all dimensions

19. Overall ParaSail Model • ParaSail has four basic concepts: • Module • has an Interface, and Classes that implement it • interface M <Formal is Int<>> is ... • Supports inheritance of interface and code • Type • is an instance of a Module • type T is [new] M <Actual>; • “T+” is polymorphic type for types inheriting from T’s interface • Object • is an instance of a Type • var Obj : T := Create(...); • mutablevalue semantics • Operation • is defined in a Module, and • operates on one or more Objects of specified Types. • are visible automatically based on types of parameters/result

20. Expression and Statement Parallelism • Within an expression, parameters/operands to an operation are evaluated in parallel • Max ( F(X), G(Y) * H(Z) ) • F(X), G(Y), H(Z) evaluated concurrently • Programmer can force parallelism across statements • P(X) || Q(Y) • Programmer can force sequentiality • P(X) then Q(Y) • Default is run in parallel if no dependences • A := P(X); B := Q(Y) -- can run in parallel • Y := P(X); B := Q(Y) -- cannot run in parallel • Work stealing used to schedule parallel computations

21. Example: Recursive Parallel Word Count func Word_Count (S : Univ_String; Separators : Countable_Set<Univ_Character> := [' ']) -> Univ_Integer is // Return count of words separated by given set of separators case Length(S) of [0] => return 0; // Empty string [1] => if S[1] in Separators then return 0; // A single separator else return 1; // A single non-separator endif; [..] => // Multi-character string; divide and conquer const Half_Len := Length(S)/2; const Sum := Word_Count(S[1 .. Half_Len], Separators) + Word_Count(S[Half_Len <.. Length(S)], Separators); if S[Half_Len] in Separators orelse S[Half_Len+1] in Separators then return Sum; // At least one separator at border else return Sum-1; // Combine words at border endif; endcase; endfunc Word_Count;

22. More Examples of ParaSail Parallelism for X => Root then X.Left || X.Right while X notnull concurrentloop Process(X.Data); // Process called on each node in parallel endloop; concurrentinterface Box<Element is Assignable<>> is func Create() -> Box; // Creates an empty box func Put(queuedvar B : Box; E : Element); // waits til empty func Get(queuedvar B : Box) -> Element; // waits til full func Get_Now(locked B : Box) -> optional Element; endinterface Box; type Item_Box is Box<Item>; var My_Box : Item_Box := Create();

23. Synchronizing ParaSail Parallelism concurrentclass Box <Element is Assignable<>> is var Content : optional Element; // starts out null exports func Create() -> Box is// Creates an empty box return (Content => null); end func Create; func Put(queuedvar B :Box; E : Element) is // waits until empty queued until B.Content is null then B.Content := E; end func Put; func Get(queuedvar B :Box) -> Result : Element is// waits until full queued until B.Content not nullthen Result <== B.Content; // “move” sets B.Content to null as side-effect end func Get; func Get_Now(locked B : Box) -> optional Element is return B.Content; end func Get_Now; endclass Box;

24. ParaSail Yin and Yang • Race-Free Parallel Programming • Mutable Value Semantics • Stack-Based Heap Management • Compile-Time Exception Handling

25. Conclusions and Ongoing Work • Simplified, pointer-free type model can still be flexible • safe by construction • Can support new capabilities • pervasive parallelism • integrated annotations checked at compile-time • Ongoing ParaSail work • Finalizing language, interpreter, and work-stealing scheduler • Integrating language with IDE • Building backend to generate compilable C, Ada, LLVM Assembler • Working With Microsoft Research: ParaSail + Dafny • Read the blog and download the prototype ... http://parasail-programming-language.blogspot.com

26. Tucker Taft taft@adacore.com http://parasail-programming-language.blogspot.com +1 (781) 750-8068 x220 AdaCore 24 Muzzey Street Lexington, MA 02421