Liberating Programming from von Neumann Style

1. David Evans http://www.cs.virginia.edu/~evans Lecture 16: Should Programming Be Liberated from the von Neumann Style? CS655: Programming Languages University of Virginia Computer Science

2. Menu • What is Functional Programming? • Higher Order Functions • Recursion Elimination • Type Inference (Intro) University of Virginia CS 655

3. What is Functional Programming? • Programming without assignment • Programming without state (same thing) • Not just a state of mind (unlike structured programming, data abstraction, object-oriented, etc.) • Lack of assignment/mutation makes a language functional • Language features associated with functional languages: lambda abstraction, higher-order functions, type inference, etc. University of Virginia CS 655

4. LISP • LISP (MIT, 1958) • LISt Processor • Not “Functional” (has setq, set!, etc.) • Many descendants: • Scheme (1975, Steele & Sussman) – simple language, popular in intro programming courses • Common Lisp – huge and complicated • Syntax simple for machines, but hard for humans (defun fact (lambda (x) (if (eq? x 0) 1 (* (x (fact (- x 1)))))) University of Virginia CS 655

5. Post-LISP “functional” languages • FP – Backus 1978 • FL (1989) – designed to have useful and simple semantics • ML – “MetaLanguage” (Milner, 1970s) • Most successful (?) modern “functional” language • Amazon lists 22 ML books; used in hundreds of university courses; many PLDI/POPL papers, etc. • ML2000 committee (Andrew Appel, Luca Cardelli, Robert Harper, John Mitchell, Jon Riecke, etc.) • Job listings at monster.com: LISP: 41, C++: > 1000 (638 in just Virginia) 0 • (2 post-docs posted on comp.lang.ml) University of Virginia CS 655

6. von Neumann Bottleneck Processor Loads and Assignments Store University of Virginia CS 655

7. Multiple processors and stores don’t help Processor Processor Processor Store Store Store Backus: Programs written with state are hard or impossible to make concurrent. So let’s get rid of the state. University of Virginia CS 655

8. Are most modern machines really von Neumann machines? • No! Even single-processor Pentium has out-of-order, multiple instruction execution, multiple in/out register file, on-chip cache • Must do lots of clever things in hardware to provide von Neumann machine illusion to programmers University of Virginia CS 655

9. Advantages of Functional Programming • No side-effects • Referential Transparency: rewriting possible • Concurrency Possible: order of evaluation doesn’t matter • Simple semantics • Conceptual clarity • No superfluous names introduced • No need to “simulate” executions to understand programs University of Virginia CS 655

10. Referential Transparency (a + b) * (a + b)  (let (x = a + b) (x * x)) In C: (++a + (i = i * 2, b)) * (++a + (i = i * 2, b))  (x = ++a + (i = i * 2, b), x * x) University of Virginia CS 655

11. Disadvantages of Functional Languages • Acceptable performance is hard • Far from machine architecture • Most real programs are hard to express without state • Imagine a bank account example • End up having to explicitly/implicitly pass around lots of state, giving up benefits of functional programming University of Virginia CS 655

12. Functional Languages Taxonomy • “Pure” – really functional • FL, Miranda, Haskell • “Impure” – not really functional: have state • ML, Scheme • Some people will claim there are degrees of purity • Once you add state, you give up the main advantage of functional languages University of Virginia CS 655

13. Lazy/Strict Evaluation • Lazy (Haskell) • Only evaluate an argument when it is needed • Strict (FL) • Always evaluate every argument • Does this matter without side effects? • Yes! Consider: s1 o [~1, nt] where { def nt  nt } University of Virginia CS 655

14. Higher-Order Functions • Zero-order functions • Mapping from nothing to value (regular data) • First-order functions • Functions that operate on zero-order functions (e.g., int  Boolean, int * int  Boolean * int) • Second-order functions • α β where α and β are first-order functions (e.g., int  (int  Boolean) (define f (lambda (x) (lambda (y) (= x y)))) (f 3) returns a function that returns true when applied to 3. • Higher-order functions • Functions operate on functions of any order University of Virginia CS 655

15. Higher-Order Functions Scheme example (define map (lambda (f) (lambda (a) (if (null? a) a (cons (f (first a)) (map f (tail a))))) (map (lambda (x) (* x 2)) ‘(1 2 3 4)) University of Virginia CS 655

16. Comparison Example def average  + / length where { def length  + o aa:~1 } proc average (vals: array[int]) returns float { int sum := 0; for (int i := 0; i < vals.size; i++) sum := sum + vals[i]; return (float) sum / (float) vals.size; } University of Virginia CS 655

17. Disadvantages • average:<1000000-element sequence> • Requires copying 1000000 element sequence (ouch!) • Looks at every element 3 times (once for +, once for aa:~1, once for +) • average(100000-element array) • Requires passing a reference • Looks at every element once University of Virginia CS 655

18. Eliminating Copies def average  div o [s2, s1] o /l:sumlength where def sumlength  issingleton  [ ~1, s1]; [ s1 o s2 + ~1, s1 + s2 o s2] def issingleton  isseq  eq o [length, ~1] • average:<2,4> • (div o [s2, s1] o /sumlength):<2, 4> • (div o [s2, s1]):<sumlength:<2, sumlength:<4>>> • (div o [s2, s1]):<sumlength:<2, <1, 4>> • (div o [s2, s1]):<2, 6> • div:(<6, 2>) • 3 YUCK! University of Virginia CS 655

19. Functional Languages Issues • Can we write programs functionally but use really smart compilers to get good performance? • Despite much effort, answer seems to be no in general (so far). (ML folks disagree, but its a “mostly functional” language.) • Are functional programs really better for humans? • Depends on the humans, no for most (?) University of Virginia CS 655

20. Rewrite Rules • Can the compiler rewrite /l:* with *? /l:*:<2 ,3 ,4> = *:<2, *:<3, *:<4>>> = *:<2, 3, 4> • Not if we care about errors: *:4  error:<“*”, “arg1”> /l:*:4  error:<“/”, “arg2”> • Safer rewrite rule: /l:*  isseq  * ; error:<“/l”, “arg2”> • What if errors included argument value? *:<`a, 4>  error:<“*”, “arg1: <`a, 4>”> University of Virginia CS 655

21. Eliminating Recursion define fact  isnonneg  isint  eq o [~0, id]  ~1 ; * o [id, fact o - o [id, ~1]] ; error:<“fact”, “arg1”> same as (?): define fact  isnonneg  isint  /l:* o intsto ; error:<“fact”, “arg1”> Can a compiler figure this out? University of Virginia CS 655

22. General Rule def f isposint  iszero  g1 ; g2 o [f o dec, g3] ; error  def f  isposint  (/l:(g2 o [s1, g3 o s2])) o al o [g1 o ~0, intsto] ; error University of Virginia CS 655

23. Generalizing def f iszero  g1 ; g2 o [f o dec, g3]  def f  isnum  isnonnegint  (/l:(g2 o [s1, g3 o s2])) o al o [g1 o ~0, intsto ] ; greater o [id, ~0]  eq o [floor, id]  (/l: (g2 o [s1, g3 o s2])) o al o [g1 o ~0.0, aa:(div o [id, ~1]) o intsto o floor] ; nt ; nt ; signal o [~“sub”, ~“arg1”, [id, ~1]] University of Virginia CS 655

24. Proof by Handwaving • Use rewrite-rules to rewrite replacement into original • Depends on fixed point machinery • Must show the fixed point is unique • Can automate most of proof (rewriting) • Requires150 steps and 953 rule firings • Can make up rules that catch most user factorial programs (but don’t work as well for real programs) University of Virginia CS 655

25. Rewrite Rules Summary • Depends on referential transparency [s1, s2] o [f, g]  [g, f] is safe because there are no side effects. (Wouldn’t be safe in ML). • Handling errors is a major pain – FL semantics demands same error behavior University of Virginia CS 655

26. Type Inference • Determine types at compile time without explicit declarations • Used to improve performance (FL), warn programmer of errors (ML, but not in FL) • Can be expensive • Depending on type system, may be exponential in size of program! • Small changes in type system make a big difference (FL is much worse for this than ML, because have to deal with errors) University of Virginia CS 655

27. Human Type Inference id x.x def length  isnull  ~0 ; + o [~1, length o tl] def f  + o [s1, s2] University of Virginia CS 655

28. Operational Type Inference • Types are just sets of values • Theory: derive a system of type constraints from the program, and solve for types • Practice: derive types bottom-up, starting with rules for primitives • Just like operational semantics, except transition rules on types instead of values • In FL, everything has type 1 (set of all possible values), so a simple type inference algorithm assigns type to 1 everything University of Virginia CS 655

29. FL Transition Rules X SeqOf (Int)   [ADD-INT] +:XFLX where :SeqOf (Int)  Int calculates the sum of a sequence of integers. V SeqOf (Int)   [TADD-INT] +:VTInt University of Virginia CS 655

30. Add Type Rewrite Rules V SeqOf (Int)   [TADD-INT] +:VTInt V SeqOf (Num)  SeqOf (Int)   [TADD-NUM] +:VTNum V SeqOf (Num)   [TADD-ERR] +:VT <err, [“+”, VSeqOf (Num)]> University of Virginia CS 655

31. Combined Rewrite Rule _______________________________ [TADD] +:VT (if V SeqOf (Int)   then Int else )  (if (V SeqOf (Num) SeqOf (Int))   then Num else ) • (if VSeqOf (Num) then < err, [“+”, VSeqOf (Num)]> else ) University of Virginia CS 655

32. sn Type Rewrite Rules V <α, ...>   [Ts1-SEQ] s1:VTα V <α, ...>   [Ts1-ERR] s1:VT <err, [“s1”, V <α, ...>]> V <α, β, ...>   [Ts2-SEQ] s2:VTβ V <α, β, ...>   [Ts2-ERR] s1:VT <err, [“s2”, V <α, β, ...>]> University of Virginia CS 655

33. Applying Rewrite Rules def f  o:<+, cons:<s1, s2>> f:x T o:(<+, cons:<s1, s2>>):x T +:(cons:<s1, s2>:x) T +:(cons:<s1:x, s2:x>) T +:<s1:x, s2:x> if x <α, β, ...>   T +:<α, β> [Ts1-SEQ], [Ts2-SEQ] if <α, β> SeqOf (Int)   T Int [TADD-INT] f has type (<Int, Int, ...>  Int)  (<Int, Int, ...>  1) University of Virginia CS 655

34. Other Rule Choices if <α, β> SeqOf (Num)  SeqOf (Int)   +:<α, β>T Num [TADD-NUM] if <α, β> SeqOf (Num)   +:<α, β>T <err, [“+”, VSeqOf (Num)]> [TADD-ERR] if x <α, ...>   s1:xT <err, [“s1”, V <α, ...>]>[Ts1-ERR] if x <α, β, ...>   s2:xT <err, [“s2”, V <α, β, ...>]>[Ts2-ERR] University of Virginia CS 655

35. Join All Choices f:V T (if V  <Int, Int, ...>  then Int else )  (if V  <Num, Num, ...>    (V  <Int, Int, ...>   ) then Num else )  (if V  < α, ...> then < err, [“s1”, V < α, ...>]> else )  (if V  < α, β, ...> then < err, [“s2”, V  < α, β, ...>]> else )  (if V  SeqOf(Num)then < err, [“+”, V  SeqOf(Num)]> else ) University of Virginia CS 655

36. Type of f f: <Int, Int, ...> Int  <Num, Num, ...>  <Int, Int, ...>  Num  <Num, Num, ...>  <err, α> University of Virginia CS 655

37. Termination • Regular (value) operational semantics won’t terminate for recursive definitions: def rev  isnull  []; ar o [rev o tl, s1] • rev:Seq T V corresponds to proof of rev :vFL v’ (which has no finite proof!) • But, should be able to derive rev: SeqOf(α)TSeqOf(α)  SeqOf (α)T <err, [“tl”, β]> University of Virginia CS 655

38. Termination Solution • Use heuristics to find “most general” solution • Not very satisfying – reason why most people don’t do type inference this way (use denotational semantics instead) • Coming up...after mock trial University of Virginia CS 655

39. Charge • Project Reports due today (11:59pm) • Read through PS3 • Today’s readings – don’t skim them! University of Virginia CS 655