A Type System for Higher-Order Modules

1. A Type System forHigher-Order Modules Derek Dreyer, Karl Crary, and Robert Harper Carnegie Mellon University POPL 2003

2. Type Theory for Module Systems • Lots of work on module system design • Theory has had impact on real language design: • Harper-Lillibridge 94, Leroy 94 ) SML ’97 • Leroy 95 ) Objective Caml • Russo 00 ) Moscow ML • No general semantic framework for understanding relationships between designs

3. A Unifying Type Theory • High-level semantic analysis ) Unifying type theory • Previous designs can be seen as subsystems • Key idea: Explain semantics of abstract data types in terms of purity and effects

4. Projectibility • When is a module expression M projectible? • When can we project out M’s type components? • “Non-projectible” module expression:if buttonIsPressed() then struct type t = int ... end else struct type t = bool ... end • “Projectible” module expression:struct type t = int; val x = 3 end

5. Projectibility • When is a module expression M projectible? • When can we project out M’s type components? • “Non-projectible” module expression:if buttonIsPressed() then struct type t = int ... end else struct type t = bool ... end • “Projectible” module expression:struct type t = int; val x = 3 end

6. Projectibility • When is a module expression M projectible? • When can we project out M’s type components? • “Non-projectible” module expression:if buttonIsPressed() then struct type t = int ... end else struct type t = bool ... end • “Projectible” module expression:struct type t = int; val x = 3 end

7. Projectibility • When is a module expression M projectible? • When can we project out M’s type components? • “Impure” module expression:if buttonIsPressed() then struct type t = int ... end else struct type t = bool ... end • “Pure” module expression:struct type t = int; val x = 3 end

8. Projectibility • When is a module expression M projectible? • When can we project out M’s type components? • “Impure” module expression:if buttonIsPressed() then struct type t = int ... end else struct type t = bool ... end • “Pure” module expression:struct type t = int; val x = ref 3 end

9. Purity • M is soundly projectible ,M is pure (w.r.t. type components), Type components of M are the same every time M is evaluated • M is impure ) Meaning of M.t not statically well-determined

10. Second-Class Modules • Second-class modules admit “phase separation” • Type components can’t depend on run-time conditions • SML and O’Caml modules are second-class because of syntactic restrictions • All second-class modules are pure • But should they all be projectible?

11. Sealing • Principal means of creating abstract data types • M :>s, aka “opaque signature ascription” • Treating sealed modules as projectible violates abstraction: • A = (M :>s) and B = (M :>s) • If (M :>s) is projectible, then A.t = (M :>s).t = B.t • But “A.t = B.t” not observable in s

12. You Can’t Handle the Truth! • In truth, sealing doesn’t affect a module’s purity • But sealing obstructs our knowledge about a module’s purity • Projectibility is a judgment of knowledge, not truth: • Sealed modules treated as impure/non-projectible

13. Total and Partial Functors • To track purity in the presence of functors: • Need to know whether applying a functor will unleash an effect or not • Distinguish types of total and partial functors: • F : Ptot s:s1.s2, body of F is known to be pure • F : Ppar s:s1.s2, body of F could be impure

14. Total , Applicative • F : Ptot s:s1.s2, M : s1, F and M are pure • structure Res1 = F(M) • structure Res2 = F(M) • F(M) known to be pure ) projectible Res1.t = F(M).t = Res2.t

15. Partial , Generative • F : Ppar s:s1.s2, M : s1, F and M are pure • structure Res1 = F(M) • structure Res2 = F(M) • F(M) possibly impure ) non-projectible Res1.t ¹ Res2.t

16. Functors with Sealing • If body of a functor contains sealing, then: • Body is impure • Functor is generative • Can be both a good and bad thing: • Gives correct semantics of abstraction for functors that use imperative features • Overly restrictive for purely functional functors

17. Importance of Generativity functor MakeSymbolTable() = (struct ... (* creates new mutable hash table *) end :> sig type symbol val string_to_symbol : string -> symbol val symbol_to_string : symbol -> string end) ) • Generativity ties the symbol type to the run-time state of the module defining it

18. Purely Functional Abstraction functor MakeSet (Elem : COMPARABLE) = (struct ... end :> sig type elem = Elem.elem type set val insert : elem * set -> set end) • What if a sealed module is purely functional? • Abstract types not tied to any run-time state • Only care about hiding type identity

19. Hoisting the Sealing • Instead of sealing the body of the functor: l s:s1. (M :>s2) • Seal the functor itself (with a total signature): (l s:s1. M) :>Ptot s:s1.s2 • Problem: Only works if M is pure • Not true if M contains sealed substructures,such as datatype definitions

20. Module Classifications Impure, non-projectible Pure, projectible Sealing M :>s

21. Static and Dynamic Effects • Split effects into two kinds: static and dynamic • Module with any kind of effects is impure • Dynamic effects occur “during execution” • Static effects occur “during typechecking”

22. Weak and Strong Sealing Statically and dynamically impure Impure, Non-projectible Statically impure, dynamically pure Weak Sealing M ::s Strong Sealing M :>s Pure, projectible

23. Set Functor Revisited functor MakeSet (Elem : COMPARABLE) = (struct ... end :: sig type elem = Elem.elem type set val insert : elem * set -> set end) • We expand totality to allow body of a total functor to contain static (but not dynamic) effects

24. A Unifying Framework • Standard ML • Only has strong sealing, all functors are partial/generative • Objective Caml / Leroy (1995) • Only has weak sealing, all functors are total/applicative • Shao (1999) • Distinguishes two kinds of functor signatures • Only tracks dynamic effects and strong sealing • Russo (2000) • Two languages, one like SML and one like O’Caml • Moscow ML combines them, but language is unsound

25. Modules as First-Class Values • Packaging modules as first-class values: • Add a new “package type” <s> • Coercions between modules and terms: • If M : s, then pack M as <s> : <s> • If e : <s>, then unpack e as s : s

26. Modules as First-Class Values structure X = struct type t = int ... end structure Y = struct type t = bool ... end M = (if buttonIsPressed() then X else Y) • Type components of M actually depend onrun-time conditions • Unpacking induces a truly dynamic effect

27. Modules as First-Class Values signature S = sig type t ... end structure X = struct type t = int ... end structure Y = struct type t = bool ... end M = unpack (if buttonIsPressed() then pack X as <S> else pack Y as <S>) as S • Type components of M actually depend onrun-time conditions • Unpacking induces a truly dynamic effect

28. The Rest of the Paper • Formalism: • Synthesis of previous work on dependent types and singleton kinds • Fully expressive higher-order functors • Via “static” module equivalence • Decidable typechecking algorithm • Avoidance problem • Restrict type theory (as little as possible) to avoid it • Unrestricted language definable by elaboration

29. Conclusion • Future Work: • Recursive modules • Using monads instead of total/partial • We’ve provided a framework for understanding: • Alternative module system designs • Semantics of abstraction, via a framework of module-level effects