Taming the Wildcards: Combining Definition- and Use-Site Variance

1. Taming the Wildcards:Combining Definition- and Use-Site Variance John Altidor

2. Motivation for Variance. Existing Approaches. Java only supports use-site variance. C# and Scala only support definition-site variance. What Is New Here:Combine Definition-Site and Use-Site Variance. Both in a single language, each using the other Case Study – Inferring Def-Site for Java. Insights into Formal Reasoning. Summary. Outline

3. Software Reusability • Writing reusable software is hard; many factors: • Generality • Adaptability • Modularity • Simplicity of interface / Ease of use • … • Programming language support • Enable reusable code without introducing bugs. • Focus on integrating two flavors of genericity: • Generics – Parametric Polymorphism • Subtyping – Inclusion Polymorphism

4. Example: Java inheritance. Subtyping – Inclusion Polymorphism class Animal { void speak() { } } class Dog extends Animal { void speak() { print(“bark”); } } class Cat extends Animal { void speak() { print(“meow”); } }

5. Subtyping – Example Client void performSpeak(Animal animal) { animal.speak(); } Can be a Dog, Cat, or any other Animal.

6. Common Code Structure class ListOfDogs { void add(Dognum) { … } Dogget(inti) { … } int size() { … } }

7. Common Code Structure (cont.) class ListOfAnimals { void add(Animalnum) { … } Animalget(inti) { … } int size() { … } }

8. Generics – Parametric Polymorphism type parameter class List<X> { void add(Xx) { … } X get(inti) { … } int size() { … } } write X read X no X • List<Animal> ≡ List of Animals • List<Dog>≡List of Dogs Customized Lists

9. Dog <: Animal (Dog is an Animal). Cat <: Animal (Cat is an Animal). List<Dog> <: List<Animal> Generics and Subtyping No! • A List<Animal> can add a Cat to itself.A List<Dog> cannot.

10. When is C<Expr1> a subtype of C<Expr2>? Variance Introduction class RList<X> { X get(inti) { … } int size() { … } } Can read from but not write to. • It is safe to assumeRList<Dog> <: RList<Animal>. • Why?

11. Assuming Dog <: Animal (Dog is an Animal). Flavors of Variance - Covariance Generic<Dog> <: Generic<Animal> Covariance

12. Assuming Dog <: Animal (Dog is an Animal). Flavors of Variance - Contravariance Generic<Dog> <: Generic<Animal> Contravariance

13. How do programmers specify variance? Four Flavors of Variance

14. Programmer specifies variance in definition as in Scala and C#. Variance of a type position. Return types: covariant. Arguments types: contravariant. Definition-Site Variance (C#/ Scala) class WList<-X> { void add(Xx) { … } int size() { … } // no method to get } class RList<+X> { X get(inti) { … } int size() { … } // no method to add }

15. Use-Site Variance (Java Wildcards) class List<X> { void add(X x) { … } X get(inti) { … } int size() { … } }

16. Use-Site Variance (Java Wildcards) List<? extends X> class List<X> { void add(X x) { … } X get(inti) { … } int size() { … } }

17. Use-Site Variance (Java Wildcards) List<? super X> class List<X> { void add(X x) { … } X get(inti) { … } int size() { … } }

18. Use-Site Variance (Java Wildcards) List<?> class List<X> { void add(X x) { … } X get(inti) { … } int size() { … } }

19. Definition-Site: Pros • Conceptual Simplicity • Simpler Type Expressions – no use-site annotations. • Subtype policy stated in class definition. • Burden on library designers; not on users. • Classes declare variance once and for all uses. covariant class RList<+X> { ... } class WList<-X> { ... } class List<X> extends RList<X>, WList<X> { ... } contravariant invariant

20. Definition-Site: Cons • Redundant Types • scala.collection.immutable.Map<A, +B> • scala.collection.mutable.Map<A, B> • Generic with n parameters ⇒ 3n interfaces (or 4n if bivariance is allowed) covariant class RList<+X> { ... } class WList<-X> { ... } class List<X> extends RList<X>, WList<X> { ... } contravariant invariant

21. Use-Site: Pros • Flexibility: • co-, contra-, bivariant versions on the fly. • Easier on library designers • Design classes in natural way • No need for fractured classes. class List<X> { void add(X x) { … } X get(inti) { … } int size() { … } }

22. Use-Site Cons: • Burden shifts to users of generics. • Library designers are often users as well. • Type signatures quickly become complicated. • Heavy variance annotation required for subtyping. • From ApacheCommons-Collections Library: Iterator<? extendsMap.Entry<? extends K, V>> createEntrySetIterator( Iterator<? extendsMap.Entry<? extends K, V>>)

23. “We simply cannot afford another wildcards” – Joshua Bloch. “Simplifying Java Generics by Eliminating Wildcards” – Howard Lovatt. Wildcards Criticism Iterator<? extendsMap.Entry<? extends K, V>> createEntrySetIterator( Iterator<? extendsMap.Entry<? extends K, V>>)

24. Take advantages. Remove disadvantages. Simpler type expressions than in Java(burden off clients). Less redundant type definitions in C# and Scala. Inferring definition-site variance from only use-site annotations. [1] Added notion of def-site variance without extending Java. VarLang Calculus: Denotational and language neutral approach. [1] PLDI 2011 (Altidor, Huang, Smaragdakis) Our Approach: Take Best of Both Worlds

25. Fewer Wildcard Annotations Iterator<? extendsMap.Entry<? extends K, V>> createEntrySetIterator( Iterator<? extendsMap.Entry<? extends K, V>>) Iterator<Map.Entry<K, V>> createEntrySetIterator( Iterator<Map.Entry<K, V>>)

26. VarLang Calculus • Language neutral approach. • Simplifies and generalizes all previous work. • Sample Applications: • Infer definition-site variance for Java. • Add use-site variance to C#.

27. Variance Lattice • Ordering details later

28. Java to VarLang Calculus class C<X> { X foo(C<? super X> arg1) { ... } void bar(D<? extends X> arg2) { ... } } class D<Y> { void baz(C<Y> arg3) { ... } } module C<X> { X+, C<-X>-, void+, D<+X>- } module D<Y> { void+, C<oY>- }

29. Infer Definition-Site Variance • Def-Site Variance of C<X> = c = +. (C is covariant) • Def-Site Variance of D<Y>= –.(D is contravariant) module C<X> { X+, C<-X>-, void+, D<+X>- } module D<Y> { void+, C<oY>- }

30. Case Study: Definition-Site Inference for Java • How much benefit if Java’s type system inferred definition-site variance? • Mapped Java classes to VarLang modules. • Applied inference to large, standard libraries • e.g., Sun’s JDK 1.6 • Example inferences • java.util.Iterator<E> is covariant. • java.util.Comparator<T>is contravariant.

31. Sample Results from Case Study • Analysis was conservative (e.g. ignored method bodies). • “foo(List<Animal> arg)” could have been“foo(List<? extends Animal> arg)”. majority

32. Insights into Formal Reasoning • Variance composition: v1⊗ v2 = v3 • Variance binary predicate: v(T; T’) • Variance lattice: v1≤ v2 • Variance of a type:var(X; T) • Relating variance to subtyping: Subtype Lifting Lemma • Variance of a position

33. Ordered by subtype constraint Standard Modeling: Variance Lattice binary predicate

34. When is C<Expr1> a subtype of C<Expr2>? What about existential types?∃X->[⊥-String].Stack<X> We answer a more general question:When is [U/X]T <: [U’/X]T? Key: defined very general predicate:var(X; T)= variance of type T with respect to type variable X. Variance of a Type

35. var(X; Iterator<X>) = +and +(Dog; Animal) ≡ Dog <: AnimalimpliesIterator<Dog> <: Iterator<Animal> Subtype Lifting Lemma • We generalize Emir et al.’s subtype lifting lemma. • Goal property of var. If: v ≤var(X; T) v(U; U’) Then: [U/X]T <: [U’/X]T

36. Variance of variable X in type A<B<C<X>>>? In general, variance of variable X in type C<E>? v1⊗v2 = v3. If: Variance of variable X in type expression E is v2. The def-site variance of class C is v1. Then: variance of X in C<E> is v3. Variance Composition Transform Operator

37. Example Case: + ⊗ – = – Class C is covariant. Type E is contravariant in X. Need to show C<E> is contravariantin X. For any T1, T2: Deriving Transform Operator • Hence,C<E> is contravariantin X.

38. Invariance transforms everything into invariance. Bivariance transforms everything into bivariance. Covariance preserves a variance. Contravariance reverses a variance. Summary of Transform

39. Definition-site variance annotations are type checked using varpredicate. For further details: [2] ECOOP 2012 (Altidor, Reichenbach, Smaragdakis) Definition of var predicate

40. General Theory – Template for Adding Variance • Variance composition: v1⊗ v2 = v3 • Variance binary predicate: v(T; T’) • Variance lattice: v1≤ v2 • Variance of a type:var(X; T) • Relating variance to subtyping: Subtype Lifting Lemma • Variance of a position: See [2] for derivation. A<B<C<X>>> Subtype Policy Comparing Variances Computing Variances Variance Soundness Check Def-Site Variance

41. Generics and subtyping coexist fruitfully. Subtyping between two different instantiations. Combine def-site and use-site variance to reap their advantages and remove disadvantages. Generalize all previous related work. Resolve central questions in the design of any language involving parametric polymorphism and subtyping. Variance of a type. Variance of a position. Summary of Contributions

42. To my collaborators: • Smaragdakis, Reichenbach, Huang, Palsberg, Tate, Cameron, Kennedy, Urban • To CLC for hosting.