User-defined type checkers for error detection and prevention in Java

1. User-defined type checkers for error detection and preventionin Java Michael D. Ernst MIT Computer Science & AI Lab http://pag.csail.mit.edu/jsr308/

2. Code without type qualifiers class DAG { Set<Edge> edges; // ... List<Vertex> getNeighbors( Vertex v) { List<Vertex> neighbors = new LinkedList<Vertex>(); for (Edge e : edges) if (e.from() == v) neighbors.add(e.to()); return neighbors; } }

3. @NonNull type qualifier @NonNullDefault class DAG { Set<Edge> edges; // ... List<Vertex> getNeighbors( Vertex v) { List<Vertex> neighbors = new LinkedList<Vertex>(); for (Edge e : edges) if (e.from() == v) neighbors.add(e.to()); return neighbors; } }

4. @NonNull type qualifier @NonNullDefault class DAG { @NonNull Set<@NonNull Edge> edges; // ... @NonNull List<@NonNull Vertex> getNeighbors( @NonNull Vertex v) { List<Vertex> neighbors = new LinkedList<Vertex>(); for (Edge e : edges) if (e.from() == v) // OK neighbors.add(e.to()); // OK return neighbors; } }

5. @Interned type qualifier @NonNullDefault class DAG { Set<Edge> edges; // ... List<Vertex> getNeighbors(@Interned Vertex v) { List<Vertex> neighbors = new LinkedList<Vertex>(); for (Edge e : edges) if (e.from() == v) // OK neighbors.add(e.to()); return neighbors; } }

6. @ReadOnly type qualifier @NonNullDefault class DAG { Set<Edge> edges; // ... List<Vertex> getNeighbors(@ReadOnly Vertex v) @ReadOnly { List<Vertex> neighbors = new LinkedList<Vertex>(); for (Edge e : edges) if (e.from() == v) neighbors.add(e.to()); return neighbors; } }

7. Using a checker • Write annotations on the code • Or use an inference tool • Compiler plugin for javac javac -processor NonnullChecker MyFile.java • Produces additional errors and warnings • All existing Java checks are performed • Type-system-specific checks • For NonNull: dereferences • For Interned: equality • Error to assign supertype to subtype: myObject = myNonNullObject; // OK myInternedObject = myObject; // error myObject = myReadOnlyObject; // error • Similarly for arguments, returns, overriding, …

8. Outline • NonNull checker • Interned checker • Javari checker • IGJ checker • Creating your own checker

9. NonNull checker • Problem: NullPointerExceptions • Example: Object obj; // might be null @NonNull Object nnobj; // never null nnobj.toString(); // OK obj.toString(); // possible NPE obj = ...; // OK nnobj = obj; // nnobj may become null • Type system:

10. NonNull case study • Programs: • Annotation file utilities (5 KLOC) • Lookup (4 KLOC, twice) • NonNull checker (1 KLOC) • Checkers framework (5 KLOC) • Two different defaults: NonNull and Nullable • Annotations with & without checker available

11. NonNull results • 1 annotation per 50 lines of code • Inference tools exist (for NonNull and other qualifiers) • 44 errors • if (x.y().z()) { somethingOptional(); }where no work need be done if x.y() is null • 83 application invariants • @NonNull indicates whether a variable may be null • @NonNull cannot indicate conditions that hold when a variable is null • entry_re == null iff exit_re == null • 43 tool weaknesses (many already fixed)

12. Comparison with other tools Program to be checked: Lookup (4KLOC) Other tools also find non-NPE bugs To reduce false positives, they discard many warnings

13. Good defaults reduce user effort • Nullable default: • Dictated by backward compatibility • Many instances of @NonNull – too verbose • Flow-sensitive analysis inferred many NonNull types for local variables, reducing annotation burden • NonNull default • Less verbose in signatures • Draws attention to exceptions rather than the rule • More annotations in method bodies • Our system: Nullable only for local variables • Not for generics on local variables • An alternative to type inference; effect may be similar

14. Interned checker • Interning (aka hash-consing, canonicalization) chooses a canonical representative • Saves memory • Permits use of == for comparisons • Problem: Equality errors • Example: String s; @Interned String is, is2; ... is = myString.intern() ... // OK if (is == is2) { ... } // OK if (s == is) { ... } // unsafe equality is = s; // unsafe assignment

15. Interned case study • Program: Daikon (250 KLOC) • Daikon’s key scalability problem is memory • 1170 lines of code/comment refer to interning • 72% of files have none, 87% have 0, 1, or 2 • 200 run-time assertions • Emacs plug-in for String equality checking • 127 annotations in 11 files (12KLOC)

16. Interned results • 9 errors • Missing calls to intern • 2 performance bugs • Unnecessary interning in inner loop of file reading • 1 design flaw • VarInfoName is not interned within its implementation, but all escaping objects are • Too hard to understand complex interning • 14 false positives (used @SuppressWarnings)

17. Javari checker • Problem: Unintended side effects (mutations) • Example: <E> List<E> sort(@ReadOnly List<E> arg) { ... // swap two elements E tmp = arg.get(i); arg.put(i, arg.get(j)); // illegal mutation arg.put(j, tmp); // illegal mutation } • Type system:

18. Javari case study • Programs (8KLOC) • JOlden benchmarks • Javari checker • Some JDK classes

19. Javari results • Mutability bug in Javari checker • Global variable indicating checker state was set on entering an inner class, not reset on exiting • The bug-fix allocates a new object instead • 3 false positives due to varargs • void format(Object...) { … }

20. IGJ checker • Problem: Unintended side effects (mutations) • Type system: • ReadOnly: Reference immutability (aliases may modify) • Immutable: Object immutability (object cannot change) • Syntax uses annotations, not generics

21. IGJ case study • Programs (128KLOC) • JOlden benchmarks • htmlparser library • tinySQL library • SVNKit subversion client • IGJ checker • 4760 annotations (62133 possible locations)

22. IGJ results • Representation exposure errors • Constructors that left object in inconsistent state • 26 false positives (arrays, varargs, casting) • In SVNKit: • some getters have side effects • some setters have no side effects • Used both reference and object immutability • Preferred Annotation IGJ to original dialect: arrays, and • List<ReadOnly, Date<Mutable>> • @ReadOnly List<@Mutable Date> • More opportunities for immutability in new code than old

23. Writing your own checker • Base checker performs all standard checks (assignment, method arguments) • Override only a couple methods, for special rules • For NonNull: dereferences • For Interned: equality • Check what you want: • General properties (tainting) • Project-specific (string formatting)

24. JSR 308: Annotation on Java types • Permits annotations in new locations List<@NonNull String> myList; class Properties extends @Immutable Map {…} • Planned for inclusion in Java 7 • Implementation is backwards compatible: • Write annotations in comments: List</*@NonNull*/ String> myList; /*@NonNull*/ String s; • Compiles with any Java compiler

25. Pluggable type system benefits • Easy to use • Scalable • Reveals errors • Prevents even more errors • Download the checkers from http://pag.csail.mit.edu/jsr308/ • Prerelease of annotation extensions in Java 7 • Backward-compatible implementation • Includes inference tools for some type checkers • Includes a framework for writing new checkers