Class Visibility Errors

1. Class Visibility ErrorsSlide 1

2. Class Visibility Errors Bryan Atsatt February 2009

3. Class Visibility Errors Not Enough Visibility (not-found) • ClassNotFoundException • NoClassDefFoundError Split Visibility (split-packages) • IllegalAccessError (package-private, different loaders) Too Much Visibility (duplicates) • LinkageError (loader constraint violation) • ClassCastException (same name, different loaders) Most developers have not seen duplicate errors… yet. Class Visibility ErrorsSlide 3

4. Duplicate Class DefinitionsTicking time bombs • A duplicate exists when there are two or more Class instances with the same name • Duplicates always have different defining loaders • Frequently result in runtime errors • LinkageError • The JVM enforces "Loading Constraints" (5.3.4) to avoid collisions: "It is essential that any type name N mentioned in the field or method descriptor denote the same class or interface when loaded by L1 and when loaded by L2.“ • ClassCastException • Well known in the wild (e.g."child-first“ loading from Servlet specification, OSGi). Very confusing since type names are same. Cause and effect widely separated in time. Class Visibility ErrorsSlide 4

5. Duplication ErrorsOracle’s experience (>= 2003) • Modules (“shared-libraries”) in application server • Named, versioned loader instances • Versioned imports complex (acyclic) graph • Rich reflective api and diagnostic machinery • Highly successful at solving the versioning problem, but… • Duplication errors suddenly common • No prevention model in place • Fancy diagnostics and education averted disaster Class Visibility ErrorsSlide 5

6. Duplication ErrorsA support nightmare • Subtle conditions, which few developers understand • Class loading is like plumbing: ignored until it breaks, messy when it does and often requiring special expertise to fix • Extremely hard to diagnose • Error messages rarely contain enough information • Loading often deferred until first use, stack trace confusing • Loading can be highly recursive, stack often very deep • Stack trace does not provide instance data, cannot determine which loader(s) are involved • No useful reflective api on ClassLoader • Most loader code pathways are non-debug or native so mostly useless in a debugger Class Visibility ErrorsSlide 6

7. How Are Duplicates Created? • Today • Copies of jars in different loaders • EE implementations that support child-first • Poorly behaved custom loaders • Tomorrow • Multiple versions of a module • Copies of classes in a module that are available in another • Wrong import choice when multiple candidates are available Class Visibility ErrorsSlide 7

8. Modules Change The Balance • Modules should reduce frequency of not-found errors at runtime, but... • Will increase frequency of them at compile time, initially, since dependency declarations must be precise. • Tools can certainly help. • Without intervention, modules will significantlyincrease frequency of duplication errors at runtime. Class Visibility ErrorsSlide 8

9. Modules == More LoadersMore loaders == more opportunities for duplicates • Finer grained class partitioning • A small, simple tree of loaders becomes a potentially large (even cyclic) graph. • Tendency to bundle classes defensively or for convenience • Multiple simultaneous module versions • Unsophisticated use of version requirements • Compiler can catch many error cases, but only for a specific environment. Class Visibility ErrorsSlide 9

10. Error CasesConflicting imports, one module • These cases use an example “animal” module system, with a simple property syntax. Multiple versions of “akita” are present in the repository. pika: Module-Imports=akita, version=1.0; newt; akita, version=0.9 • Module “pika” imports module “akita” twice, “newt” once; silly, but possible. • If not prevented, duplicates from akita will be visible to pika. See AnimalModuleTest.conflictingImportsFail() Class Visibility ErrorsSlide 10

11. Error CasesConflicting module imports == LinkageError vole: Module-Imports=akita, version=0.9 wombat: Module-Imports=vole; akita • Vole ctor takes Akita: public Vole(Akita akita) • Wombat instantiates Vole: new Vole(new Akita()); See AnimalModuleTest.moduleImportsCauseLinkageError() Class Visibility ErrorsSlide 11

12. Error CasesConflicting package imports == LinkageError raven: Imports=akita, version=0.9 shrew: Imports=raven; akita • Raven ctor takes Akita: public Raven(Akita akita) • Shrew instantiates Raven: new Raven(new Akita()); See AnimalModuleTest.packageImportsCauseLinkageError() Class Visibility ErrorsSlide 12

13. Error CasesConflicting imports == ClassCastException newt: Module-Imports=akita, version=0.5 oryx: Module-Imports=akita; newt • Newt ctor publishes Akita instance to static List<Akita>. • Oryx.toString() tries to return loader name for instance in static list. • Unspecified version in oryx selects newer version. • Compiler generated downcast to Akita in Oryx fails. • The same modules do not fail when only the 0.5 version of akita is present. See AnimalModuleTest.moduleImportsCauseClassCast() and AnimalModuleTest.moduleImportsDoNotCauseClassCast() Class Visibility ErrorsSlide 13

14. Error CasesEmbedded copy == ClassCastException newt: Module-Imports=akita, version=0.5 quetzal: Imports=newt • Quetzal contains a copy of classes from akita, same generics downcast results in error when it tries to reference Akita instance from the list in newt. See AnimalModuleTest.embeddedCopyCausesClassCast() Class Visibility ErrorsSlide 14

15. Duplicates Acceptable if Not Shared • If C does not expose classes from B2, either by re-exporting them or by sharing instances, then A‘s import of C will not result in failures. • Duplicates that are hidden implementation details are fine. • If not allowed, A is forced to move to B2, or to find a version of C that does not conflict, neither of which is required here; in a complex system, this behavior would be a serious usability issue. x  y == x imports y Class Visibility ErrorsSlide 15

16. So… rule out all visible duplicates? • Maybe, but first, to detect them, we need to be able to determine what classes or packages are "exposed" by each module. • So assume that we can at least ask each module what packages it exports. • Now, we can collect transitive dependencies, discover all exported packages, and ensure that modules share a common provider where packages intersect. Class Visibility ErrorsSlide 16

17. But… We Can Do Better • If package level imports are possible (ever), then we want a slight refinement. • Given that a single module will likely export many packages, an importer may select a subset of them; if classes from that subset don’t expose the duplicate, that importer is shielded from errors. • If C exports packages x, y and z, and only y refers to classes from B, then importers of x or zdon't care about the duplicate. Class Visibility ErrorsSlide 17

18. Maximize Resolution FlexibilityImplied dependencies • To support this refinement, we need to know what package imports are “implied” by a given package export. • That is, if an exported class uses a class from another package (e.g. in a method signature), as long as we can determine that fact at runtime, we can enforce the no duplicates rule. • And we enforce it at a granularity that maximizes resolution flexibility. Class Visibility ErrorsSlide 18

19. Error Avoided • Identical scenario as raven & shrew, but declare 'implied' dependency: toad: Exports=toad, depends=akita Imports=akita, version=0.9 uakari: Imports=toad; akita • Toad ctor takes Akita: public Toad(Akita akita) • Uakari instantiates Toad: new Toad(new Akita()); • System ensures that toad and uakari both select akita 0.9. See AnimalModuleTest.packageImportsAndDependsDoesNotCauseLinkageError() Class Visibility ErrorsSlide 19

20. Summary • Many failures occur only when multiple versions are present and will therefore manifest very late in the cycle. • Duplication is a double-edged sword: • Useful to avoid forced split of dependency graph when duplicates are hidden implementation details. • Dangerous when duplicate types are surfaced. • Implied dependency declarations enable disambiguation. Class Visibility ErrorsSlide 20

21. Summary, continued. • Early detection is critical to avoid support nightmare: • compile time (one "model" environment) • resolution time (actual environment) • Enumeration of package level exports is required for duplicate detection and warning/failure at resolution time. • Class space "consistency" model is required, but enforcement must be controlled declaratively; tools should generate this information. Class Visibility ErrorsSlide 21

22. Resolution Constraints • A module must fail to resolve if a required import cannot be satisfied. • A failed module must not be used to satisfy imports. • Importing a module M is equivalent to importing all packages exported by M. • A module may declare that it both exports and imports package p; once resolved it may either export or import p, not both. • If multiple providers exist for an import, resolved providers are preferred over unresolved, and higher versions over lower. • Modules that share a dependency must select a common provider or fail; a shared dependency on package s exists between modules M1 and M2 when all of the following are true: • M1 imports package p from M2 • Package p depends on package s • M1 either imports or exports package s Class Visibility ErrorsSlide 22

23. AppendixOSGi ‘uses’ example from R4 specification • Given • A: Import-Package: q; version=”[1.0,1.0]” • Export-Package: p; uses:=”q,r” • B: Export-Package: q; version=1.0 • C: Export-Package: q; version=2.0 • Shared dependency on q means D fails to resolve • D: Import-Package: p, q; version=2.0 Class Visibility ErrorsSlide 23

24. Class Visibility ErrorsSlide 24