Flexible Dynamic Linking for .NET

1. Flexible Dynamic Linkingfor .NET Anders Aaltonen Alex Buckley Susan Eisenbach Acknowledgement: Microsoft Research

2. Agenda • Introduction to dynamic linking • Flexibility and safety at link-time • Developer-centric flexibility • Implementation • Conclusion

3. Dynamic linking Turns ldfldMemberDescriptor generated at compile-time into ldfld0x100405 in the run-time environment by using assembly and class definitions

4. Dynamic linking • Is taken as given in modern execution environments • Saves space by sharing code between programs • Enables configurable binding policies • User/system-wide upgrades (v1.0 → v2.0) • Servicing policy (v1.1.9 → v1.2.0) • Unification policy (use version corresponding to the CLR) • Local and remote probing (check GAC, then URL) • Supports bytecode verification on the user’s machine

5. call void[mscorlib]System.Console::WriteLine(string) Linker MSCorLib v1.1.1322 PublicKey=99999 Culture=US Calc v1.2.3.4 PublicKey=12345 Culture=UK Assembly metadata import mscorlib = MSCorLib, v1.1.1322, PK=9999, Culture=US Assembly metadata Class metadata export System.Console IL code Class metadata export A,B,C,D IL code namespace System;class Console {void WriteLine(string s){…}}

6. Linking is constrained by compiler decisions Console.WriteLine(“Hi”); Known to the compiler call void[mscorlib]System.Console::WriteLine(string) Available at runtime call void[monolib]System.Console::WriteLine(string)

7. Platform- or vendor-specific assemblies might be available at runtime only • Generic ODBC v. SQLServer ODBC • Microsoft FTP v. third-party SecureFTP • Imperial’s LTSA model-checkercan use non-redistributable NASA algorithms • Avoid compile-time dependencies on NASA code? • (Separate compilation still checks dependencies) • Have to use error-prone reflection

8. How can I bind to the runtime environment when the compiler forces its choices on me?orInstead of the database library,how can I target a database library?

9. Initial idea: make bytecode more flexible Assembly type variable call void [X]System.Console::WriteLine(…) call void [mscorlib]System.Console::WriteLine(…) Class type variable call void [mscorlib]Y::WriteLine(…) call void [mscorlib]System.Console::WriteLine(…) Substitute compiler’s X with mscorlib at link-time Substitute compiler’s Y at link-time

10. Flexible bytecode: True late binding • Extend Fusion servicing policies to re-naming • Use “generic” classes without naming a specific assembly • Different versions of an assembly: Development/Testing/Production/Archive • Develop binary components off-site, using stub assemblies,then execute on-site, using full assemblies • Simplified command-line compilation (fewer /r: args) • Use a specific assembly without mentioning a class • Code to the interface, not the implementation[mscorlib]List l = new [mscorlib]X();

11. Flexible bytecode: Orthogonal to generics • Generics make an interface type-safe, but any given type parameter needs a specific implementation • Type parameters in a generic class are determined by a particular (dynamic) instantiationList<Person> l = new ArrayList<Person>(); • Type variables in flexible bytecode are determinedby arbitrary agents (VM, programmer, user)with different scopes (global, assembly, class)

12. Flexible bytecode: Type-safety • The substitute for an assembly or class should provide members used by the programmer, e.g. • Assembly X provides interface List • Interface List provides ‘void compare(List l);’ • Class Y is a subclass of/implements List So: • Collect constraints from bytecode • Search the GAC for suitable assemblies at run-time • OK?

13. Problems with constraints • Subtype constraints require data-flow analysis • Unreachable code may be over-constraining • Constraints say nothing about behaviour • Debugging is impractical if truly arbitrary components are chosen at run-time

14. Semantic substitutions • Policy: Programmer must know valid assemblies/classes • Custom attributes declare possible substitutions [LinkAssembly(A1, A2)] [LinkAssembly(A1, A3)] [LinkClass(C1, C2)] [LinkClass(C1, C3)] • Any assembly or class can be independently substituted →All types are type variables • Need member and subtype constraintsto avoid resolution errors (but not guide substitutions) [LinkMember(A1, C1, ‘B m(D)’)] [LinkSubtype(A1, C1, A2, C2)]

15. Attribute scoping • We do not always wantflexiblebytecode • Minimise impact by choosing the right scope [assembly: LinkClass(C,…)] [LinkClass(C,…)] class App { void m() { …C… } // Search class,assembly [LinkClass(C,…)] void n() { …C…} // Use method’s [LinkClass] } • Only the most local scope is used

16. Substitution interfaces • Group substitutions by platform/vendor/maturity: [LinkAssembly(A1,A1_32,”win32”)] [LinkAssembly(A2,A2_32,”win32”)] [LinkAssembly(A1,A1_64,”win64”)] [LinkAssembly(A2,A2_64,”win64”)] • On a Win32 machine, only the A1→A1_32 and A2→A2_32 substitutions will be possible • Once A1 or A2 has been substituted, we should stay within the “win32” interface

17. Interface policies [LinkAssembly(A1,A1_32,”win32”, LOCAL_INTERFACE)] Demand “win32” [LinkAssembly]+[LinkClass] substitutions [LinkAssembly(A1,A1_32,”win32”, LOCAL_INTERFACE_PREFERRED)] Try “win32” substitutions first, but allow others on failure [LinkAssembly(A1,A1_32,”win32”, LOCAL_INTERFACE_EAGER)] Eagerly check that all “win32” substitutions will succeed [LinkAssembly(A1,A1_32,”win32”,ANY_INTERFACE)] No restrictions on later substitutions

18. Preparing bytecode for flexible linking Hello.cs Hello.il [assembly: LinkClass] [LinkClass] class App { void m() { … } [LinkClass] void n() { … } } .assembly Hello { .custom instance LinkClass .custom instance LinkMember .class App extends … { .custom instance LinkClass .custom instance LinkMember ILASM Metadata is backward - compatible Infer member constraints Hello.exe Assembly metadata Hello.il Class metadata Compiler .assembly Hello { .custom instance LinkClass .class App extends … { .custom instance LinkClass IL code ILDASM Must be able to compile in some default environment Avoid source code: compilers are hard to change, and there are many of them

19. Just-In-Time substitution • CLI-compliant linking is very flexible • Verification is optional • Resolution may occur as early as install time, as late as execution time • CLR/SSCLI is lazy: resolution and verification happen at JIT-compilation • Extend resolution to handle [Link*()] attributes • “JIT-substitution”

20. Modifying the SSCLI x86 code Verifier/JIT compiler Attribute collection CEEInfo::findClass/Field/Method Resolution cache Standard resolution Constraint verification FDL resolution Assembly/class loading Fusion Filesystem

21. Attribute collection • A LinkContext encapsulates a single resolution attempt, e,g, call [A]C::m … = A fully-qualified member reference needing resolution + Nearest [LinkAssembly] and [LinkClass] attributes in scope + Set of constraints applying to these attributes • JIT-compiling a method creates a MasterLinkContext • Finding custom attributes is easy with Metadata Importers • LinkContexts in a method share a MasterLinkContext • Need a LinkContext for caller’s and callee’s scope

22. Issue: Static resolution of flexible fields • C# 1.x compiler resolves fields statically • Effect is to “hide” members that should be substituted • Java 1.3 did the same; changed in 1.4 // Compile-time env class A { String f; } class B : A {} [LinkClass(B,C)] new B().f; [LinkClass(B,C)] ldfld […]A::f // Does not match // the LinkClass(B,…) compiles to

23. Conclusion • .NET is a good home for flexible dynamic linking • Different frameworks have different API implementations→ more choices for the programmer in [Link*] • Resolution guided by rich metadata • Easy to represent substitutions and constraints • Flexible bytecode is still verifiable (type-safe) • Tiny amounts of code in the SSCLI are very effective • Future work • Modify compiler(s) for independent compilation • Enforce consistent representation for substituted types • Overcome static resolution problem

24. Thank you