Specialization Tools and Techniques for Systematic Optimization of System Software

1. Specialization Tools and Techniques for Systematic Optimization of System Software Presented By: Ashwini Kulkarni Operating Systems Winter 2006

2. Agenda • Introduction • Specialization Overview • Steps in Specialization • Specialization Toolkit • Evaluation of the toolkit • Lessons learned • Conclusions

3. Introduction • Key dilemma for OS designers: • Correctness vs. • Performance • Conventional Approach: • Write general-purpose code optimized for a few common cases • Incorporate customizability into system structure

4. Possible Solutions • There are 2 types of customization: • Explicit customization: • OS customized for currently observed common conditions • Customization designed into the OS • Actual customized code manually written and injected into the system Drawbacks of explicit customization: • Significant burden placed on system tuners • Optimization opportunities are reduced • Inferred customization: • Automatically derive optimizations instead of writing by hand

5. Specialization • Paper’s approach: inferred customization based on specialization • Create optimized code for the common cases • Restricting code for improved performance • Extension vs. Specialization • Benefits of this approach: • Reduced burden on system tuner • Specialized components can take advantage of any state in the system • Drawbacks of this approach: • Involves complex analysis of the system • Generating specialized code can be tedious and error-prone • Results in systems which are complex and harder to debug

6. Specialization (Contd.) • To address drawbacks, paper introduces specialization toolkit • Reduce amount of manual work required to specialize OS • Tools: Tempo, TypeGuard, MemGuard, Replugger

7. Specialization Overview • Related Concepts: • Specialization predicates: States of the system known in advance • Partial evaluation: Take a generic source program plus a set of specialization predicates that apply to it • Residualization: Combining lifted static output with remaining dynamic parts to form the specialized program

8. Specialization Overview (Contd.) • Three kinds of specialization: • Static • Specialization predicates known at compile time • Partial evaluation applied before system execution • Dynamic • Run-time specialization • Predicates established at some point during execution • Once established, predicates hold for remainder of the execution • Optimistic • Predicates only hold for bounded time intervals • Detecting when specialization predicates hold and cease to hold (Guarding) • Enabling and disabling specialized components (Replugging)

9. Steps in Specialization • Steps to specialize a system: • Identify specialization predicates • No tool available • Generate specialized code • Tempo partial evaluator tool • Check when specialization predicates hold • TypeGuard and MemGuard tools • Replace specialized code • Replugger tool

10. Tempo: Specialized Code Generator • A partial evaluator – Generates C code • Binding-time analysis: separate static from dynamic parts • Can perform both compile-time and run-time specialization • Challenges • Pointers and aliases • Structures and arrays • Functions with side-effects

11. Enabling and Disabling Specialized Code • Specialized code is only correct when specialization predicates hold • Static specialization – Specialization predicates are invariant • Dynamic and optimistic specialization – Specialized code should not be enabled before specialization predicates are established • Tools: TypeGuard and MemGuard locate the code that establishes or destroys specialization predicates

12. TypeGuard • Operates on program source code • Uses type information to locate sites that should be guarded • Place guards at the site of modifications to specialization predicate terms • Uses a two-phase approach: • First Phase: Static analysis: • To identify structure types whose fields are specialization predicate terms • Include Specialization Predicate ID (SPID) • Identify statements that update a guarded field • Second Phase: Dynamically set SPID field when specialized code is enabled and clear it when specialized code is disabled

13. MemGuard • Absence of type-safe language, any type-based guarding tool cannot guarantee complete coverage • Solution: Use memory protection hardware • Write-protect pages containing specialization predicate terms • Protection fault handler checks for violation before writes complete • Guarantees to capture all writes to specialization predicate terms

14. Replugger • Replace current code when: • Dynamic specialization predicate is established • Optimistic specialization predicate is violated • Use an asymmetric synchronization mechanism • Two factors affecting design of replugging mechanism: • Concurrent invocation: • Affected by scope of repluggable functions • Avoid concurrent invocation using threads • Concurrency between replugging and invocation

15. Evaluation • Specialization toolkit used to specialize three disparate system components: • Static specialization of Sun RPC • Dynamic specialization of Berkeley Packet Filter (BPF) programs • Optimistic specialization of Linux signal delivery mechanism

16. Specialization Experiment: BSD Packet Filters • Provides programmable interface for selecting packets from network interface • Enables user applications to download packet filter programs into kernel or library-resident packet filter interpreter (Relation to extensibility) • Single BPF program executed many times to examine thousands of packets – Ideal candidate for specialization • Code being specialized – Packet filter interpreter • Specialization predicates – Derived from a particular packet filter program

17. Specialization Experiment: BSD Packet Filters (Contd.) • Packet is read and filtered by calling bpf_filter function • Parameters of the function are: • Packet filter program (Entire program passed as a parameter) • Packet • Length of the original packet • Amount of packet’s data • Packet filter program always remains the same during a session – Derive specialization predicates from it in order to eliminate interpretation

18. Specialization Experiment: BSD Packet Filters (Contd.) while(true) { switch (pc->opcode) { case LD: // do load instruction case JGT: if (accumulator > index_register) pc = pc->target_true else pc = pc->target_false case RET: // return instruction result =... break; } pc++ } Basic loop for BPF interpreter

19. Specialization Experiment: BSD Packet Filters (Contd.) • Basic structure of BPF interpreter: • Interpreter consists of an infinite loop • Each iteration fetches instruction pointed to by pc • Uses a case statement to decode and execute instruction • Increments pc • Approach is problematic: • Value assigned to pc depends on dynamic interpreter state • Makes the interpreter unspecializable • Alternative approach amenable to specialization – Use recursion

20. Specialization Experiment: BSD Packet Filters (Contd.) case JGT: if (accumulator > index_register) return(bpf_filter(pc->target_true, c, wirelen, buflen)) else return(bpf_filter(pc->target_false, c, wirelen, buflen)) Using recursion to make pc static • while loop is replaced by a tail-recursive function which gets called for each new value of pc • BPF interpreter was manually restructured using recursion in order to perform experiments

21. Specialization Experiment: BSD Packet Filters (Contd.) Specializing BPF: • Option 1 - Static specialization • Option 2 - Dynamic specialization – Statically specialize BPF interpreter for a constant packet filter program known in advance - Dynamically specialize when packet filter program value is known just before execution • fill template holes, evaluate static parts

22. Specialization Experiment: BSD Packet Filters (Contd.) • Specialized interpreter for a simple packet filter program which counts packets • Comparison between specialized program and the unspecialized interpreter on the same packet filter program • Construct a null packet filter to incur the unspecializable overheads of packet filter mechanism Time taken to process 10MB data (~10,000 packets)

23. Lessons for System Tuners and System Designers • System Tuners: • Session-oriented operations such as file open/close, socket open/close, etc. • Domain-specific language interpreters or compilers used by a system • System Designers • Explicit relationships between components • Recognize interconnections from repeated patterns of actions

24. Conclusions • Specialization is a powerful technique for optimizing operating systems • Automatic specialization enables significant performance optimizations • Eases the burden on the system tuner: less error prone • Opens up new ways of designing operating systems which combine high performance and clean structure