iProbe: A Lightweight User-Space Instrumentation Tool

1. iProbe: A Lightweight User-Space Instrumentation Tool • Nipun Arora, Hui Zhang, Junghwan Rhee, Kenji Yoshihira, Guofei Jiang • Autonomic Management Group • Princeton, NJ

2. Motivation

3. Background } Blackbox but slow • Trampoline (DTrace, DynInst) • Just-in-Time Compilation (PIN, Valgrind) • Source-code based techniques (Log4j, Log4c) • Compiler driven techniques(gprof) } Developer/Compiler driven but fast Trampoline mechanism H/W Interrupt Trap Mechanism Overhead because of extra-jumps, and simulating overwritten instructions High overhead because of h/w trap mechanism Monitored Application Monitored Application High overhead because of context switch to kernel space *High Overhead because of complex Safety checks User Space Logs Kernel Space Logs D-Trace/SystemTap DynInst

4. The Core Idea: Hybrid Instrumentation • Traditional Instrumentation uses either purely compiler based techniques or purely binary based techniques • Hybrid Instrumentation uses both compiler and binary instrumentation to gain a significant performance advantage Development Phase Production Phase Source Code Files iProbe GCC Compiler Flags + iProbe packaged software Run-time Monitoring iProbe packaged software

5. State-Diagram ColdPatch Phase HotPatch Phase • Compile source code files with GNU compiler flag “-finstrument-functions" • Use cold-patch script to replace all instrumentation calls with NOP instructions • Create a meta-data file with location of each NOP placeholder • iProbe presents the user the set of functions they can select to instrument at run-time using the probe-list • The HotPatcher then replaces the NOP instructions with a call to the instrumentation function • Extremely low overhead since there is no overwriting of instructions <Basic Block Begin> <func_foo>: push %EBP call <foo_begin> pop inc …. … … call <foo_end> pop <Basic Block End> <Basic Block Begin> <func_foo>: push %EBP call<begin_instr> pop inc …. …. …. call<end_instr> pop <Basic Block End> <Basic Block Begin> <func_foo>: push %EBP NOP <90> pop inc …. … … NOP <90> pop <Basic Block End> <Basic Block Begin> <func_foo>: push %EBP pop inc …. … … pop <Basic Block End> NOP replaced in run-time with calls to instrumentation functions Native binary Compiled with instrumentation flag Replaced by NOP OpCode

6. Evaluation : ColdPatch • An iProbe enabled cold-patched application has NOP instructions added as placeholders to the binary • Evaluation on SPEC CPU 2006 benchmarks - negligible on most applications (<1%)

7. Evaluation: HotPatch • Evaluation of application performance in comparison to existing tools was found to have an order of magnitude better performance and scaled significantly better 107 103 108 104 105 106 10 102 10-3 10 10-2 0 104 102 10-1 103

8. Conclusion • iProbe can provide for extremely light-weight instrumentation in the user-space, with a stable and robust design, which avoids most complications that other tools deal with. • No developer effort needed (new instrumentation etc.), uses pre-existing compiler flags • We have an advanced version which supports binary rewriting or user-driven macros to generate place-holders, and can be used instead of compiler flags for customized instrumentation points. • iProbe-enabled applications can provide secure instrumentation especially when used with code-obfuscation techniques • iProbe can be used as a monitoring framework to develop further more intelligent instrumentation and monitoring applications

9. Please visit us !!! Nipun Arora NEC Laboratories America Princeton, NJ http://www.nipunarora.net/iprobe-demo.html nipun@nec-labs.com