210 likes | 354 Views
GILK: A Dynamic Instrumentation Tool for the Linux Kernel. David J. Pearce, Paul H.J. Kelly, Tony Field and Uli Harder d.pearce@doc.ic.ac.uk www.doc.ic.ac.uk/~djp1/gilk.html. Introduction. Do programmers really know what is going on? Need deeper understanding of program behaviour
E N D
GILK: A Dynamic Instrumentation Tool for the Linux Kernel David J. Pearce, Paul H.J. Kelly, Tony Field and Uli Harder d.pearce@doc.ic.ac.uk www.doc.ic.ac.uk/~djp1/gilk.html
Introduction • Do programmers really know what is going on? • Need deeper understanding of program behaviour • Debug/model/validate performance issues • Program instrumentation can provide this • GILK is about doing this for the Linux Kernel • Adds instrumentation on the fly • No need to restart • No need to recompile • No need to interrupt the web server!
Introduction • Do programmers really know what is going on? • Need deeper understanding of program behaviour • Debug/model/validate performance issues • Program instrumentation can provide this • GILK is about doing this for the Linux Kernel • Adds instrumentation on the fly • No need to restart • No need to recompile • No need to interrupt the web server!
Motivating Example • Research into Self Similarity @ Imperial • Using TCPDUMP to obtain network timing information • Produced fair amount of data • But occasional zero or negative inter-arrival time was spotted • Are results so far wasted? • Is TCPDUMP useable or not? • GILK deployed for comparison with TCPDUMP • GILK provides high resolution packet arrival times • Previous results valid – statistical properties unchanged • TCPDUMP does generate erroneous results • What We Did • Next slide
Motivating Example • Research into Self Similarity @ Imperial • Using TCPDUMP to obtain network timing information • Produced fair amount of data • But occasional zero or negative inter-arrival time was spotted • Are results so far wasted? • Is TCPDUMP useable or not? • GILK deployed for comparison with TCPDUMP • GILK provides high resolution packet arrival times • Previous results valid – statistical properties unchanged • TCPDUMP does generate erroneous results • What We Did • Next slide
Motivating Example • Research into Self Similarity @ Imperial • Using TCPDUMP to obtain network timing information • Produced fair amount of data • But occasional zero or negative inter-arrival time was spotted • Are results so far wasted? • Is TCPDUMP useable or not? • GILK deployed for comparison with TCPDUMP • GILK provides high resolution packet arrival times • Previous results valid – statistical properties unchanged • TCPDUMP does generate erroneous results • What We Did • Next slide
What We Did Linux Network Stack { BSD Socket Layer sock_recvmsg { Ethernet Driver speedo_rx
What We Did Linux Network Stack { BSD Socket Layer sock_recvmsg RDTSC { Ethernet Driver RDTSC speedo_rx
Where was the Success? • We’re not Kernel Programmers • Don’t know beforehand what was needed • Time wasted trying different instrumentation • GILK is fast • Turn around time only a few minutes • Allows us to try many different instrumentations • Fits exploratory nature of debugging
Where was the Success? • We’re not Kernel Programmers • Don’t know beforehand what was needed • Time wasted trying different instrumentation • GILK is fast • Turn around time only a few minutes • Allows us to try many different instrumentations • Fits exploratory nature of debugging
Where was the Success? • We’re not Kernel Programmers • Don’t know beforehand what was needed • Time wasted trying different instrumentation • GILK is fast • Turn around time only a few minutes • Allows us to try many different instrumentations • Fits exploratory nature of debugging
How it Works • GILK uses code splicing technology • Allows instrumentation of active kernel • Instrumentation performed at machine code level • Works on unmodified kernel image • No patching, restarting, recompiling etc • Instruments written in ‘C’ • Implemented as Kernel Modules • Access to full language features • Access to kernel functionality • System Requirements • Only Intel x86 Architecture supported • Kernel versions 2.0 – 2.2 supported • Extension to 2.4 is trivial
How it Works • GILK uses code splicing technology • Allows instrumentation of active kernel • Instrumentation performed at machine code level • Works on unmodified kernel image • No patching, restarting, recompiling etc • Instruments written in ‘C’ • Implemented as Kernel Modules • Access to full language features • Access to kernel functionality • System Requirements • Only Intel x86 Architecture supported • Kernel versions 2.0 – 2.2 supported • Extension to 2.4 is trivial
How it Works • GILK uses code splicing technology • Allows instrumentation of active kernel • Instrumentation performed at machine code level • Works on unmodified kernel image • No patching, restarting, recompiling etc • Instruments written in ‘C’ • Implemented as Kernel Modules • Access to full language features • Access to kernel functionality • System Requirements • Only Intel x86 Architecture supported • Kernel versions 2.0 – 2.2 supported • Extension to 2.4 is trivial
Instruction Boundaries { { { Kernel Function Kernel Function Code Patch Before After splice unused instrumentation Relocated sequence
Conclusion • GILK quickly instruments Linux Kernel • Runtime code splicing useful for instrumentation • Much left unsaid, however • Future work … • Simplifying instrumentation process • GUI, Instrumentation Strategies, Visualization • Programmatic instrumentation is possible • Paradyn (WISC) attempts this • Non-instrumentation based options • Modifying Program Behaviour • Non-Intel platforms? • KernInst/Paradyn (WISC) on UltraSparc/Solaris • User-Space programs? • User-Space Threading is an issue … • Our Research group is now doing this for Java …
Conclusion • GILK quickly instruments Linux Kernel • Runtime code splicing useful for instrumentation • Much left unsaid, however • Future work … • Simplifying instrumentation process • GUI, Instrumentation Strategies, Visualization • Programmatic instrumentation is possible • Paradyn (WISC) attempts this • Non-instrumentation based options • Modifying Program Behaviour • Non-Intel platforms? • KernInst/Paradyn (WISC) on UltraSparc/Solaris • User-Space programs? • User-Space Threading is an issue … • Our Research group is now doing this for Java …
d.pearce@doc.ic.ac.ukwww.doc.ic.ac.uk/~djp1/gilk.html Conclusion • GILK quickly instruments Linux Kernel • Runtime code splicing useful for instrumentation • Much left unsaid, however • Future work … • Simplifying instrumentation process • GUI, Instrumentation Strategies, Visualization • Programmatic instrumentation is possible • Paradyn (WISC) attempts this • Non-instrumentation based options • Modifying Program Behaviour • Non-Intel platforms? • KernInst/Paradyn (WISC) on UltraSparc/Solaris • User-Space programs? • User-Space Threading is an issue … • Our Research group is now doing this for Java …