1 / 12

KFT & Tracing Collaboration

KFT & Tracing Collaboration. Tim Bird Sony Electronics. KFT Quickie Overview. KFT = Kernel Function Trace Uses a compiler switch (-finstrument_functions) to instrument EVERY function in kernel Low-intrusion patch to kernel source Optimized for finding latency problems

cheng
Download Presentation

KFT & Tracing Collaboration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. KFT&Tracing Collaboration Tim Bird Sony Electronics

  2. KFT Quickie Overview • KFT = Kernel Function Trace • Uses a compiler switch (-finstrument_functions) to instrument EVERY function in kernel • Low-intrusion patch to kernel source • Optimized for finding latency problems • Filters for long-duration function calls • Maintains a function stack from which short duration functions are constantly popped • Historically used to find places to improve bootup time

  3. KFT Example • kftstatic.conf • Output: (cat /proc/kft_data | addr2sym -m System.map) trigger start entry start_kernel trigger end entry to_userspace filter mintime 500 Entry Delta PID Function Called At ----- ------ --- ------------------------- ------------------ 1 0 0 start_kernel L6+0x0 14 8687 0 setup_arch start_kernel+0x35 39 891 0 setup_memory setup_arch+0x2a8 53 872 0 register_bootmem_low_pages setup_memory+0x8f ...

  4. More Information • Has triggers, filters • Can trace very early - from first kernel instruction • Uses procfs for control and data (everything ASCII) • Post-processing tool shows: • functions sorted by duration, call count, local time, etc. • call tree, including interrupts • Google: “KernelFunctionTrace” (all one word)

  5. Tracing Collaboration

  6. Why Collaborate • Want to avoid duplicating effort • Want to avoid conflicts and re-inforce efforts • E.g. Kernel will support AT MOST one set of static tracepoints • Want to produce awesome tools with great capabilities and usefulness • It’s open source, dang-it! • How to proceed? • Groups have talked before - was there any outcome?

  7. Tracing Terminology • Phases: • trace definition/configuration • trace generation • activation, control, capture, termination • post-processing • formatting the data and visualization • Kinds of data gathering • event-based • sample-based

  8. Tracing Terminology • tracepoint/probe point - a location in the traced software, which, when encountered, “emits” an event • trace/probe handler - the code to execute when an event occurs or a sample is taken • trace definition - the set of tracepoints, handlers and constraints which determine what events are collected and how they are processed • post-processing - manipulation of the trace data after the trace is collected • aggregation - updating statistics or other analytical information, based on trace events or samples • filters - criteria used to limit the events that are captured or processed • triggers - criteria used to start and stop tracing automatically • visualization tool - tool to present trace data graphically to make it easier to analyze • Tracepoints and trace handlers can be defined either in-source (as compiled-in, or static tracepoints) orexternally (inserted as dynamic tracepoints)

  9. Tracer Comparison Chart SystemTap KFT Overhead LTTng Specialized tracers Flexibility

  10. Tracer Comparison Table

  11. Tracer Commonalities • Ummm… <crickets chirping>

  12. Ideas • Integrate KFT into LTTng • Have LTTng support externally-defined, but compiled-in, tracepoints • Stabilize trace buffer mechanism in kernel • Build best-of-systemtap functions module, which can be compiled-in • Standardize trace file format • What else???

More Related