230 likes | 372 Views
Embedded Linux Debugging with JTAG ICE. Kyoto Microcomputer Co., Ltd. http://www.kmckk.co.jp/eng. [Translated by ikoma]. Linux Debugging. Limitation of GDB(1). ptrace() system call in kernel is used for debugging. Debugging daemon ( gdbserver) is one of user applications. User Space.
E N D
Embedded Linux Debugging with JTAG ICE Kyoto Microcomputer Co., Ltd. http://www.kmckk.co.jp/eng [Translated by ikoma]
Limitation of GDB(1) • ptrace()system call in kernel is used for debugging. Debugging daemon(gdbserver) is one of user applications. User Space gdbserver Target Application For Debugging Kernel/Loadable Modules ptrace Driver Kernel Space
Limitation of GDB(1) • As kernel is one program, ptrace() can not run when breakpoint reached in device driver. User Space gdbserver Target Application For Debugging Breakpoint reached Kernel stops Kernel/Loadable Modules ptrace Driver Kernel Space
Limitation of GDB(1) • While stopped in driver, it is not possible to peek into memory or to set break points on debug target app. User Space gdbserver Target Application For Debugging Kernel stops and gdbservercannot work Breakpoint reached Kernel stops Kernel/Loadable Modules ptrace Driver Kernel Space
Issue in GDB(2) • Issue of debugging more than one processesat the same time gdbserver Process B kernel Process A Break request signal execution stops Process B may run Break request signal execution stops Kernel Space User Space
Debugging Targets on Linuxand Their Features *1 Depending on CPU, Initialization Section may be “Paged Out” PARTNER-JetSupports All Above
Technical Challenges • Relocatable • Actual addresses are unknown until kernel loads image onto memory. • On Demand Paging • When setting breakpoints or referring variables, it is not assured that the page is on memory. • Multiplexed Virtual Spaces on MMU • An address may designate different memory locations if the process IDs are different • Support of hardware breakpoint • Support of trace to process (ETM/AUD etc.)
Resolving Relocation(Loadable Modules) • Tricks for Debugging on Loadable Modules Inserting software break before module initialization At insmod, execution of software break transfers control onto ICE. With that address, .text is resolved. .data and .bss are resolved using informoation embedded around SBP. As this method does not depend on Linux, the same method has also been appliedfor other systems using relocatable object. #define module_init(x) \ int init_module(void) __attribute__((alias(#x))); \ static inline __init_module_func_t __init_module_inline(void) \ { return x; } #define module_init(x) \ int init_module(void) \ { __kmc_module_debug_start(); return x(); } \ static inline __init_module_func_t __init_module_inline(void) \ { return x; } asm(" .text"); asm(" .long 0x4c434d81"); asm("__kmc_driver_start:"); asm(" .long 0xdeeedeee"); asm(" .long 0x4c434d80"); asm(" .long __kmc_driver_name"); asm(" .long __kmc_driver_tmp");
Resolving Relocation(Shared Library) • PARNTER Obtains Info from Kernel Referring • task_struct • file_struct • mm_struct of the process, /proc/(pid)/maps-like info is obtained. With the info obtained, .text, .data and .bss are relocated.
Resolving On Demand Paging • Send program from ICE to cause read access at desired memory address whose page is not on memory • Kernel loads the page • User does not have to care about and PARTNER-Jet automatically does this operation • PARTNER-Jet analyzes various info in kernel to make sure to access memory
Nikkei Electronics 2006/1/2 Issue KMC”Virtualizing Technologies to Support Debugging on Linux and Multicore Environment” P121 Logical Address Space on MMU Kernel Debugger Engine User Kernel loads pages of exec code Debugger engine stops execution of process User specify a break point on source code Debugger engine let execute the code calling code with specified break point Page fault occurs and kernel loads page of new exec code Set the break point at the logical addr corresponding to user specified location User orders to resume process Debugger engine restarts the process Kernel executes the process to the breakpoint Debugger engine stops the process at the breakpoint Fig.5 Intentional Page Fault Linux kernel does not load all execution code onto main memory at once. “Page fault”, detecting that code to execute has not been loaded on memory,causes the execution code to be read into. In order to make it possible to set breakpoint at point kernel has not yet loaded on memory, we have intentionally made page fault occur. We have implemented the function to call beforehand execution code, which correspondis to the user specified breakpoint.
Resolving Multiplexed Memory Spaces • ICEManages Virtual Spaces • While debugging process, tracks the management structure of the process(task_struct) • When breakpoint reached, keeps the state of the process at that timing • Minimal Kernel Patches • Essential to dispatch process of real time trace(sched.c). And some others. void __kmc_schedule(prev,next) struct task_struct *prev, *next; { int index_next; index_next=__kmc_schedules_index & (KMC_MAX_SHEDULE_LIST-1); ++__kmc_schedules_index; __kmc_schedules_list_pid[index_next]=next->pid; } ・・・・・・ #ifdef CONFIG_KMC_TRACE_EXT __kmc_schedule_call(prev,next); #endif /* * This just switches the register state and the * stack. */ switch_to(prev, next, prev); __schedule_tail(prev);
Tricks for Linux • Attaching to Running Process • Possible at any timing, while running, while blocking in kernel, or while bloking in process • ICE saves PC which has been saved as context, and modifies it to address in attach area. When rescheduled, attach area is executed and breaks. • Automatic Loading of Debugging Information • Loading automattically all debugging information of shared libraries which the process is using, and relocating automatically • Analyzing the control structure of the process • Function Trace for Process • Even when stopped in kerenl space, backtrace of the specified thread can be referred • ICE refers to the PC, SP, FP and stacks saved as context and replays • Application Mode(See following slides) • Debugging while keeping other processes than target process alive
Application Mode- Virtual ICE Technology - • ICE Dedicated to Process • Can debug only processes on virtual spaces with external ICE • Memory swapping is fully supported • Debugging of Target Process Only • Debuggable with other processes than target and kernel to keep running • Effective Use of ICEFunctions • Hardware breakpoints, traces
Application Mode- Virtual ICE Technology - ターゲットシステム Process1 Process2 Process3 Device Drivers Operating System CPU Target System JTAG ICE PARTNER-Jet PARTNER Debug Control DLL Debugger PARTNER/Win VirtualICE VirtualICE VirtualICE Debugger PARTNER/Win Debugger PARTNER/Win Debugger PARTNER/Win Debugging Host PC Debugging of kernel & modules Debugging of Process1 Debugging of Process2 Debugging of Process3
Profiling Features for Linux • Execute Profile from JTAG ICE • No target memory used to keep trace data (for memory-tight embedded systems) • Implemented with Minimal Patches to Kernel • 4 patches in 3 source file for ARM11 • Efficient communication by using ARM DCC static inline unsigned int jtag_com_stat11(void) { unsigned int stat; __asm__("mrc p14,0,%0,c0,c1":"=r"(stat)); return stat; } static inline void jtag_com_put11(unsigned int data) { __asm__("mcr p14,0,%0,c0,c5"::"r"(data)); return; }
Profiling for Linux • Key Features • Profilable integrating both kernel and processes • At 100usecsampling, load on target is 1/100 or less (in case of ARM9) • Get transitions of generation/distruction/switching of processes and threads, displaying with process IDs/process names • Get execution time ratio per process • Get execution time ratio of each function in processes/kernel • Get execution time ratio of processes/kernel • Referring on debugger, as well as saving data in CSV format to analyze yourself withEXCEL etc.
Efficient Debugging on Embedded Linux • Fully Enjoy Being Open Source • Debugging environment becomes more powerful with combination of ICE + kernel modification, than with ICE only • It is easier for us, tool vendor, to develop tools for Linux because of open source • PARNTER-Jet is monitoring major data structures in Linux kernel(task_struct,mm_struct,file_struct etc. ) • Matching of trace of ETM/AUDetc. and souce code enables to investigate in detail of Linux dynamics
Interaction of Linuxand Tools • Transfers trace and logging data from kernel and processes through JTAG • Memory freeze, trace freeze, and analysis functions at specified location in kernel Analyzing App App A App B App C JTAG ICE Kernel Drivers
Contact Headquarter 2-44 Ooe-nakayama-cho, Nishikyo-ku, Kyoto, 610-1104, Japan TEL +81-75-335-1050 FAX +81-75-335-1051 Tokyo Office R Bldg 5F, 2-14-4 Shimbashi, Minato-ku, Tokyo, 105-0004, Japan TEL +81-3-5157-4530 FAX +81-3-5157-4531 URL:http://www.kmckk.co.jp/eng jp-info@kmckk.co.jp er-info@kmck.co.jp