260 likes | 454 Views
Clairvoyant: A Comprehensive Source-Level Debugger for Wireless Sensor Networks. Jing Yang , Mary Lou Soffa, Leo Selavo, and Kamin Whitehouse Department of Computer Science University of Virginia. SenSys’07 Nov. 8, Sydney, Australia. Our offer.
E N D
Clairvoyant: A Comprehensive Source-Level Debugger for Wireless Sensor Networks Jing Yang, Mary Lou Soffa, Leo Selavo, and Kamin Whitehouse Department of Computer Science University of Virginia SenSys’07 Nov. 8, Sydney, Australia
Our offer $25 million for 100,000 node volcano monitoring network You will probably: • Write code • Test in simulation • Test a single node • Build testbeds • Test small deployments Finally, you deploy the production system and …
… it malfunctions • It always reports an eruption • What do you want to do (ideally) ? • Stop the execution • Step through each statement • Insert breakpoints • Print variables • Call functions A source-level debugger
Debugging in the WSN domain • EnviroLog • DiagMsg • Sympathy • SNMS • Marionette Simulation Deployment Not source-level debugging tools Scale EmStar ICE Realism Source-level debugging tools
Clairvoyant goals • Source-level debugging after deployment • Application runs on nodes • Operate over multi-hop networks • Minimize “Heisenbugs” • Network interference • Changes to execution timing • RAM (stack overflow) Simulation Deployment Scale EmStar ICE Realism
Interpreter Executor Executor User Debugging Unit GDB CDB UI Wrapper Clairvoyant overview >> break BlinkM.nc:18 breakpoint 1 is inserted >> continue breakpoint 1 is reached >> print state 0 >> next BlinkM.nc:19 is reached >> interrupts SPI (vector_17) >> call vector_17() function is called >> logRadio BlinkM.nc:20 state logpoint 2 is inserted >> gcontinue Deployed multi-hop WSN Terminal 4 Classic GDB cmds Terminal 1 Terminal 5 Terminal 2 Source-level debugging environment New cmds Terminal 6 Terminal 3 GDB-like text-based terminal
Outline • Classical GDB remote debugging • Minimize program size • Reduce network traffic • Preserve execution timing
Classical GDB remote debugging Example: conditional breakpoint Send values back Breakpoint reached Trap Value Debugger PC • GDB remote protocol Dynamic binary instrumentation • Insert “trap” into program • Switch context • Transfer control to debugger • Execute any debugging commands • Switch context again • Execute replaced instructions • Return control to program + Read condition variables Print Continue Continue execution Trampoline App Context switch Jmp To debugger Context switch Execute original instructions
Apply classical GDB RD to WSNs Debugger PC No appropriate for WSNs • Example: conditional breakpoint • Check condition on PC • 4 radio messages • Save full context • Time-consuming • Save clock registers • Timing-sensitive • Timer-dependent Trampoline App Context switch Jmp To debugger Context switch Execute original instructions
Preserve clock consistency 0 10 12 18 20 Timer fired Timer fired Breakpoint reached Continue execution 0 10 12 18 26 Timer fired Timer fired Breakpoint Reached Continue execution Save clock registers Restore clock registers
Apply classical GDB RD to WSNs Debugger PC No appropriate for WSNs • Example: conditional breakpoint • Check condition on PC • 4 radio messages • Save full context • Time-consuming • Save clock registers • Timing-sensitive • Timer-dependent • Use a separate trampoline • 256 bytes of program memory Trampoline 1 App Context switch Trampoline 2 Jmp Context switch To debugger Context switch Trampoline 3 Jmp To debugger Context switch Context switch Execute original instructions Jmp To debugger Execute original instructions Context switch Execute original instructions
Cond. breakpoint evaluation 962 ms Classical GDB Remote Debugging 256 8 4 0 0 Execution Overhead # Radio Messages # Flash Writes Data Memory Program Memory Skewed Clock Cycles
Outline • Classical GDB remote debugging • Minimize program size • Reduce network traffic • Preserve execution timing
Shared trampoline Application Shared trampoline Clairvoyant Stack Inst 1 1 Context switch 2 Interpret commands Application stack Inst 2 Call Inst 3 4 Jump to Clairvoyant Return address Inst 4 Inst 5 Context switch Clairvoyant stack Call Inst 6 Ret …… 3 Stack layout Inst m Inst n Call • Trade-offs: • Reduce program memory • Increase flash writes …… Process
Cond. breakpoint evaluation 962 ms Classical GDB Remote Debugging 962 ms Shared Trampoline + GDB Remote Protocol 256 8 8 4 4 4 0 0 0 0 Execution Overhead # Radio Messages # Flash Writes Data Memory Program Memory Skewed Clock Cycles
Outline • Classical GDB remote debugging • Minimize program size • Reduce network traffic • Preserve execution timing
GDB remote protocol Application Shared trampoline Clairvoyant Send values back Breakpoint reached Inst 1 Context Switch Interpret commands PC Inst 2 Read condition variables Continue execution Call Inst 3 Inst 4 Inst 5 Inst 6 • Example: conditional breakpoint • Shared trampoline + GDB remote protocol • All functionality is on PC • Excess network traffic …… Inst m Inst n ……
Moving functionality to nodes Application Shared trampoline Clairvoyant Inst 1 Context Switch Interpret commands PC Inst 2 Inst 3 Call Read condition variables Inst 4 Inst 5 Check values Inst 6 …… • Example: conditional breakpoint • Move condition checking to nodes • Trade-offs: • Reduce network traffic • Increase data memory • Reduce conformity with GDB Inst m Inst n ……
Cond. breakpoint evaluation 962 ms Classical GDB Remote Debugging 962 ms Shared Trampoline + GDB Remote Protocol Shared Trampoline + Functionality Moving 256 26 186 ms 8 8 4 4 4 4 0 0 0 0 0 0 0 Execution Overhead # Radio Messages # Flash Writes Data Memory Program Memory Skewed Clock Cycles
Outline • Classical GDB remote debugging • Minimize program size • Reduce network traffic • Preserve execution timing
Absolute vs. relative timing Create “Heidenbugs” 0 10 12 18 26 Timer fired Timer fired Node 1 Breakpoint Reached Continue execution Save clock registers Restore clock registers 0 10 20 Timer fired Timer fired Node 2
Faster context switch Application Separate trampoline Shared trampoline Clairvoyant Inst 1 Context Switch Context Switch Interpret commands PC Inst 2 Inst 3 Jmp Read condition variables Inst 4 Inst 5 Check values Inst 3 + Inst 4 Inst 6 • Example: conditional breakpoint • Use a separate trampoline • Save only necessarily registers (no clock registers) • Trade-offs: • Reduce execution overhead • Reduce flash writes • Increase program memory • Increase skewed clock cycles …… Inst m Inst n ……
Cond. breakpoint evaluation 962 ms Classical GDB Remote Debugging 962 ms Shared Trampoline + GDB Remote Protocol Shared Trampoline + Functionality Moving Faster context switch + Functionality Moving 256 256 26 27 21 186 ms 8 8 4 4 4 4 27 cc 0 0 0 0 0 0 0 0 0 Execution Overhead # Radio Messages # Flash Writes Data Memory Program Memory Skewed Clock Cycles
Conclusions • WSNs are notoriously difficult to debug • Embedded systems • Distributed systems • Event-driven, I/O-oriented computing • Multi-hop, wireless networking • Not the final solution, but an important step • 1st post-deployment source-level debugger • Identified and solved several important challenges • Timing preservation • Function placement • Flash writes • Network interference • Made different trade-offs for different requirements
Future work • Re-implementation on MSP430 • Instructions can be executed from RAM • Possibly reduce number of flash writes • More commands to find distributed bugs • Better integration between logging and interactive debugging • Better support for eavesdropping-based debugging
Questions • Demo this afternoon • 14:00 - 17:15 • Code release • www.cs.virginia.edu/~jy8y/projects/clairvoyant.htm