Debugging Applications for MS .Net and Windows

1. Debugging Applicationsfor MS .Net and Windows Ch.2 Getting Started Debugging

2. (this title intentionally left blank) Presentation for chapter 2 of John Robbins’ book “Debugging Applications for MS .Net and MS Windows” Part of the ongoing Testing Seminar meetings Coordinated by Dr. Fawcett. Date: Friday, October 3, 2003 Presenter: Kanat Bolazar, kanat2@yahoo.com Syracuse University, Syracuse, New York

3. Track Changes Until You Throw Away the Project Code evolves • Documentation updated? • Requirements specification document • Design (architecture) documents Track changes: • Version control • Bug tracking, bug number Advantages: • Bringing a new team member up to speed • Ability to build earlier deployed versions of code • ... (what else?) Disadvantages?

4. What To Put Under Version Control? • Source code • Design documents • End user manual • Test plans • Automated tests (harness & test cases) • External dependencies: libraries, tools

5. The Importance of Including Unit Tests Test harness • Comparator Test set • Test case • Oracle (expected output) Functional / black box testing Structural / white box / glass box testing • By developer Junit, Nunit, etc (see links at www.junit.Org)

6. Controlling Changes Size, change and stability Allow none? Allow all? • Green time: Anyone can check in anything • Yellow time: Only bug fixes; review each. • Red time: Code freeze / key milestone / testing. Is this bug critical? • Crash, data corruption: Yes! • Other: • How many people does it affect? • Is it at the core or periphery of the product? • How much retesting will be needed?

7. The Importance of Version Labels Definition Label all: • Milestones • Transitions from green to yellow to red times • Builds sent outside • Branching points in the development tree • Successful runs of daily build and smoke tests Consistent naming • (eg. three-part <Project><Milestone/Reason><Date>) QA tells developer the version label for bug

8. Having Trouble Reproducing Builds Sent Outside? Burn everything on CD/DVD for builds sent outside: • Source • Executables • Documentation • Intermediate files (object files, etc) • Installation kit • External libraries • [Build tools]

9. Bug Tracking Systems • Check bug tracking history: • Module with high bug frequency: • More error-prone • Do more review • Run more tests • Jot down reminders, notes, to-do items, in bug-tracking system as well. • Assign them to self • Set at lowest priority • Ignore these in bug metrics reports

10. Levels of access to bug tracking systems • Developers • Sales engineers (technical experts who teach salespeople) • Sales and marketing people: • May enter bugs, but may not view them • Must be taught how to enter bugs • They'll run your program before & during demos to client • They'll hear complaints from the field: • From alpha and beta testers • From end users

11. Choosing the Right Systems For You Small team? Small shop? Windows only? High end requirements? Multi-platform? Don't use home grown solutions Use commercial or open source solutions Evaluate well before committing to it Tools should integrate well Analogy: Code: Lifeblood of development Version control and bug-tracking systems: Arteries They keep the lifeblood flowing in the right direction.

12. Schedule Time For Building Debugging Systems Build (probably project-specific): • Crash handlers (ch. 13) • File data dumpers • Other tools to reproduce problems reported from the field Error handling system • A product feature • Helps handle bugs proactively

13. Error condition Handle consistently in code (pick one): • Return values • Exceptions (setjmp, longjmp in C) • Global error variable (as in C's "errno") C# • System.Exception at root of exception hierarchy (note: Users should inherit from ApplicationException) • Throw-catch vs. return values • Don't use exceptions for regular boundary cases • Throw fills stack trace, may be slow

14. C++ exceptions Robbins: "Forget that C++ exceptions exist.“ • Good in theory • Not so in practice • In COM: Can't cross apartment / process boundaries • Cause of many bugs Dr. Fawcett disagrees: “Use C++ exceptions, but don’t use exception specifications.”

15. Build All Builds With Debugging Symbols .PDB: Program database file Build release code with debugging symbols as well. • Release code is often optimized • May be just-in-time(JIT)-compiled • May be compiled by native compiler • Hard to debug • Some code optimized out/shared: Debugger jumps around. • Optimized call stack: Can't trace stack as well. • Larger files (with debugging symbols)

16. C#: Building with debugging symbols • csc.exe /debug:full • Visual Studio: Project properties Configuration Properties Build Generate Debugging Information

17. C++: Building With Debugging Symbols Use /Zi switch Visual Studio: Debug Information Format : Program Database (PDB) Also set: /INCREMENTAL:NO /DEBUG /PDB ... (many more options; refer to book) /OPT:REF -- Don't link functions not called: Vital for size Robbins wrote tool SettingsMaster (ch. 9) • Simplifies changing these options • Defaults to his preferred settings for all these options

18. For Managed Code, Treat Warnings As Errors C# Visual Studio .Net: Visual Basic and Visual C# Reference Visual C# Language C# Compiler Options Compiler Errors CS0001 Through CS9999 Errors Many more and tighter than C/C++ Warnings Levels: 1 - 4 Code should compile cleanly, without any warnings!

19. Treating Warnings As Errors in C# /WARN:4 (on by default) /WARNASERROR+ (off by default) Visual Studio: Project properties Configuration Properties Build Errors and Warnings Treat Errors As Warnings : True

20. When Warnings Get Annoying Ignore insignificant warnings? A sea of warnings: Which are insignificant? Fix code; avoid cause of warning • CS1573: Parameter 'parameter' has no matching param tag in XML comment (but other parameters do) • You were using the C# /DOC switch for XML documentation • A parameter to a function is undocumented • Document it!

21. When A Warning Kills Your Build • CS1596: XML documentation not updated during this incremental rebuild; use /incremental- to update XML documentation. Insignificant /DOC warning kills build! A real problem. Robbins suggests: • Debug builds: Build incrementally but without docs for • Release builds: Do full builds with docs

22. For Native Code, Treat Warnings As Errors -- Mostly C++: /W4 /WX Annoyed by warnings? Don't ignore; be pedantic. Bertrand Myers once said: If you think this is pedantic, remember that no one can be as pedantic as a machine running a code with a bug. Solve the problem if you can: C4100: 'identifier': Unreferenced formal parameter int baad(int i ) { return 25; } // Warning: C4100 int good(int /* i */) { return 25; } // ok.

23. Locally Disabling Warnings in Native Code If you can’t solve the problem, disable warning, temporarily, locally. To change warning level temporarily: #pragma warning ( push , 3 ) #include "DiesAtW4.h"; #pragma warning ( pop ) To disable one warning type temporarily: #pragma warning ( disable : 4201 ) ... #pragma warning ( default : 4201 )

24. The STL Supplied With Visual Studio .Net Has "Issues" C++ STL implementation that comes with Visual Studio: • Gives errors and warnings that are impossible to parse • Has lots of problems with its warnings • Critical with /WX (treat warnings as errors) switch • Solutions: • Include only precompiled headers • Use STLport instead: • www.stlport.org • Supports multithreading • Has extra support for debugging • Gives readable errors and warnings • May be used in commercial products

25. For Native Code, Know Where Your DLLs Load Program crashes at a known memory address • Which dynamically loaded library sits there? DLLs don't have set addresses in memory Solve problem proactively Rebase your DLLs to know their addresses in memory

26. Base Addresses for Managed Module DLLs CLR is very different from native code It's all interpreted There is no directly executable code Still, Robbins recommends rebasing .Net DLLs as well

27. Additional Compiler Options to Debug Managed C# There are a few

28. Compiler and Linker Options to Debug Native Code There are too many to count

29. Design a Lightweight Diagnostic System For Release Builds One customer (out of many) has a problem How to catch flow of execution? Log! Level of detail in logging: About to open file X Did lseek(25) on file X Read file X to buffer Use consistent format You can use Perl to easily extract the part you need If XML, you can use XSLT

30. Performance Issues of Logging Robbins recommends a "ready, fire, aim" approach: • Log everything that you feel you should • Profile the program • Find percentage of time spent on logging • Trim your logging calls in code, as needed

31. Frequent Builds Are Mandatory Build daily: Automated nightly build But what if my project is large? • How large? • Larger than MS Win XP with 40 million lines of code? Someone's code breaks the build • This is a serious crime, a major sin! • Punishment! (public flogging and donuts) • Humiliation! (public acknowledgement) • Slavery! (make him/her the builder) Pull daily • Pull the binaries; don't recompile on each machine

32. Smoke Tests Are Mandatory Term comes from electronics: • Develop the circuit • Plug it in • Does it smoke? These are pre-tests for software • First test: Start; stop; OK. • Next: Simple tests, one per feature • Performance benchmarks

33. Regression Testing Tool • Automates UI testing • Logs user interactions with the system • Can replay them after user has left Example: Tester (ch. 16) • Logs mouse and keyboard input • Saves them into a JScript / VBScript file • Allows playback

34. When to Freeze Updates to Compiler and Other Tools Feature complete = Beta 1 Much testing went into the product (executable). Don't try to fix it if doing so may break it. Freeze tools; don't upgrade versions from now on.

35. Build the Installation Program Immediately First impressions Build installation tool before anything else Allow option to install release or debug version Testing can start sooner Managed applications: • Much easier • Database connections • Global assembly cache COM: • Much harder • Much work in the registry

36. QA Must Test With Debug Builds QA could use diagnostics in debug builds But QA may use batch-mode testing • Turn off assertion outputs Balance debug and release build testing: • Before alpha: Two-three days of debug build testing per week • Before beta 1: Two days per week • After beta 2: One day per week • After release candidate: Do only release build testing

37. Symbol StoreandSymbol Server

38. Crash Dumps minidumps (crash dumps) • Introduced by Microsoft • Is like core dumps • Can be loaded up in a debugger • You can inspect the point at which the program crashed Issues: • Needs: • Symbols • Binaries • For: • Your code • OS code

39. Version Issues with Operating System Symbols and Binaries OS code symbols and binaries: You must have the correct type/version of OS and service pack ... ... of the customer that sent the minidump Your currently installed version and service pack may not match Solution: Install all the OS symbols and binaries and set up a symbol store.

40. Symbol Server Machine Give name such as //SYMBOLS Will hold OS binaries, OS symbols, your own symbols Does not need too much horsepower Needs much disk space: 40 - 80 GB or more Create shared directories • OSSYMBOLS • PRODUCTSYMBOLS

41. Download "Debugging Tools for Windows" www.microsoft.com/ddk/debugging Get the latest version Four key binaries (should have R/W access to server file system) • SYMSRV.DLL Symbol server (also helps folks behind proxy servers) • DBGHELP.DLL • SYMCHK.EXE Downloads OS symbols from Microsoft site • SYMSTORE.EXE Downloads OS binaries from Microsoft site Adds your (product) binaries and symbols to the symbol store

42. Set Master (Machine-Wide) Environment for SYMBOL PATH My Computer (right-click) / Properties / Advanced / Environment Variables _NT_SYMBOL_PATH If not defined, define new. Set its value to (all on one line): SRV*\\Symbols\OSSymbols*http://msdl.microsoft.com/download/symbols;SRV*\\Symbols\ProductSymbols If already defined, just add these before “;”, followed by old value.

43. Loading OS Binaries and Symbols • May be loaded as needed from Microsoft site • From Windows NT 4 to Windows XP • May be preloaded • About 1 GB disk space needed: • Per OS • Per service pack

44. OS Binaries and Symbols Download Automation: OSSYMS.JS (Robbins) • Uses SYMCHK.EXE and SYMSTORE.EXE • Loads all OS symbols and OS binaries • Steps to get all symbols for all service packs and hot fixes: • Install OS without service packs or hot fixes • Run OSSYMS.JS • Apply each service pack and hot fix, one at a time • Rerun OSSYMS.JS once after applying each

45. Adding Your Own Product's Binaries and Symbols • Use SYMSTORE.EXE: symstore add /r /f d:\build\*.* /s \\Symbols\ProductSymbols /t "MyApp" /v "Build 632" /c "01/22/03 Daily Build" /r add files or directories recursively /f File /s Store /t Product /v Version /c additional comment

46. Which Product Binaries and Symbols Should We Add? • What to add: • Add EXE, DLL, PDB files • After each daily build • After each milestone • For any build sent outside • What not to add: • Developers' local builds

47. Disk Size Issues with Product Symbols OS symbols: Only public symbols Your product's symbols: Includes private and local symbols Your product's symbols take much space Especially with daily builds Clean up: Delete older product symbols, regularly Delete daily builds older than four weeks

48. Deleting From the Symbol Server • Look at HISTORY.TXT in the 000admin special directory • Every item in history contains transaction number such as 0000000009 • Use: symsore del /i 0000000009 /s \\Symbols\ProductSymbols • Ignores errors silently, and gives no feedback!! • Check HISTORY.TXT: Deletion logged there?

49. Review • Track changes until you throw away the project • Schedule time for building debugging systems • Frequent builds and smoke tests are mandatory • QA must test with debug builds • Install the operating system symbols and set up a symbol store