Dynamic Program Analysis

1. Dynamic Program Analysis Vykintas Vyšniauskas Vilnius, 2014

2. Overview • Program analysis • Benefits and pitfalls of program analysis • Static analysis vs. dynamic analysis • Dynamic program analysis • Types of dynamic test execution tools • Summary Dynamic Program Analysis

3. Program analysis • The definitions for testing outline program analysis objectives that relate to evaluation, (revealing) defects, and quality. • Two approaches can be used to achieve these objectives: • Static analysis methods where the software artifact is examined manually, or with a set of tools, but not executed; • Dynamic analysis methods where the software artifact is executed using a set of input values, and its output behavior is then examined and compared to what is expected. • Testers use understanding of both static and dynamic testing techniques to develop test cases. Dynamic Program Analysis

4. Benefits of program analysis • Better understanding of your application and architecture • Reduced hardware and maintenance costs • Less hardware to setup and maintain • Learn how to be a better coder Dynamic Program Analysis

5. Rule of thumb • 80/20 rule • 80% of the runtime using only 20% of the code • Some people say 90/10 Dynamic Program Analysis

6. Pitfalls • Pre-optimization, waist of time • Optimizing the 80% of the code that only runs 20% of the time • Don’t fully understand the architecture or workload • Over optimize code • Can overcomplicate code Dynamic Program Analysis

7. Dynamic program analysis • Dynamic program analysis is the analysis of computer software that is performed by executing programs. • For dynamic program analysis to be effective, the target program must be executed with sufficient test inputs to produce interesting behavior. • Care must be taken to minimize the effect that instrumentation has on the execution of the target program. Dynamic Program Analysis

8. Dynamic test execution tools • Assertion analyzers • Capture replay • Coverage/frequency analyzers • Debuggers • Emulators • Network analyzers • Performance/timing analyzers • Runtime error checkers • Simulators • Status displayers/session documents • Test execution managers Dynamic Program Analysis

9. Assertion analyzers • Instrument the code with logical expressions that specify conditions or relations among the program variables. Dynamic Program Analysis

10. Capture replay tools • Automatically record test inputs/outputs using capture scripts • Replay the recorded tests using playback scripts • Useful for retesting when changes are made Dynamic Program Analysis

11. Coverage/frequency analyzers • Assess the degree of coverage of test cases with respect to: • executed statements • branches • paths • modules Dynamic Program Analysis

12. Debuggers • Not strictly a testing tool • Support the location of defects revealed during testing Dynamic Program Analysis

13. Emulators • May be used in place of missing or unavailable system components • Usually operate at the real-time speed of the components being emulated Dynamic Program Analysis

14. Network analyzers • Analyze the traffic on the network to identify problem areas and conditions • Allow simulation of the activities of multiple terminals Dynamic Program Analysis

15. Performance/timing analyzers • Monitor timing characteristics of software components or entire systems Dynamic Program Analysis

16. Runtime error checkers • Monitor programs for: • memory referencing • memory leaking • memory allocation errors Dynamic Program Analysis

17. Simulators • Used in place of missing or unavailable system components Dynamic Program Analysis

18. Status displayers/session documents • Provide test status information • Record selected information about a test run Dynamic Program Analysis

19. Test execution managers • Automate various functions of: • setting up test runs • performing a variety of tests • cleaning up after a test to reset the system Dynamic Program Analysis

20. Summary • Dynamic program analysis is the analysis of computer software that is performed by executing programs. • Dynamic analysis is used to detect defects and to evaluate quality attributes of the code. • Care must be taken to minimize the effect that instrumentation has on the execution of the target program. Dynamic Program Analysis

21. Sources of information • IleneBurnstein. PracticalSoftwareTesting. Springer, 2003. • Dynamicprogramanalysis. Wikipedia.http://en.wikipedia.org/wiki/Dynamic_program_analysis • Profiling and Detecting Bottlenecks in Software. Bryan Call, OSCON 2011.http://cdn.oreillystatic.com/en/assets/1/event/61/Profiling and Detecting Bottlenecks in Software Presentation.pptx Dynamic Program Analysis