1 / 40

Lecture 18

Lecture 18. Static Analysis Model-based Testing. Roadmap for Today . Static Analysis – intro, findbugs Model-based Testing – intro, nmodel. Static Analysis. Intuitively, “ grep ” source code for information, automatic code review Discover facts/properties/models about programs

eron
Download Presentation

Lecture 18

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 18 Static Analysis Model-based Testing

  2. Roadmap for Today • Static Analysis – intro, findbugs • Model-based Testing – intro, nmodel

  3. Static Analysis • Intuitively, “grep” source code for information, automatic code review • Discover facts/properties/models about programs • Histories: • 1954 Fortran compilers • First used to find information for optimization • Type check and type systems

  4. Static Analysis and Model Checking • Both are kinds of searchand both can be used to find bugs in the code • Both can find deeply embedded bugs that testing cannot find • Model checking: • Based on model of programs and properties • Find property violations • Path-Sensitive • … • Static analysis: • Can be applied directly on the source code • Can be path-sensitive or path-insensitive • Various purposes • …

  5. Static Analysis used by Software Engineers • Can be performed on the source code, binary code, models constructed from the code – CFGs, PDGs • Applications: • Bug checking – code pattern that indicates a potential bug at runtime • Reverse engineering – control flow analysis for generating sequential diagrams • Automatically generate test inputs • Debugging

  6. Static Analysis for Detecting Buffer Overflows- directly from source bar(); s = (char*)malloc(80); x[10] = ‘\0’; if(strlen(t)<8) strcpy(s,t); else strcat(x,t); • Path-Insensitive Analysis: simple pattern matching • Look for “strcpy”, “strcat” in the code

  7. Static Analysis for Detecting Buffer Overflows- from models of source bar() 1 Info1 bar(); s = (char*)malloc(80); x[10] = ‘\0’; if(strlen(t)<8) strcpy(s,t); else strcat(x,t); s = (char*)malloc(80) 2 Info2 x[10] = ‘0’ 3 Info3 strlen(t) < 8 Info4 4 yes no 5 strcpy(s,t) Info5 strcat(x,t) Info6

  8. Static Analysis for Detecting Buffer Overflows- from models of source Resolution bar(); s = (char*)malloc(80); x[10] = ‘\0’; if(strlen(t)<8) strcpy(s,t); else strcat(x,t); bar() 1 80/8>=len(t) len(t)<8 : safe s = (char*)malloc(80) 2 size(s)>= len(t) len(t) < 8 x[10] = ‘0’ 3 Safe size(s)>= len(t) len(t) < 8 strlen(t) < 8 4 yes no 5 strcpy(s,t) strcat(x,t) 6 Query size(s)>= len(t)

  9. Findbugs – from Bill Pugh’s talk public String foundType() { return this.foundType(); }

  10. Infinite RecursiveLoops: Sun JDK history

  11. More Bugs Control c = getControl(); if (c == null && c.isDisposed()) return; • // Eclipse 3.0.0M8 • String sig = type.getSignature(); • if (sig != null || sig.length() == 1) { • return sig; • } • // JDK 1.5 build 42 • if (name != null || name.length > 0) {

  12. Bug Categories • Correctness • Bad Practice - bad naming • Dodgy – dead store to local variable • Performance • Multithreaded correctness • Malicious code vulnerability

  13. Why False Positives • Report warnings but actually not a bug • Reason: code patterns do not necessarily sufficient to predict real execution problems, e.g., infeasible paths

  14. Challenges of software testing • Generate test input (unit, system) • Determine test inadequacies • Produce test oracle • Regression testing ……

  15. Model based Software Testing • Mode-based testing: automatic derivation of concrete • test cases from models of the system under testing • Derive all combination of tests associated with the requirements represented in the model to automate both the test design and test execution process • Some kind of black box testing • Requirement -> Model -> Test Cases

  16. Model based testing - fromMarkUtting + abstract tests + automatic execution + auto regression testing + auto design of tests + systematic coverage + measure coverage of model and requirements - modelling overhead

  17. Challenges of MBT • Testers need to be familiar with modeling techniques • Challenges of building good models • State space explosion problems

  18. Common Application Domains • Effective for testing small applications: • Communication protocols, • Web applications, • Embedded control systems, • Phone, and • Graphical user interfaces • Mostly Function test • Can be Unit test as well as System test • Sometimes Testing for Reliability and performance

  19. Model Types • State-based: Z, OCL (pre/post conditions) • Transition-based: UML state machines, FSM • Functional: Mathematical Algebra • Operational: Activities Diagrams

  20. Coverage Criteria • Cover paths (sequence) in the models • Random • Chinese postman: visit all transitions ASAP (the shortest tour of a graph which visits each edge at least once) • All Transition pairs • All variant of a given use cases (driven by use cases, cover important use case scenario)

  21. Step 1: Generate tests Input : • Model • Test algorithm – coverage • Number of tests Output: • Sequences of transitions • Coverage of transitions

  22. Step 2: Execute Test Cases • Offline: generate test cases, store in a file, then Write scripts to execute transitions • pro: repeatedly run • Online: choose which edge/state to cover when runs - pros: non-determinism can be considered

  23. A Case Study: NModel NModel: https://nmodel.codeplex.com/ University of Washington and Microsoft Research a model-based testing and analysis framework for model programs written in C#

  24. NModel: packages • A library of attributes and data types for writing model programs in C# • A visualization and analysis tool mpv (Model Program Viewer) • An alternative visualization tool mp2dot (Model Program to Dot) • A test generation tool otg (Offline Test Generator) • A test runner tool ct (Conformance Tester), for both offline and on-the-fly testing

  25. Steps to Use NModel • Construct models – model programs written in C# • Check the correctness of the model - visualize and analyze the model with mpv • Write a test harness in C# • Test input generation: • Online (ct): generate the test on-the-fly from your model program as the test run execute • Offline (otg): create tests from the model in advance

  26. Additional Features • Write a custom strategy in C# that ct uses to maximize coverage according to criteria you define. • Write simple finite state machines (FSMs), then use composition to achieve scenario control during testing or to check temporal properties during analysis • Combine model programs in a well-structured way

  27. FSM • FSM(Off, AcceptingStates(), Transitions(t(Off, PowerOn(), On), t(On, PowerOff(), Off))) Initial state, accept states, transitions (current, action, next)

  28. Model Program

  29. Visualize Models: mpv

  30. Another Model • FSM speed control: FSM(Low(), AcceptingStates(Low()), Transitions(t(Low(), IncrementSpeed(), Medium()), t(Medium(), IncrementSpeed(), High()), t(High(), IncrementSpeed(), Low())))

  31. Compositions of Models • mpv /r:PowerSwitch.dll /fsm:SpeedControl.txtPowerSwitch.Contract.Create

  32. Offline Test Generator • Generate Test Cases that Achieve the Coverage of FSM • otg /r:PowerSwitch.dll PowerSwitch.Contract.Create • TestSuite(TestCase(PowerOn(),PowerOff())) • It is typical to compose a scenario expressed as an FSM with a contract model program in C# in order to limit the size of the generated FSM.

  33. Run Testing: ct – conformance tester Input: • Implementation • Test case (programs) Output: • Fail/success - in the test harness we will set exception routines • Traces

  34. Coffee Make Demo

  35. Further Reading • Model-based Software Testing and Analysis with C# Jonathan Jacky, MargusVeanes, Colin Campbell, Wolfram SchulteCambridge University Press, 2008 (available December 2007)

  36. ModelJUnit • The ModelJUnit Model-Based Testing Tool http://www.cs.waikato.ac.nz/research/mbt/modeljunit/

  37. AsmL: Abstract State Machine Language • https://research.microsoft.com/en-us/projects/asml/

  38. Other MBT Tools • PyModel (model based testing for Python) : http://staff.washington.edu/jon/pymodel/www/ • A list of commercial tools: http://www.cs.waikato.ac.nz/research/mbt/Tools.pdf • Reactis® by Reactive System is targeted for model based testing of embedded systems, and it generates the tests from Simulink® and Stateflow® models. • Statemate™ Automatic Test Generator™ / Rhapsody® Automatic Test Generator™ (ATG) by I-Logix: generates test cases from state charts of the system and from UML state machine. The state charts have to be designed with the included designer tool. • ZigmaTEST Tools by ATS: uses a finite state machine based models, from which it can generate test sequences to cover the state machine transitions. The finite state machines must be created by hand.

  39. Todo • Homework5: Due Friday 11/11 7:00pm

  40. Last Wild & Crazy Ideas Class Picture

More Related