1 / 11

CS527 Topics in Software Engineering (Software Testing and Analysis)

CS527 Topics in Software Engineering (Software Testing and Analysis). Darko Marinov September 9, 2010. Schedule. First few lectures to help you select projects Testing: ReAssert, UDITA, Pex, Randoop Today: JPF (model checking), note: journal paper Sep 14: CHESS (multithreaded testing)

beaumont
Download Presentation

CS527 Topics in Software Engineering (Software Testing and Analysis)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS527Topics in Software Engineering(Software Testing and Analysis) Darko Marinov September 9, 2010

  2. Schedule • First few lectures to help you select projects • Testing: ReAssert, UDITA, Pex, Randoop • Today: JPF (model checking), note: journal paper • Sep 14: CHESS (multithreaded testing) • Sep 16: Regression testing, note: survey paper • Sep 21: Static (code) analysis? • Sep 23: GUI testing? • Sep 28: Analysis of code comments? • Testing distributed or real-time systems? • Your suggestions?

  3. Paper Today • Model Checking Programs by W. Visser, K. Havelund, G. Brat,S. Park and F. Lerda(J-ASE, vol. 10, no. 2, April 2003) • JPF web site • http://babelfish.arc.nasa.gov/trac/jpf • First open-source project from NASA • Slides courtesy of Peter Mehlitz and Willem Visser

  4. Paper Overview • Problem • Model checking of real code • Terminology: Systematic testing, state-space exploration • Solution • Specialized Java Virtual Machine • Supports backtracking, state comparison • Many optimizations to make it scale • Publicly available tool (Java PathFinder) • Evaluation/applications • Remote Agent Spacecraft Controller • DEOS Avionics Operating System

  5. Questions for Discussion (1) • Reports on time but not early: [CR], [AS], [JC] • Can an explicit-state model checker really scale to million lines of code? [AA] • Tools are top down (execute the program and evaluate it as it runs) but code is written bottom up (unit by unit)? [AL] • When is a state revisited? [JN] • Can we move some part of search and bytecode interpretation to C? [MG] • How to use JPF for sequential programs? [QL]

  6. Questions for Discussion (2) • Omitted? [DM] • Reminder: you can omit up to two reports • How many kinds of abstraction are available? [ST] • Too much work for too little benefit? [HY] • Does it only find bugs that other tools can find? • Make JPF distibuted/parallel? [MK] • Combine JPF and Radoop (or X and Y)? • How can false positives be managed for a better user experience?

  7. Questions for Discussion (3) • Level of adoption outside NASA/Research? • Have any high profile bugs been found? • If JPF a valid approach outside of real-time systems and safety/mission critical software? • How do you know the tool is correct? • Especially first version with Java -> Promela • What about differences between JPF and JVM • Does it find bugs or prove software correct? • What about non-Java languages?

  8. Old Questions for Discussion (1) • Which part of JPF can be incorrect? • Can a compiler be model checked? • How difficult is it to set environments? • What to check in sequential programs (some examples are multi-threaded)? • What about checking design not code? • Comparison of testing, model checking, and static analysis

  9. Old Questions for Discussion (2) • When is model checking applicable? • What kind of bugs it can find? • How well does it scale? • What about memory requirements? • What about time requirements? • Why work with Java not C/C++? • How can I apply this on my code?

  10. Old Questions for Discussion (3) • What’s recent work in this area? • There’s some work in my group if interested • How do they compare states? • How good are heuristics? • How to use the tool in practice? • Hopefully the tutorial is helpful • You can download from NASA servers • Problem set based on some tool???

  11. Sample Project Ideas • Apply JPF on some software • Did you read the optional reading/report? • Compare with more tools and techniques • Improve JPF • Add new functionality (e.g., backtrackable time) • Implement more library methods (e.g., reflection) • Patch some bugs • Use for another language (Scala, X10…) • Automate creation of environments • Check 427 Final Project Proposals

More Related