1 / 10

CS527 Topics in Software Engineering (Software Testing and Analysis)

CS527 Topics in Software Engineering (Software Testing and Analysis). Darko Marinov September 15, 2011. Schedule. First few lectures to help you select projects Shared memory: CHESS, IMUnit , CAPP Message passing: Setac Comment analysis: iComment Regression testing: survey paper

callia
Download Presentation

CS527 Topics in Software Engineering (Software Testing and Analysis)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS527Topics in Software Engineering(Software Testing and Analysis) Darko Marinov September 15, 2011

  2. Schedule • First few lectures to help you select projects • Shared memory: CHESS, IMUnit, CAPP • Message passing: Setac • Comment analysis: iComment • Regression testing: survey paper • Today: Model-based testing • Next week two papers • 3 votes: generation/evaluation of test assertions • 2 votes: mock-based testing, GUI testing • 1 vote: genetic algorithms, concolic testing, web testing • Your project can be something we don’t cover

  3. Paper Today • Model-based quality assurance of protocol documentation: tools and methodologyWolfgang Grieskamp, Nicolas Kicillof, Keith Stobie and Victor Braberman (STVR 2010) • Journal paper • Not too long • Otherwise, we prefer conference papers • Follows: personal slides on models for data, slides on MBT, questions • Thanks to Wolfgang Grieskamp for MBT slides

  4. Developed Two Kinds of Test Abstractions • Declarative: describe what inputs look like • Two kinds of languages used • Declarative language for properties of desired inputs[SOFTMC’01, ASE’01, FME’02, OOPSLA’02, SAT’03, MIT’03, J-ASE’04, SAT’05, LDTA’06, ALLOY’06] • Properties written in the Alloy modeling language • Uses Alloy Analyzer for generation of test inputs • Imperative language for properties of desired inputs[ISSTA’02, TR’03, MIT’04, ICSE-DEMO’07, STEP’07, FSE’07] • Properties written in implementation language (Java, C#…) • Developed Korat for generation of test inputs • Imperative: describe how to generate inputs • More recent work on ASTGen[FSE’07]

  5. Korat at Microsoft Research • Korat implemented in the AsmLT test tool in Foundations of Software Engineering group • Predicates in Abstract state machine Language (AsmL), not in Java or C# • GUI for setting finitization and manipulating tests • Korat can be used stand-alone or to generate inputs for method sequences • Some extensions • (Controlled) non-exhaustive generation • Generation of complete tests from partial tests • Library for faster generation for common datatypes

  6. AsmLT/Korat at Microsoft • Used by testers in several product groups • Enabled finding numerous errors • XML tools • XPath compiler (10 code errors, test-suite augmentation) • Serialization (3 code errors, changing spec) • Web-service protocols • WS-Policy (13 code errors, 6 problems in informal spec) • WS-Routing (1 code error, 20 problems in informal spec) • Others • SSLStream • MSN Authentication • … • Errors found in • Important real-world applications • Code already well tested using best testing practices

  7. Some Comments from Microsoft • Positive comments on AsmLT and Korat • “So far our stateless AsmL models are pretty successful.” • “AsmL parameter generation tool is quite convenient and powerful.” • Negative comments on AsmLTnot Korat • “Most of our testers prefer to write as much C# as possible.” • “Very difficult to debug AsmL.” • Result: new tool for C# (.NET) SpecExplorer Korat is Korat

  8. Questions for Discussion (1) • Are there any open source tools equivalent to Spec Explorer? [KN] • How would this methodology change if applied to something but protocols? [JT] • In the case where the number of internal unit-tests that need to be tested, is it worth it at all to do MBT? [AY] • What's the right time to apply model-based testing? [XW] • What other selection strategies exist but transition coverage? [AK]

  9. Questions for Discussion (2) • … possible to scale up MBT to make it work on large-sized systems? [AD] • Does model based testing scale down to small projects? [DeG] • NLP … to aid in … converting technical documentation into testing reqs? [CM] • Is it possible to reverse engineer a spec from a program or test suite? [SB] • test models generated automatically? [YL] • How to totally/partially automate slicing? [FS] • How does the slicing affect the success? [SO]

  10. Questions for Discussion (3) • integrate with regression testing? [KB] • generate a test that checks to see if an action disallowed by the model program is in fact disallowed by the System Under Test? [DwG] • Can Spec be used to generate proprieties for Chess? [SB] • How are bugs in the model treated? [AK] • Is the modeling language enough to model various types of applications? [SO]

More Related