1 / 13

Code Contracts Parameterized Unit Tests

Code Contracts Parameterized Unit Tests. Tao Xie. ?. =. Example Unit Test Case. +. Expected Outputs. Test inputs. Program. Outputs. Test Oracles. void addTest() { ArrayList a = new ArrayList( 1 ) ; Object o = new Object(); a.add(o); AssertTrue(a.get( 0 ) == o); }.

ulema
Download Presentation

Code Contracts Parameterized Unit Tests

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Code Contracts Parameterized Unit Tests Tao Xie

  2. ? = Example Unit Test Case + Expected Outputs Test inputs Program Outputs Test Oracles void addTest() { ArrayList a = new ArrayList(1); Object o = new Object(); a.add(o); AssertTrue(a.get(0) == o); } Test Case = Test Input + Test Oracle • Appropriate method sequence • Appropriate primitive argument values • Appropriate assertions

  3. Levels of Test Oracles • Expected output for an individual test input • In the form of assertions in test code • Properties applicable for multiple test inputs • Crash (uncaught exceptions) or not, related to robustness issues, supported by most tools • Properties in production code: Design by Contract (precondition, postcondition, class invariants) supported by ParasoftJtest, Google CodeProAnalytiX • Properties in test code: Parameterized unit tests supported by MSR Pex, AgitarOne X. Xiao, S. Thummalapenta, and T. Xie. Advances on Improving Automation in Developer Testing. In Advances in Computers, 2012 http://people.engr.ncsu.edu/txie/publications.htm#ac12-devtest

  4. Economics of Test Oracles • Expected output for an individual test input • Easy to manually verify for one test input • Expensive/infeasible to verify for many test inputs • Limited benefits: only for one test input • Properties applicable for multiple test inputs • Not easy to write (need abstraction skills) • But once written, broad benefits for multiple test inputs

  5. Assert behavior of multiple test inputs Design by Contract • Example tools: ParasoftJtest, Google CodeProAnalytiX, MSR Code Contracts, MSR Pex • Class invariant: properties being satisfied by an object (in a consistent state) [AgitarOne allows a class invariant helper method used as test oracles] • Precondition: conditions to be satisfied (on receiver object and arguments) before a method can be invoked • Postcondition: properties being satisfied (on receiver object and return) after the method has returned • Other types of specs also exist http://research.microsoft.com/en-us/projects/contracts/

  6. Microsoft Research Code Contracts publicvirtualint Add(object value){ Contract.Requires( value != null );Contract.Ensures( Count == Contract.OldValue(Count) + 1 ); Contract.Ensures( Contract.Result<int>() == Contract.OldValue(Count) );if(count == items.Length) EnsureCapacity(count + 1); items[count] = value; returncount++;} • Features • Language expression syntax • Type checking / IDE • Declarative • Special Encodings • Result and Old [ContractInvariantMethod]voidObjectInvariant() {Contract.Invariant( items != null );} - Slide adapted from MSR RiSE http://research.microsoft.com/en-us/projects/contracts/

  7. Parameterized Unit Testing [Tillmann&Schulte ESEC/FSE 05] • Parameterized Unit Test = Unit Test with Parameters • Separation of concerns • Data is generated by a tool • Developer can focus on functional specification void TestAdd(List list, int item) { Assume.IsTrue(list != null); var count = list.Count; list.Add(item); Assert.AreEqual(count + 1, list.Count); } http://research.microsoft.com/apps/pubs/default.aspx?id=77419

  8. Parameterized Unit Testsare Formal SpecificationsAlgebraic Specifications • A Parameterized Unit Test can be read as a universally quantified, conditional axiom. void TestReadWrite(Res r, string name, string data) {Assume.IsTrue(r!=null & name!=null && data!=null); r.WriteResource(name, data);Assert.AreEqual(r.ReadResource(name), data); }  string name, string data, Res r: r ≠ null ⋀ name ≠ null ⋀ data ≠ null ⇒ equals( ReadResource(WriteResource(r, name, data).state, name), data)

  9. Parameterized Unit Tests in Pex http://research.microsoft.com/pex/

  10. Parameterized Unit TestingGetting Popular Parameterized Unit Tests (PUTs) commonly supported by various test frameworks • .NET: Supported by .NET test frameworks • http://www.mbunit.com/ • http://www.nunit.org/ • … • Java: Supported by JUnit 4.X • http://www.junit.org/ Generating test inputs for PUTs supported by tools • .NET: Supported by Microsoft Research Pex • http://research.microsoft.com/Pex/ • Java: Supported by AgitarAgitarOne • http://www.agitar.com/

  11. ParameterizedTest-Driven Development Bug in PUT Write/refine Contract as PUT Bug in Code Write/refine Code of Implementation Fix-it (with Pex), Debug with generated tests failures Run Pex no failures Use Generated Tests for Regression

  12. SoftwareAgitation Observationson code behavior, plus Test Coverage data Code If an Observationreveals a bug, fix it Compile If it describes desired behavior, click to create a Test Assertion Code Agitate Review Assert behavior of multiple test inputs Software Agitation in AgitarOne http://www.agitar.com/ - Slide adapted from Agitar Software Inc.

  13. Software Agitation in AgitarOne Image from http://www.agitar.com/

More Related