1 / 109

Isolated Parameterized Unit Testing with Pex and Moles

Isolated Parameterized Unit Testing with Pex and Moles. Nikolai Tillmann, Jonathan “Peli” de Halleux Microsoft Research. Learning objectives After I attend this class I will be able to. Write Unit Tests Coverage, assertions, isolation Use Moles to Isolate Unit Tests

jenny
Download Presentation

Isolated Parameterized Unit Testing with Pex and Moles

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Isolated Parameterized Unit Testingwith Pex and Moles • Nikolai Tillmann, Jonathan “Peli” de Halleux • Microsoft Research

  2. Learning objectives After I attend this class I will be able to... • Write Unit Tests • Coverage, assertions, isolation • Use Moles to Isolate Unit Tests • Test legacy code • Write Pex Parameterized Unit Tests • Achieve high code coverage

  3. Preparation • We will use Pex for all exercises • Pex includes Moles • Visual Studio 2010 Power Tools • http://research.microsoft.com/Pex • http://www.pexforfun.com • Works with .NET 2, 3.5, 4, x86 and x64 • Visual Studio 2008, 2010, Command line • (Alpha) Silverlight support

  4. Preparation • Install latest publicversion pex.powertool.x86.msi

  5. Unit Testing

  6. Quiz: Unit testing • What is a unit test?

  7. Unit Testing • A unit test is a small program with assertions • Test a single (small) unit of code void AddAndCount() { // Arrange int item = 3; // Act var list = new List(); list.Add(item); // Assert Assert.AreEqual(1, list.Count); }

  8. static string ReadFooValue() { string[] lines = File.ReadAllLines(@"t:\myapp.ini"); foreach(var line in lines){ intindex = line.IndexOf('='); string name = line.Substring(0, index); if (name == "Foo") { string value = line.Substring(index); return value; } } return null; } t:\myapp.ini A=B Foo=C C=D

  9. Quiz: Code Coverage • How much block coverage do we need? • 50% • 80% • 100% • Block coverage alone is not enough

  10. Quiz: Coverage • How much block coverage do we need? • 50% • 80% • 100% • Block coverage alone is not enough • Research: no correlation between high code coverage and quality

  11. Quiz: White box testing [TestMethod] void ExistingFoo() { File.WriteAllLines(@"t:\myapp.ini",new string[]{“Foo=b”}); Reader.ReadFooValue();} • Do we need more tests to get 100% cov.? [TestMethod] void MissingFoo() { File.WriteAllLines(@"t:\myapp.ini", new string[0]); Reader.ReadFooValue();}

  12. Quiz: Assertions • Why write Assertions (or not)? • Documentation • Double check your program • Please your manager • Prevent future bugs • Validate user inputs • Catch errors early

  13. Quiz: Assertions • Why write Assertions (or not)? • Documentation • Double check your program • Please your manager • Prevent future bugs • Validate user inputs • Catch errors early

  14. Quiz: Assertions • Which Assertions should you write? • Assert.IsTrue(value == “b”); • Assert.IsTrue(value == null); • Assert.IsTrue(String.IsNullOrEmpty(value)) • Assert.IsTrue(true); • No assertions [TestMethod] void MissingFoo() { File.WriteAllLines(@"t:\myapp.ini",new string[]{“a=b”}); string value = Reader.ReadFooValue(); Assert.IsTrue(????);}

  15. Quiz: Assertions • Which Assertions should you write? • Assert.IsTrue(value == “b”); • Assert.IsTrue(value == null); • Assert.IsTrue(String.IsNullOrEmpty(value)) • Assert.IsTrue(true); • No assertions [TestMethod] void MissingFoo() { File.WriteAllLines(@"t:\myapp.ini",new string[]{“a=b”}); string value = Reader.ReadFooValue(); Assert.IsTrue(????);}

  16. Quiz: Coverage +Assertions • What gives you confidence in the code? • High coverage, few assertions • Low coverage, many assertions • High coverage, many assertions • Low coverage, no assertions • I wrote it

  17. Quiz: Coverage +Assertions • What gives you confidence in the code? • High coverage, few assertions • Low coverage, many assertions • High coverage, many assertions • Research: Experienced developers write good assertions, junior developers write ‘debugging’ assertions • Low coverage, no assertions • I wrote it

  18. string ReadFooValue() { string[] lines = File.ReadAllLines(@"t:\myapp.ini"); foreach(var line in lines){ intindex = line.IndexOf('='); string name = line.Substring(0, index); if (name == "Foo") { string value = line.Substring(index + 1); return value; } } return null; } t:\myapp.ini A=B Foo=C C=D

  19. Quiz: Isolation string ReadFooValue() { string[] lines = File.ReadAllLines(@"t:\myapp.ini"); ... • In the example, what are the external dependencies? • Network Share • Local Disk • No file system, all in memory

  20. Quiz: Isolation string ReadFooValue() { string[] lines = File.ReadAllLines(@"t:\myapp.ini"); ... • In the example, what are the external dependencies? • Network Share • Local Disk • No file system, all in memory

  21. Quiz: Isolation • What is the problem with a Local Disk? • Mapping already exists • Cannot run tests concurrently • Disk full • Access rights

  22. Quiz: Isolation • What is the problem with a Local Disk? • Mapping already exists • Cannot run tests concurrently • Disk full • Access rights

  23. Unit TestingExercise • Map local directory:> mkdir c:\foo> net use t: \\[machinename]\c$\foo • Create C# class library, copy in Reader snippet • Create test project, write unit tests • Run unit tests • Optional: Measure code coverage

  24. Definition of Unit Test • What is a good Unit Test? • A Unit Test is a program that runs fast the code under test, without environment dependencies, withassertions • What is a good Unit Test Suite? • A set of Unit Tests which achieves high code coverage

  25. 10 Minutes Break

  26. Isolation withMoles

  27. Dependency hell • Code under test should not depend on hard-coded environment dependencies: • How do you mitigate the Local Disk issues? • Always run on the same machine, same hardware, same credentials, same time, same temperature, same solar system configuration • Refactoring: use Streams • Refactoring: introduce IFileSystem • Refactoring: pass the lines as parameter • Change implementation of File.ReadAllLines var lines = File.ReadAllLines(@"t:\myapp.ini");

  28. Dependency hell • Code under test should not depend on hard-coded environment dependencies: • How do you mitigate the Local Disk issues? • Always run on the same machine, same hardware, same credentials, same time, same temperature, same solar system configuration • Refactoring: use Streams • Refactoring: introduce IFileSystem • Refactoring: pass the lines as parameter • Change implementation of File.ReadAllLines var lines = File.ReadAllLines(@"t:\myapp.ini"); Reality check Refactoring not always an option

  29. Dependency hell • Code under test should not depend on hard-coded environment dependencies: • How do you mitigate the Local Disk issues? • Always run on the same machine, same hardware, same credentials, same time, same temperature, same solar system configuration • Refactoring: use Streams • Refactoring: introduce IFileSystem • Refactoring: pass the lines as parameter • Change implementation of File.ReadAllLines var lines = File.ReadAllLines(@"t:\myapp.ini");

  30. Dependency hell • Code under test should not depend on hard-coded environment dependencies: • How do you changeFile.ReadAllLines? • Override static method • Changing the CLR (and recompiling it) • Rewrite application in JScript • Code instrumentation var lines = File.ReadAllLines(@"t:\myapp.ini");

  31. Dependency hell • Code under test should not depend on hard-coded environment dependencies: • How do you changeFile.ReadAllLines? • Override static method • Changing the CLR (and recompiling it) • Rewrite application in JScript • Code instrumentation – the Moles framework var lines = File.ReadAllLines(@"t:\myapp.ini");

  32. Motivation for Moles • Why another isolation framework? • Specifically designed to enable Pex • Simple, Well-defined semantics • “Replace any .NET method” • Type safe

  33. Moles = Replace any .NET with a delegate var lines = File.ReadAllLines(@"t:\myapp.ini"); What if we could replace File.ReadAllLines? File.ReadAllLines = delegate(string fn) MFile.ReadAllLinesString= delegate(string fn) { return new string[0]; }; Moles

  34. Mole Types Code Generation // System.IO public static class File{ public static string[] ReadAllLines(string fn);} // System.IO.Moles public class MFile{ public static Func<string, string[]> ReadAllLinesString { set; } } // delegate R Func<T, R>(T t);

  35. Injecting Detours at Runtime // System.IO public static class File { public static string[] ReadAllLines(string fn) { if (MFile.ReadAllLinesString != null) return MFile.ReadAllLines(fn); … original code } } Automatically injected at runtime

  36. Demo

  37. Quiz: Func<T> • Match the delegates with the methods? • Func<string> • Action • Action<string> • Func<bool,string> • Func<string, bool> • Action<int> • Action<List<T>, int> • Func<string,string[]> • boolFile.Exists(string) • Console.WriteLine(string) • void Flush() • String.Empty {get;} • List<T>.Capacity {set;} • string[] File.ReadAllLines(string)

  38. Quiz: Func<T> • Match the delegates with the methods? • Func<string> • Action • Action<string> • Func<bool,string> • Func<string, bool> • Action<int> • Action<List<T>, int> • Func<string,string[]> • boolFile.Exists(string) • Console.WriteLine(string) • void Flush() • String.Empty {get;} • List<T>.Capacity {set;} • string[] File.ReadAllLines(string)

  39. C# 3.0 Lambdas MFile.ReadAllLinesString = delegate(string fileName) { return new string[]{“a=b”}; }

  40. C# 3.0 Lambdas MFile.ReadAllLinesString = (fileName) => { return new string[]{“a=b”}; }

  41. C# 3.0 Lambdas MFile.ReadAllLinesString = (fileName) => new string[]{“a=b”};

  42. C# 3.0 Lambdas MFile.ReadAllLinesString = fileName => new string[]{“a=b”};

  43. Quiz: Lambdas • Match the Lambdas with the methods • () => “” • () => {} • s => {} • (s) => “” • (s) => false • boolFile.Exists(string) • Console.WriteLine(string) • void Flush(); • String.Empty {get;} • string ToString();

  44. Quiz: Lambdas • Match the Lambdas with the methods • () => “” • () => {} • s => {} • (s) => “” • (s) => false • boolFile.Exists(string) • Console.WriteLine(string) • void Flush(); • String.Empty {get;} • string ToString();

  45. MolesExercise I • Add moles for mscorlibto the test project • Add new Item • Moles and Stubs for Testing • mscorlib.moles • Write test using moles • Run test • Debug test

  46. Exercise II [TestMethod, ...] void ReadFooValueTest() { MFile.BehavedAsNotImplemented(); ... static string ReadFooValue() { if (!File.Exists(@"t:\myapp.ini")) return null; string[] lines = File.ReadAllLines(@"t:\myapp.ini"); ... Exercise II

  47. Constructors and Instance methods static string ReadFooValue() { var reader = new StreamReader(“t:\myapp.ini”); string content = reader.ReadToEnd(); void ReadFooValueTest(string content) { MStreamReader.ConstructorString = delegate(StreamReader me, string file) => { var mole = new MStreamReader(me); mole.ReadToEnd= () => content; };

  48. Exercise II static string ReadFooValue() { var reader = new StreamReader(@"t:\myapp.ini"))var lines = reader.ReadToEnd().Split(‘\n’); foreach(var line in lines){ intindex = line.IndexOf('='); string name = line.Substring(0, index); if (name == "Foo") { string value = line.Substring(index + 1); return value; } } return null; }} Exercise III

  49. Quiz: Moles Usage • When should you use Moles (and not)? • Always use Moles to solve isolation issues • With Moles, one does not need to use interfaces anymore • Moles only should be used for 3rd party API, use interfaces for isolation in your APIs • Moles can be used in production code • Moles lets you get away with untestable APIs • Moles make test cases more robust • With Moles, you do not need integration tests anymore • Moles make tests easier to understand • Moles is for poor programmers, real programmers rely on interfaces

  50. Quiz: Moles Usage • When should you use Moles (and not)? • Always use Moles to solve isolation issues • With Moles, one does not need to use interfaces anymore • Moles only should be used for 3rd party API, use interfaces for isolation in your APIs • Moles can be used in production code • Moles lets you get away with untestable APIs • Moles make test cases more robust • With Moles, you do not need integration tests anymore • Moles make tests easier to understand • Moles is for poor programmers, real programmers rely on interfaces

More Related