1.09k likes | 1.2k Views
Isolated Parameterized Unit Testing with Pex and Moles. Nikolai Tillmann, Jonathan “Peli” de Halleux Microsoft Research. Learning objectives After I attend this class I will be able to. Write Unit Tests Coverage, assertions, isolation Use Moles to Isolate Unit Tests
E N D
Isolated Parameterized Unit Testingwith Pex and Moles • Nikolai Tillmann, Jonathan “Peli” de Halleux • Microsoft Research
Learning objectives After I attend this class I will be able to... • Write Unit Tests • Coverage, assertions, isolation • Use Moles to Isolate Unit Tests • Test legacy code • Write Pex Parameterized Unit Tests • Achieve high code coverage
Preparation • We will use Pex for all exercises • Pex includes Moles • Visual Studio 2010 Power Tools • http://research.microsoft.com/Pex • http://www.pexforfun.com • Works with .NET 2, 3.5, 4, x86 and x64 • Visual Studio 2008, 2010, Command line • (Alpha) Silverlight support
Preparation • Install latest publicversion pex.powertool.x86.msi
Quiz: Unit testing • What is a unit test?
Unit Testing • A unit test is a small program with assertions • Test a single (small) unit of code void AddAndCount() { // Arrange int item = 3; // Act var list = new List(); list.Add(item); // Assert Assert.AreEqual(1, list.Count); }
static string ReadFooValue() { string[] lines = File.ReadAllLines(@"t:\myapp.ini"); foreach(var line in lines){ intindex = line.IndexOf('='); string name = line.Substring(0, index); if (name == "Foo") { string value = line.Substring(index); return value; } } return null; } t:\myapp.ini A=B Foo=C C=D
Quiz: Code Coverage • How much block coverage do we need? • 50% • 80% • 100% • Block coverage alone is not enough
Quiz: Coverage • How much block coverage do we need? • 50% • 80% • 100% • Block coverage alone is not enough • Research: no correlation between high code coverage and quality
Quiz: White box testing [TestMethod] void ExistingFoo() { File.WriteAllLines(@"t:\myapp.ini",new string[]{“Foo=b”}); Reader.ReadFooValue();} • Do we need more tests to get 100% cov.? [TestMethod] void MissingFoo() { File.WriteAllLines(@"t:\myapp.ini", new string[0]); Reader.ReadFooValue();}
Quiz: Assertions • Why write Assertions (or not)? • Documentation • Double check your program • Please your manager • Prevent future bugs • Validate user inputs • Catch errors early
Quiz: Assertions • Why write Assertions (or not)? • Documentation • Double check your program • Please your manager • Prevent future bugs • Validate user inputs • Catch errors early
Quiz: Assertions • Which Assertions should you write? • Assert.IsTrue(value == “b”); • Assert.IsTrue(value == null); • Assert.IsTrue(String.IsNullOrEmpty(value)) • Assert.IsTrue(true); • No assertions [TestMethod] void MissingFoo() { File.WriteAllLines(@"t:\myapp.ini",new string[]{“a=b”}); string value = Reader.ReadFooValue(); Assert.IsTrue(????);}
Quiz: Assertions • Which Assertions should you write? • Assert.IsTrue(value == “b”); • Assert.IsTrue(value == null); • Assert.IsTrue(String.IsNullOrEmpty(value)) • Assert.IsTrue(true); • No assertions [TestMethod] void MissingFoo() { File.WriteAllLines(@"t:\myapp.ini",new string[]{“a=b”}); string value = Reader.ReadFooValue(); Assert.IsTrue(????);}
Quiz: Coverage +Assertions • What gives you confidence in the code? • High coverage, few assertions • Low coverage, many assertions • High coverage, many assertions • Low coverage, no assertions • I wrote it
Quiz: Coverage +Assertions • What gives you confidence in the code? • High coverage, few assertions • Low coverage, many assertions • High coverage, many assertions • Research: Experienced developers write good assertions, junior developers write ‘debugging’ assertions • Low coverage, no assertions • I wrote it
string ReadFooValue() { string[] lines = File.ReadAllLines(@"t:\myapp.ini"); foreach(var line in lines){ intindex = line.IndexOf('='); string name = line.Substring(0, index); if (name == "Foo") { string value = line.Substring(index + 1); return value; } } return null; } t:\myapp.ini A=B Foo=C C=D
Quiz: Isolation string ReadFooValue() { string[] lines = File.ReadAllLines(@"t:\myapp.ini"); ... • In the example, what are the external dependencies? • Network Share • Local Disk • No file system, all in memory
Quiz: Isolation string ReadFooValue() { string[] lines = File.ReadAllLines(@"t:\myapp.ini"); ... • In the example, what are the external dependencies? • Network Share • Local Disk • No file system, all in memory
Quiz: Isolation • What is the problem with a Local Disk? • Mapping already exists • Cannot run tests concurrently • Disk full • Access rights
Quiz: Isolation • What is the problem with a Local Disk? • Mapping already exists • Cannot run tests concurrently • Disk full • Access rights
Unit TestingExercise • Map local directory:> mkdir c:\foo> net use t: \\[machinename]\c$\foo • Create C# class library, copy in Reader snippet • Create test project, write unit tests • Run unit tests • Optional: Measure code coverage
Definition of Unit Test • What is a good Unit Test? • A Unit Test is a program that runs fast the code under test, without environment dependencies, withassertions • What is a good Unit Test Suite? • A set of Unit Tests which achieves high code coverage
10 Minutes Break
Dependency hell • Code under test should not depend on hard-coded environment dependencies: • How do you mitigate the Local Disk issues? • Always run on the same machine, same hardware, same credentials, same time, same temperature, same solar system configuration • Refactoring: use Streams • Refactoring: introduce IFileSystem • Refactoring: pass the lines as parameter • Change implementation of File.ReadAllLines var lines = File.ReadAllLines(@"t:\myapp.ini");
Dependency hell • Code under test should not depend on hard-coded environment dependencies: • How do you mitigate the Local Disk issues? • Always run on the same machine, same hardware, same credentials, same time, same temperature, same solar system configuration • Refactoring: use Streams • Refactoring: introduce IFileSystem • Refactoring: pass the lines as parameter • Change implementation of File.ReadAllLines var lines = File.ReadAllLines(@"t:\myapp.ini"); Reality check Refactoring not always an option
Dependency hell • Code under test should not depend on hard-coded environment dependencies: • How do you mitigate the Local Disk issues? • Always run on the same machine, same hardware, same credentials, same time, same temperature, same solar system configuration • Refactoring: use Streams • Refactoring: introduce IFileSystem • Refactoring: pass the lines as parameter • Change implementation of File.ReadAllLines var lines = File.ReadAllLines(@"t:\myapp.ini");
Dependency hell • Code under test should not depend on hard-coded environment dependencies: • How do you changeFile.ReadAllLines? • Override static method • Changing the CLR (and recompiling it) • Rewrite application in JScript • Code instrumentation var lines = File.ReadAllLines(@"t:\myapp.ini");
Dependency hell • Code under test should not depend on hard-coded environment dependencies: • How do you changeFile.ReadAllLines? • Override static method • Changing the CLR (and recompiling it) • Rewrite application in JScript • Code instrumentation – the Moles framework var lines = File.ReadAllLines(@"t:\myapp.ini");
Motivation for Moles • Why another isolation framework? • Specifically designed to enable Pex • Simple, Well-defined semantics • “Replace any .NET method” • Type safe
Moles = Replace any .NET with a delegate var lines = File.ReadAllLines(@"t:\myapp.ini"); What if we could replace File.ReadAllLines? File.ReadAllLines = delegate(string fn) MFile.ReadAllLinesString= delegate(string fn) { return new string[0]; }; Moles
Mole Types Code Generation // System.IO public static class File{ public static string[] ReadAllLines(string fn);} // System.IO.Moles public class MFile{ public static Func<string, string[]> ReadAllLinesString { set; } } // delegate R Func<T, R>(T t);
Injecting Detours at Runtime // System.IO public static class File { public static string[] ReadAllLines(string fn) { if (MFile.ReadAllLinesString != null) return MFile.ReadAllLines(fn); … original code } } Automatically injected at runtime
Quiz: Func<T> • Match the delegates with the methods? • Func<string> • Action • Action<string> • Func<bool,string> • Func<string, bool> • Action<int> • Action<List<T>, int> • Func<string,string[]> • boolFile.Exists(string) • Console.WriteLine(string) • void Flush() • String.Empty {get;} • List<T>.Capacity {set;} • string[] File.ReadAllLines(string)
Quiz: Func<T> • Match the delegates with the methods? • Func<string> • Action • Action<string> • Func<bool,string> • Func<string, bool> • Action<int> • Action<List<T>, int> • Func<string,string[]> • boolFile.Exists(string) • Console.WriteLine(string) • void Flush() • String.Empty {get;} • List<T>.Capacity {set;} • string[] File.ReadAllLines(string)
C# 3.0 Lambdas MFile.ReadAllLinesString = delegate(string fileName) { return new string[]{“a=b”}; }
C# 3.0 Lambdas MFile.ReadAllLinesString = (fileName) => { return new string[]{“a=b”}; }
C# 3.0 Lambdas MFile.ReadAllLinesString = (fileName) => new string[]{“a=b”};
C# 3.0 Lambdas MFile.ReadAllLinesString = fileName => new string[]{“a=b”};
Quiz: Lambdas • Match the Lambdas with the methods • () => “” • () => {} • s => {} • (s) => “” • (s) => false • boolFile.Exists(string) • Console.WriteLine(string) • void Flush(); • String.Empty {get;} • string ToString();
Quiz: Lambdas • Match the Lambdas with the methods • () => “” • () => {} • s => {} • (s) => “” • (s) => false • boolFile.Exists(string) • Console.WriteLine(string) • void Flush(); • String.Empty {get;} • string ToString();
MolesExercise I • Add moles for mscorlibto the test project • Add new Item • Moles and Stubs for Testing • mscorlib.moles • Write test using moles • Run test • Debug test
Exercise II [TestMethod, ...] void ReadFooValueTest() { MFile.BehavedAsNotImplemented(); ... static string ReadFooValue() { if (!File.Exists(@"t:\myapp.ini")) return null; string[] lines = File.ReadAllLines(@"t:\myapp.ini"); ... Exercise II
Constructors and Instance methods static string ReadFooValue() { var reader = new StreamReader(“t:\myapp.ini”); string content = reader.ReadToEnd(); void ReadFooValueTest(string content) { MStreamReader.ConstructorString = delegate(StreamReader me, string file) => { var mole = new MStreamReader(me); mole.ReadToEnd= () => content; };
Exercise II static string ReadFooValue() { var reader = new StreamReader(@"t:\myapp.ini"))var lines = reader.ReadToEnd().Split(‘\n’); foreach(var line in lines){ intindex = line.IndexOf('='); string name = line.Substring(0, index); if (name == "Foo") { string value = line.Substring(index + 1); return value; } } return null; }} Exercise III
Quiz: Moles Usage • When should you use Moles (and not)? • Always use Moles to solve isolation issues • With Moles, one does not need to use interfaces anymore • Moles only should be used for 3rd party API, use interfaces for isolation in your APIs • Moles can be used in production code • Moles lets you get away with untestable APIs • Moles make test cases more robust • With Moles, you do not need integration tests anymore • Moles make tests easier to understand • Moles is for poor programmers, real programmers rely on interfaces
Quiz: Moles Usage • When should you use Moles (and not)? • Always use Moles to solve isolation issues • With Moles, one does not need to use interfaces anymore • Moles only should be used for 3rd party API, use interfaces for isolation in your APIs • Moles can be used in production code • Moles lets you get away with untestable APIs • Moles make test cases more robust • With Moles, you do not need integration tests anymore • Moles make tests easier to understand • Moles is for poor programmers, real programmers rely on interfaces