Automating Coded UI Testing using Microsoft Testing Tools in Visual Studio 2012

1. Automating Coded UI Testing using Microsoft Testing Tools in Visual Studio 2012 Clive G Hermann

2. Introductions • Name • Title/Function • Job Responsibility • Testing Experience • C# Experience • Expectations for the Course

3. Course Materials • Lab Manual - Printed • Course notes PowerPoint Slides • Virtual Machines for TFS and VS 2012 • Links and Whitepapers (Favorites)

4. Course Contents • Overview of Automated Code UI Testing • Walkthrough steps from beginning-to-end • Creating Manual Tests • Creating a coded UI test Using Coded UI Test Builder • Automating Coded UI Test Cases

5. Course Contents2 • Data Driven Coded UI Tests • Anatomy of Code UI Tests • Introduction to Lifecycle Management • Extending Coded UI Tests and Action Recordings • Testing Large Applications with Multiple UI Maps • Using HTML5 Controls in Coded UI Tests • Visual Studio 2012 ALM Features • Upgrading Coded UI Tests from Visual Studio 2010

6. Course Contents3 • Using WPF with Coded UI Tests • Best Practices for Coded UI Tests • Using the Coded UI Test Logs • Using SharePoint with Coded UI Tests • ExcelCodedUIHelper

7. 1) Overview of Automated Coded UI Testing

8. 1) Overview of Automated Code UI Testing • UI Testing • Functional Testing • Controls • User Interface • Whole Application • The earlier we catch a “bug” or potential problem the less it costs! • Image • Manpower

9. Reality of Automated Testing • 70% of all Testing is manual

10. What is Coded UI Testing • Replacing a Manual User Interface Test with an Automated Test. • To Allow: • Replication • Consistency • Regression Testing For: • Multiple Environments • Multiple Versions

11. IT Has a Problem

12. Test Early and Often • Catching defects as early as possible is the least expensive way to ensure software quality. • Kent Beck and Cynthia Andres wrote • "Here is the dilemma in software development: defects are expensive, but eliminating defects is also expensive”. • “However, most defects end up costing more than it would have cost to prevent them."

13. The Testing TrendsForwards • Test-Driven Development • Design the Test • Then Write the Code

14. The Testing Trends:Backwards and Circular • Linking it all together • Help-Desk • Test • Story Board • Task • Unit • Programmer

15. The Ultimate Goal of Software Testing • To find the bugs before your users do! • UI Testing is END USER TESTING • OR TOTAL SYSTEM TESTING • OR INFRASTRUCTURE TESTING

16. Manual to Coded UI

17. Testers’ Multiple Roles • Guarantee SOFTWARE QUALITY • Regression Test each Version or Build • Test all Environments • Provide Developers with all the information required: • To REPRODUCE THE “BUG” • Trace the steps to get the BUG • Screen shots, IntelliTrace, Logs and Stack Traces • NO-REPRO must be ELIMINATED

18. Tiers of Software Testing • Clear Separation of Functions • Clear Definition of Communication and Collaboration • TEAM of Software Quality

19. Module 2 Walkthrough steps from beginning-to-end

20. 2)Walkthrough steps from beginning-to-end • Lesson 1 Manual Testing • Using Microsoft Test Manger to create and manage Test Plans • Running Test Cases and Publishing Results • Conducting Exploratory Testing • Fast Forward for Manual Testing • Lesson 2 Coded User Interface Testing • Automated Functional Tests – Using CUIT • Create a CUIT from Scratch • Create a CUIT from action recording • Making CUIT more Robust and Flexible

21. A manual testing process

22. Building an AppFor Manual Testing • Demo of Simple Calculator

23. Automating the Calculator • Demo

24. Adding Data • Demo

25. Introducing an Error • Demo

26. Lab 1 • The Calculator

27. 3 Creating Manual Tests

28. Visual Studio 2012Supported Test Types

29. Microsoft Test Manager • Provides a single environment to author and manage Tests

32. Quick Start Guide for Manual Testing using Microsoft Test Manager • http://msdn.microsoft.com/en-us/library/dd380763.aspx

33. Easily reproducing issues through manual testing • Introducing Managed Exploratory Testing • The 5 Minute “elevator” Video • http://channel9.msdn.com/Series/Visual-Studio-2012-Premium-and-Ultimate-Overview/Visual-Studio-Ultimate-2012-Easily-reproducing-issues-through-manual-testing

34. 4) Creating a coded UI test Using Coded UI Test Builder

35. 4) Creating a coded UI test Using Coded UI Test Builder • Test Types • Microsoft Test Manager • Exploratory Testing • Easily reproducing issues through manual testing

36. Test TypesAuthoring a TEST case • Demo • Lab

37. Microsoft Test Manager • The first time you run MTM it will prompt • Add Team Foundation Manager • “If you don’t know the Name, contact your administrator” • It might require “ServerName:Port Number”

38. TFS and MTM

39. Connecting Microsoft Test Manager to TFS • MTM is always connected to TFS

40. Test Plan Properties Figure 21-5. Test plan properties

41. MTM Test Suites • Static suite: The content of this suite is manually added test cases. • Query-based suite: A query-based suite lists all test cases matching a given work item filter. • Requirements-based suite: This suite shows the test cases associated with a selected TFS requirement.

44. With Visual Studio 2012 Lab Management it is possible to create an environment based on existing infrastructure without installing SCVMM. This is great because we can now start using Lab Management without any additional installation required. The lab machines in a standard environment can be physical or virtual (or both) and the virtual machine does not have to be managed by SCVMM or Hyper-V. • To create a standard environment you only need to know the names of the machine you want to add to the environment and a user account to be used to log in and deploy the test agent service onto the machine. • A standard environment can be connected to directly from MTM and the Test Runner making the integration with lab environments really smooth.

45. The alternative to standard environments is to use a SCVMM environment. The key features of a SCVMM environment include • •Create new machines from templates. New lab machines can automatically be created by anyone (having permissions) running MTM. It is no longer required to send a request to the operations department to get machines created for a new environment or to do other maintenance operations such as starting, stopping, snapshotting, or restoring machines. • •Support for snapshots. It is possible to work with snapshots from MTM, which enable testers to save state during testing. This is a powerful feature both to unlock the tester if a test case fails (no need to block an environment while waiting for a resolution) but also to give a developer a fixed point in time to reproduce a bug. • •Connection inside MTM. A SCVMM environment can be connected to MTM and the Test Runner just like with the standard environment.

46. There are a number of metrics to can use as KPIs for software testing. • •Number of bugs per state. How many bugs do we have that are active, resolved, or closed? Are the number of active bugs increasing and number of resolved and closed bugs constant? Then we need to look into how we perform our testing. • •Number of bugs sent back from testers for more information. A large number might indicate that communication between developers and testers must improve. • •Code coverage. This shows us how much of the code has been covered by our automated unit tests. We get the value as a percentage of the whole code. • •Tests run results. How are our tests performing? Do we have many failed tests? If so, what can we do to improve this? • •Percent Requirements covered by test cases. Do we write test cases for all our requirements? If not, what is the reason? • •Percent Requirements covered by testing. Do we actually run the tests we have test cases for? If this figure is low and the figure for Percent Requirements covered by test cases is high we might have an issue we need to deal with.

47. The application tier is the core of TFS. Here we find all services TFS offers. • First we find all web services for team project collections. We all recognize this so no further explanation is necessary: • •Version control • •Build service • •Lab management • •Work Item tracking • •Team Foundation Framework services. • This set of services is running on the Application Tier of TFS and allows us to access the different aspects of Team Foundation Server, including administration, security, and events. They enable us to extend Team Foundation functionality by developing TFS extensions and adapters to integrate third-party tools with TFS 2012. • We also find the Application Web services: