USER INTERfACE TESTING

1. USER INTERfACE TESTING Lecture 9

2. What Is? • graphical user interface testing is the process of testing a product's graphical user interface to ensure it meets its written specifications • GUI testing is a process to test application's user interface and to detect if application is functionally correct • GUI testing involves carrying set of tasks and comparing the result of same with the expected output and ability to repeat same set of tasks multiple times with different data input and same level of accuracy • GUI testing includes how the application handles keyboard and mouse events, how different GUI components like menubars, toolbars, dialogs, buttons, edit fields, list controls, images etc. reacts to user input and whether or not it performs in the desired manner. • Implementing GUI testing early in the software development cycle speeds up development, improves quality and reduces risks towards the end of the cycle

3. UI Testing • GUIs make software easy to use, and developers are dedicating a larger portion of code to implementing them. • GUIs can constitute as much as 60 percent of an application’s total code today. • The use of GUIs in safety-critical systems is also growing, making their correct operation imperative. • testing GUIs for correctness can enhance the entire system’s safety, robustness, and usability

4. UI Testing • Current GUI testing techniques are incomplete, ad hoc, and largely manual. • The most common tools use record-playback techniques. A test designer interacts with the GUI, generating mouse and keyboard events. • The tool records the user events, captures the GUI session screens, and then stores the session—usually as a script. • The tester later plays back the recorded sessions to re-create the events with different inputs. • This process is extremely labor intensive, often relying on the test designer’s ability to generate interesting GUI interactions. • An automatic test case generator can provide a higher level of support, but a programmer must code it for all possible decision points in the GUI. • The record-playback approach misses important GUI decisions. A popular alternative is to release beta copies of the software and let the users do part of the testing (Microsoft tested part of its Windows 95 software by releasing almost 400,000 beta copies).

5. GUI TESTING PITFALLS • Software testing is already labor and resource intensive—often accounting for 50 to 60 percent of total software development costs • GUI testing - difficulties that traditional software testing techniques do not adequately address • Conventional software uses coverage criteria as a guideline for determining testing adequacy (define sets of rules that test designers use to determine the type and amount of underlying code to test) • a criterion for “event coverage” might require all reachable events in a GUI to execute at least once during a complete cycle of test cases.

6. GUI TESTING PITFALLS • traditional coverage criteria do not work well for GUIs. • GUI software differs from the underlying application code in its level of abstraction, so mapping between GUI events and the underlying code is not straightforward. • Code-based coverage criteria do not necessarily address problematic interactions between the GUI’s user events and the application. • even when experienced test designers focus on specific parts of a GUI, they may still find it impractical to generate all possible test cases for these parts. • If the designers have to generate a subset of all possible test cases, they often must select the subset during test case generation. The difficulty of anticipating a test case’s fault-detection capability makes it difficult, in turn, to select the most effective subset

7. Test Oracles • Verifying whether the GUI executes correctly poses a problem. • The traditional verification tool is a test oracle— a separate program that generates expected results for a test case and compares them with actual results. • In conventional software testing, the tester invokes the oracle after the test case executes and compares the final output with the oracle’s expected output. • By contrast, a GUI test case requires interleaving the oracle invocation with the test case execution because an incorrect GUI state can lead to an unexpected screen, which in turn can make further test case execution useless (for example, the test case might involve a button on the GUI screen that no longer exists).

8. Test Oracles • a GUI test case should terminate as soon as the oracle detects an error. • if the oracle does not verify the GUI after each execution step in a test case, pinpointing the error’s actual cause can become difficult, especially when the final output is correct but the intermediate outputs are incorrect. • Consequently, in GUI test case execution, a tester gives the inputs one step at a time and compares the expected output with the GUI’s output after each step. • This interleaving complicates GUI testing.

9. Regression Testing • regression testing -ensure that code modifications do not introduce new errors into already tested code. • Regressing the testing - special challenges for GUIs • both inputs and outputs to a GUI depend on the layout of graphical elements. • Changes in the layout—button placements, menu organization, and so on—can change the input-output mapping and render older test cases useless. • the expected outputs used by oracles may become obsolete. • developers typically use rapid prototyping for GUIs - requires efficient regression testing mechanisms that can detect the frequent software modifications and adapt the old test cases to them.

10. GUI TESTING PROCESS • Addressing the specific pitfalls of GUI testing requires a clear methodology that employs tools and techniques integrated to use a standardized GUI representation. • This integration ensures that all test results are compatible with each other. • Other goals include • task automation to simplify the test designer’s work; • efficient overall test cycle to minimize the frustration of an inherently tedious and expensive process; • robust testing algorithms to detect each time the GUI enters an unexpected state and to report all information necessary to debug the GUI • portable tools and techniques to allow test information—test cases, oracle results, coverage reports, and error reports—generated on one platform to be used on all other execution platforms

11. GUI Testing Steps • Determine what to test by defining coverage criteria. A GUI coverage criterion might require the execution of each user interface event to determine whether it behaves correctly. • Generate test case inputs from software specifications and structure. For GUIs, these inputs consist of events such as mouse clicks, menu selections, and object manipulations. • Generate expected output to compare with actual output. In GUIs, the expected output includes screen snapshots and window positions and titles. • Execute test cases and verify output. Test cases execute on the software, and the tester compares the output with the expected output from, for example, an oracle. • Determine whether the GUI was adequately tested. Once all test cases have executed, the tester analyzes the software to check which of its parts were actually tested. In GUIs, the analysis checks events and resulting GUI states. • The last step is especially important in GUI testing, where coverage criteria may not always be available or sufficient.

12. What to Check in GUI Testing • Check all the GUI elements for size, position, width, length and acceptance of characters or numbers. For instance, you must be able to provide inputs to the input fields. • Check you can execute the intended functionality of the application using the GUI • Check Error Messages are displayed correctly • Check for Clear demarcation of different sections on screen • Check Font used in application is readable • Check the alignment of the text is proper • Check the Color of the font and warning messages is aesthetically pleasing • Check that the images have good clarity • Check that the images are properly aligned • Check the positioning of GUI elements for different screen resolution.

13. Manual Based Testing • Under this approach, graphical screens are checked manually by testers in conformance with the requirements stated in business requirements document.

14. Record and Replay • GUI testing can be done using automation tools. This is done in 2 parts. During Record , test steps are captured into the automation tool. During playback, the recorded test steps are executed on the Application Under Test. Example of such tools - QTP

15. Model based testing • A model is a graphical description of system’s behavior. It helps us to understand and predict the system behavior. Models help in a generation of efficient test cases using the system requirements. Following needs to be considered for this model based testing: • Build the model • Determine Inputs for the model • Calculate expected output for the model • Run the tests • Compare the actual output with the expected output • Decision on further action on the model • Some of the modeling techniques from which test cases can be derived: • Charts – Depicts the state of a system and checks the state after some input. • Decision Tables – Tables used to determine results for each input applied • Model based testing is an evolving technique for generating the test cases from the requirements. Its main advantage, compared to above two methods, is that it can determine undesirable states that your GUI can attain.

16. GUI Interaction Testing • GUI behavior is tightly integrated with the context of their usage • As users invoke sequences of events (e.g., ClickOnCancelButton, TypeInText) on GUI widgets (e.g., CancelButton, TextBox), the underlying software responds (typically via the execution of an event handler e.g., an ActionListener) in one of several ways. • This may include a change to the software state, which may impact the execution of subsequent events. • the context, established by the sequence of preceding events, in which an event executes may have an impact on how it executes • This context-sensitive and state-based execution behavior of GUI events creates problems for testing; each event needs to be tested in multiple contexts • Current model-based GUI testing techniques either test only a subset of event sequences by restricting the sequence to length two or length three or use a random method, rather than a systematic one, to test longer sequences

17. GUI Testing • means that a GUI-based software application is tested solely by performing sequences of events on GUI widgets • the correctness of the software is determined by examining only the state of the GUI widgets • the types of faults uncovered are varied • thousands of real faults detected via GUI testing are in the underlying business logic of the application, rather than in the GUI code itself

18. GUI Testing • a representation of events within windows (or components) called an event-flow-graph (EFG) • Much like how a control-flow graph (CFG) encodes all possible execution paths in a program • an EFG represents all possible sequences of events that can be executed on the GUI. • Coverage criteria based on the EFG have provided a notion of coverage of a GUI’s event space for functional correctness • It is usually only possible to satisfy these criteria for length two event sequence coverage because the number of sub-paths grows exponentially with length • these criteria, as is the case with CFG path coverage, are useful from a theoretic perspective; they have limited practical significance.

19. GUI Interaction Testing • there are a large number of events that do not interact with the business logic of the GUI application (those responsible for opening and closing windows) • a refinement of the EFG was developed called an event-interaction graph (EIG). • In an EIG events are modeled that do not pertain to the structural nature of the GUI (opening, closing windows, etc.) but that, instead, interact with the underlying application, called system interaction events. • We refer to testing only these events as GUI interaction testing

20. GUI Interaction Testing • A primary problem for GUI testing is that the length of the event sequence invoked by the user is often unbounded. • There are an enormous number of possible permutations of these events which in turn means the context for testing is very large; testing all possible sequences a user may invoke is not possible. • Example: a user can invoke any of the following events on a drawing canvas in any order: {copy, paste, resize, rotate90, color, erase}. • The sequence <rotate90, color, copy, paste> may behave differently than the sequence <rotate90, color, paste, copy> because execution of the event handler code for copy and paste may differ, e.g., different values of variables may be read/written, different branches or paths may be executed. • This relatively small set of events leads to 36 unique length-two sequences, over 7,500 unique length-five sequences, and more than 60 million unique length-ten sequences

21. Semi-automated UT GUI Tools • JFCUnit • Abbot • Pounder and • Jemmy Module • used to manually create unit GUI test cases, which are then automatically executed. • Assertions are inserted in the test cases to determine whether the classes/methods in the unit under test function correctly • The part of the GUI state space explored by these test cases depends largely on the experience and knowledge of the testers and the quality of the user sessions.

22. Model Based UI Testing • used to automate certain aspects of GUI testing • 2 approaches: • State machine models – depends on “state” interpretation in a GUI • Graph models – based on EFG and EIG - the nodes in these graphs represent GUI events and the edges represent different types of relationships between pairs of events.

23. EFG Models • EFG - all possible event sequences that may be executed on a GUI. • It is a directed graph that contains nodes (one for each event in the GUI) and edges that represent a relationship between events. • An edge from node nxto node ny means that the event represented by ny may be performed immediately after the event represented by nx –this relationship is called follows • The EFG is represented by two sets: • (1) a set of nodes N representing events in the GUI and • (2) a set E of ordered pairs (ex, ey), where {ex, ey} ⊆ N, representing the directed edges in the EFG; (ex, ey) ∈ E iffey follows ex • EFG can be constructed semi-automatically using a reverse engineering technique called GUI Ripping • A GUI Ripper automatically traverses a GUI under test and extracts the hierarchical structure of the GUI and events that may be performed on the GUI- the result of this process is the EFG

24. EIG Models • EIG nodes do not represent events to open or close menus, or open windows. • The result is a more compact, and hence more efficient, GUI model. • An EFG can be automatically transformed into an EIG by using graph rewriting rules

25. Example • a GUI that consists of four events, Cut, Copy, Paste, and Edit. • Figure 1(b) shows the GUI’s EFG; • The four nodes represent the four events; the edges represent the follows relationships. • For example, in this EFG, the event Copy follows Edit, represented by a directed edge from the node labeled Edit to Copy • Figure 1(c) shows the corresponding EIG. • the EIG does not contain the Edit event. • the graph-rewriting rule used to obtain this EIG was to (1) delete Edit because it is a menu-open event, (2) for all remaining events exreplace each edge (ex,Edit) with edge (ex, ey) for each occurrence of edge (Edit, ey), and (3) for all ey, delete all edges (Edit, ey) and store the mapping “ey → (Edit, ey)” for use during test execution. • The GUI’s EIG is fully connected with three nodes representing the three events.

26. Generating Test Cases • A technique to generate test cases, each corresponding to an EIG edge has been developed; these test cases are called smoke tests • Two examples of such length two smoke test cases for our example of Figure 1(c) are <Copy, Cut> and <Cut,Paste>. • There are a total of nine such tests – one for each EIG edge. Because EIG nodes do not represent events to open/close menus or open windows, other events (in this case Edit) needed to reach the EIG events are automatically generated at execution time using the mappings {Cut → (Edit, Cut), Paste → (Edit, Paste), Copy → (Edit, Copy)} stored earlier, yielding an executable test case . • The two test cases will “expand” to <Edit, Copy,Edit, Cut> and <Edit, Cut,Edit,Paste>.

27. UI Testing •  test cases must cover all the functionality of the system and fully exercises the GUI itself. • The difficulty in accomplishing this task is twofold: • to deal with domain size - a GUI has many operations that need to be tested. A relatively small program such as MicrosoftWordPad has 325 possible GUI operations. In a large program, the number of operations can easily be an order of magnitude larger. • to deal with sequences- Some functionality of the system may only be accomplished by following some complex sequence of GUI events. For example, to open a file a user may have to click on the File Menu and then select the Open operation, and then use a dialog box to specify the file name, and then focus the application on the newly opened window. This can become a serious issue when the tester is creating test cases manually. • % • In addition, the tester faces more difficulty when they have to do regression testing.

28. Validation Testing • This emphasizes on the Validation aspect of the Application. • Validation testing mainly depends on the fields set in the dialog box and the functions it has to perform. • common rules that can be applied. 1. For text box fields where value entered has to be numeric check following: • It should accept numbers only. 2. For text box fields where value entered has to be alpha-numeric check following: • It should accept alphabets and numbers only. 3. If the field is a combo box then it has to be checked for following points: • Check the combo box has drop down values in it, it is not empty. • Drop down values should be alphabetically sorted. This might change as per requirement but as standard practices it should be alphabetically sorted. • Selection of any drop down value is displayed on closing and opening the same dialog box. • Avoid displaying the first default value in the list.

29. Validation Testing 4. If the field is a list box then it has to be checked for following points: • Check the list box has values in it, it is not empty. • List box values should be alphabetically sorted and displayed. This might change as per requirement but as standard practices it should be alphabetically sorted. • Selection of any list box value should put a check before the value and houlddisplay the correct value(s) selected on closing and opening of the same dialog box. • If the list box supports multiple selection then check whether multiple values can be selected. 5. If the field is a list of radio button then it has to be checked for following points: • Check whether as per requirements all the values are listed. • Same selected value should be displayed on closing and opening of the same dialog box. 6. Data Controls are to be tested as part of functionality testing.

30. Coded User Interface Tests (MS) • Coded UI Tests – automated tests that drive your application through the interface • Include functional testing of UI controls • let you verify that the whole application, including its user interface, is functioning correctly • Coded UI Tests are particularly useful where there is validation or other logic in the user interface, for example in a web page. • They are also frequently used to automate an existing manual test.

31. Coded User Interface Tests (MS)

32. Coded User Interface Tests (MS) • Creating a CUIT: • You perform the test manually while the CUIT Test Builder runs in the background. • You can also specify what values should appear in specific fields. • The CUIT Test Builder records your actions and generates code from them. • After the test is created, you can edit it in a specialized editor that lets you modify the sequence of actions.

33. Web application UI checklist COLORS • Are hyperlink colors standard? • Are the field backgrounds the correct color? • Are the field prompts the correct color? • Are the screen and field colors adjusted correctly for non-editable mode? • Does the site use (approximately) standard link colors? • Are all the buttons in standard format and size? • Is the general screen background the correct color? • Is the page background (color) distraction free?

34. Web application UI checklist CONTENT • All fonts to be the same • Are all the screen prompts specified in the correct screen font? • Does content remain if you need to go back to a previous page, or if you move forward to another new page? • Is all text properly aligned? • Is the text in all fields specified in the correct screen font? • Are all the heading left aligned • Does the first letter of the second word appears in lowercase?

35. Web application UI checklist IMAGES • Are all graphics properly aligned? • Are graphics being used the most efficient use of file size? • Are graphics optimized for quick downloads? • Assure that command buttons are all of similar size and shape, and same font & font size. • Banner style & size & display exact same as existing windows • Does text wrap properly around pictures/graphics? • Is it visually consistent even without graphics?

36. Web application UI checklist INSTRUCTIONS • Is all the error message text spelt correctly on this screen? • Is all the micro-help text(i.e tool tip) spelt correctly on this screen? • Microhelp text(i.e tool tip) for every enabled field & button • Progress messages on load of tabbed(active screens) screens

37. Web application UI checklist NAVIGATION • Are all disabled fields avoided in the TAB sequence? • Are all read-only fields avoided in the TAB sequence? • Can all screens accessible via buttons on this screen be accessed correctly? • Does a scrollbar appear if required? • Does the Tab Order specified on the screen go in sequence from Top Left to bottom right? This is the default unless otherwise specified. • Is there a link to home on every single page? • On open of tab focus will be on first editable field • When an error message occurs does the focus return to the field in error when the user cancels it?

38. Web application UI checklist USABILITY • Are all the field prompts spelt correctly? • Are fonts too large or too small to read? • Are names in command button & option box names are not abbreviations. • Assure that option boxes, option buttons, and command buttons are logically grouped together in clearly demarcated areas "Group Box" • Can the typical user run the system without frustration? • Do pages print legibly without cutting off text? • Does the site convey a clear sense of its intended audience? • Does the site have a consistent, clearly recognizable "look-&-feel"? • Does User cab Login Member Area with both UserName/Email ID ? • Does the site look good on 640 x 480, 600x800 etc.? • Does the system provide or facilitate customer service? i.e. responsive, helpful, accurate? • Is all terminology understandable for all of the site’s intended users?

39. Web Applications Testing • 2 types of web sites: • Provide services • Provide content • need to understand user behavior on the site and gather key usage pattern - analytic code • Analytic code -a java script snippet which is included in all the pages of your website. • does not have any behavioral impact on the application and is invisible to the end-user in most of the cases. • every time a page is loaded, these scripts get executed and store key information about the users browsing pattern

40. Analytic code • Information provided by analytic code: • From which link users have come to this page • For how long users are staying on this page • Where are users going from this page • From which part of the world users are coming • Which browser is used • information is extremely valuable for the organization - marketing activities. • useful for people who want to advertise on a the site as it gives them data about the popularity of site.

41. Test Automation for Web Applications • Test automation - using a software tool to run repeatable tests against the application to be tested • Test automation supports: • Frequent regression testing • Rapid feedback to developers • Virtually unlimited iterations of test case execution • Support for Agile and extreme development methodologies • Disciplined documentation of test cases • Customized defect reporting • Finding defects missed by manual testing • Advantages: • repeatability of the tests • the speed at which the tests can be executed. • Selenium - the most widely-used open source solution

42. Selenium • Javascriptframework that runs in a web browser • Works anywhere Javascript is supported • other supported languages Java, Ruby, Python • Can simulate a user navigating through pages and then assert for specific marks on the pages • HTML knowledge needed to start using it right away

43. Selenium • Selenium is a set of different software tools each with a different approach to supporting test automation • operations are highly flexible, allowing many options for locating UI elements and comparing expected test results against actual application behavior. • support for executing one’s tests on multiple browser platforms

44. Testing Static Content • The simplest type of test, a content test, is a simple test for the existence of a static, non-changing, UI element. • Does each page have its expected page title? This can be used to verify your test found an expected page after following a link. • Does the application’s home page contain an image expected to be at the top of the page? • Does each page of the website contain a footer area with links to the company contact page, privacy policy, and trademarks information? • Does each page begin with heading text using the <h1> tag? Does each page have the correct text within that header? • If your page content is not likely to be affected then it may be more efficient to test page content manually.

45. Testing links • A frequent source of errors for web-sites is broken links or missing pages behind links • Testing involves clicking each link and verifying the expected page. • If static links are infrequently changed then manual testing may be sufficient. • if web designers frequently alter links, or if files are occasionally relocated, link tests should be automated.

46. Function Tests • tests of a specific function within your application, requiring some type of user input, and returning some type of results. • will involve multiple pages with a form-based input page containing a collection of input fields, Submit and Cancel operations, and one or more response pages • User input can be via text-input fields, check boxes, drop-down lists, or any other browser-supported input. • Function tests are often the most complex tests you’ll automate, but are usually the most important. • Typical tests can be for login, registration to the site, user account operations, account settings changes, complex data retrieval operations, among others. • Function tests typically mirror the user-scenarios used to specify the features and design or your application.