1 / 57

Chapter 19: GUI Testing

Chapter 19: GUI Testing. Csci 565 Fall 2012 H.Reza. Tentative Presentation Schedule. objectives. Event-based systems GUI Testing Concepts Strategies Models Process Model-based testing of GUI The Currency Conversion Program and Event-Based Petri Nets. GUI?. What is GUI?

jackie
Download Presentation

Chapter 19: GUI Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 19: GUI Testing Csci 565 Fall 2012 H.Reza

  2. Tentative Presentation Schedule

  3. objectives • Event-based systems • GUI Testing • Concepts • Strategies • Models • Process • Model-based testing of GUI • The Currency Conversion Program and Event-Based Petri Nets

  4. GUI? • What is GUI? • Universal client alternative to traditional forms/commands based user interfaces that accepts as input user-generated and/or system-generated events from a set of finite events and produces predicable graphical outputs

  5. GUI: Formal definitions • A GUI is a hierarchical, graphical front-end to a software system that accepts as input user-generated and system-generated events from a fixed set of events and produces deterministic graphical output. • A GUI contains graphical objectsw (widgets +containers), each with a set of properties p having discrete values v at run-time. At any time during the execution, the values of the properties of each widget of a GUI define the GUI state: {… (w, p, v), …} • A graphical evente is a state transformer that takes the GUI from current state to next state.

  6. Elements of GUIs • GUIs are organized in hierarchical fashion and Composed of objects: • Menu • provide almost continuous availability of the various features of the systems; • Buttons & keyboard shortcuts • enable the user to navigate and access the various functions of their application. • User interacts and manipulate objects using events (or sequence of events) • Windows allow multiple windows to be displayed at the same time • provide functionality using fields in which text/numeric data can be entered • operate in two mode: • Modal (restrict the user's focus and interactions to one window) • Modeless ( does not restrict the GUI interactions)

  7. GUI-based Testing • One of the practical and popular methods used to show the presence of errors in a computer program is to exercise it using its Graphical User Interface(GUI) GUI Our program “Inputs” Is the output correct?

  8. GUI-based Testing: four main questions • At which level conducting the testing? • Unit • Integration • System • Regression • How to choose inputs? • using the specifications/use cases/requirements • using the code • Using the GUI (functionality, structure) • How to identify the expected output? • test oracles • How good test cases are? • when we can stop the testing activity

  9. GUI-based Testing vs non-Gui Testing • Testing GUI software systems is different from testing non-GUI software • Non-GUI testing: • suites are composed of test cases that invoke methods of the system and catch the return value/s; • GUI-based testing: • suites are composed of test cases that are: • able to recognize/identify the components of a GUI; • able to exercise GUI events (e.g., mouse clicks); • able to provide inputs to the GUI components (e.g., filling text fields); • able to test the functionality underlying a GUI set of components; • able to check the GUI representations to see if they are consistent with the expected ones;

  10. GUI testing difficulties • GUI test automation is difficult • Often GUI test automation is technology-dependent • Observing and tracing GUI states is difficult • UI state explosion problem • A lot of possible states of the GUI • Controlling GUI events is difficult • Explosion of the possible combinations of events to do the same thing • GUI test maintenance is hard and costly • Regression Testing (RT) of GUI? • ……

  11. Which type of GUI-based testing? • System testing • Test the whole system • Acceptance testing • Accept the system • Regression testing • Test the system w.r.t. changes

  12. GUI-based Acceptance Testing • Acceptance Tests • specified by the customer and analyst to test that the overall system is functioning as required • Do developers build the right system? How? • Manual Acceptance testing. • User exercises the system manually using his creativity. • Acceptance testing with “GUI Test Drivers” (at the GUI level). • These tools help the developer do functional/acceptance testing through a user interface such as a native GUI or web interface. • Table-based acceptance testing. • Starting from a user story, the customer enters in a table the expectations of the program’s behaviour. • Black-Box (aka functional) approaches • can be used to define test specification then executed manually, by means of the GUI or by table-based testing.

  13. Approaches for GUI-based testing • Manual based • Based on the domain and application knowledge of the tester • Model-based testing • Based on the execution of user sessions selected from a model of the GUI • Which type of model to use? • Event-based model • State-based model • Domain model • How to build the model ? • Specification-based model • Model recovered from existing software systems • Etc.

  14. Coverage criteria for GUI-based testing • Conventional code-based coverage is not applicable • GUIs are implemented in terms of event-based system, • the abstraction level is different w.r.t. the conventional system code. • So mapping between GUI events (or models) and system code is required • Possible coverage criteria: • Event-coverage: • all events of the GUI need to be executed at least once • State-coverage: • “all states” of the GUI need to be exercised at least once • Functionality-coverage: .. using a functional point of view

  15. Event-based Model • Model the space of GUI event interactions as a graph • Given a GUI: • create a graph model of all the possible sequences that a user can execute • use the model to generate event sequences • covert the generated events into Code (Java, C++) so they can be executed against the system under test

  16. Event-based Model “Event-flow graph” File Edit New Open Save … Copy Paste Replace Undo … Cut Editbox1 Match case Editbox0 Cancel Replace all Find next Replace Top level

  17. Event-based Model Model Type: • Complete event-model • Partial event-model Event types: • Structural events (Edit, Replace) • Termination events (Ok, cancel) • System interaction events (Editbox0, Find next, copy) Coverage criteria • Event coverage • Event coverage according the exercised functionality • ….

  18. e1 e3 e4 e5 e2 State-based Model • Model the space of GUI event interactions as a state model, e.g., by using a finite state machine (FSM): • States are screenshot/representation of the GUI • Transitions are GUI events that change the GUI state • Given a GUI: • create a FSM of the possible sequences that a user can execute, considering the GUI state • use the FSM to generate event sequences

  19. <ok> <fill> <click> <click> <click> <fill + ok> <fill> <ok> <fill + ok> State-based Model • FSM: • State: GUI screenshot • Transition: event that changes the GUI state

  20. legal implications of GUI Design/testing • There are serious legal issues related to both design and testing of GUI • Legal issues • Privacy/security • E.g., access to the privileged/stored data • Safety/ reliability • E.g., GUI for Safety critical systems can affect life-or-death decisions (functionality and ease of use) • Accessibility • Equal access for disabled users

  21. A list of GUI Errors • GUI errors include • Data validation • Incorrect field defaults • Mishandling of server process failures • Mandatory fields, not mandatory • Wrong fields retrieved by queries • Incorrect search criteria • Field order • Multiple database rows returned, single row expected • Currency of data on screens • Window object/DB field correspondence • Correct window modality? • Window system commands not available/don’t work • Control state alignment with state of data in window? • Focus on objects needing it? • Menu options align with state of data or application mode? • Action of menu commands aligns with state of data in window • Synchronization of window object content • State of controls aligns with state of data in window?

  22. GUI Testing: Level of Testing • Testing Strategies include • Functionality • Unit testing (e.g., Buttons) • Integrations Testing (synchronizations) • System Testing (functional behaviors according to requirements or functional testing) • Non-Functionality(Quality) • Usability testing • Security testing • …

  23. A Model-Based Strategy to perform GUI Testing • Model-Based Approach to test GUI requires: • Identify components and Events • Select a hierarchal formal model (e.g., EFSM) to represent GUI elements (components) • Intra vs. inter components • Use model to generate threats or event sequences of various lengths corresponding to visible and observable system outputs • Total number of events sequences increases when the length increase • It might be impractical to test for ALL possible event sequences • Need to be selective?

  24. Example: Currency Conversion Program (CCP) • Unit and Integration testing minimally needed • Main focus is on System Testing • Model? • Event-Driven Petri Nets (EDPNs) • Testing threads (sequences of transitions )

  25. Currency Program • Event driven • Unit “button level” testing • Not much integration testing • System level testing Models may include • UML Behavioral models • StateCharts • Finite State Machines • Petri Nets

  26. Currency converter GUI

  27. Unit testing for the CCP • Starts testing (functional and structural) of user-supplied code for buttons • Compute • Valid and invalid data (e.g., non-numerical, etc) • Clear • Quit

  28. System testing CCP • Heavily usage of System testing • Need event-driven model • Event-Driven PN (EDPN) to specify and to generate scenarios/threads • Tables 19.1 and Table 19.2

  29. Strategy to test EDPN • The process to generate threads (positive/negative scenarios) include • Map the elements of CCP to EDPN • Map input/out events to input /output events of EDPN • Map Data to EDPN data places • Map each Action to the transition in EDPN • Create EDPN for each ASF (tasks) • Compose the EDPNs • Create ASF graph • Generate a subset (or suitable?) of threads

  30. When to stop generating threads? Coverage Criteria? • How many threads are needed? • Coverage criteria? • Reasonable coverage includes • Test ALL port inputs • Test All port outputs • Test All ASF (tasks or requirements) • Difficult to test each ASF • Test Threats (scenarios) • There are too many of them • Need to be selective!

  31. Event Driven PN? Definition • Definition: EDPN is tripartite-directed graph (P, D, S, In, Out) • P: a set of port events • D: a set of data places • S: a set of transitions • In: a set of ordered Paris from (P D) S • Out: a set of ordered pairs from S  (P  D)

  32. Example of EDPN

  33. Marking Represents the snap shot of the simulating EDPN

  34. Table 19.1 Port Input Event • Port: Input Events • P1: Enter US Dollar Amount • P2: Click on Brazil • P3: Click on Canada • P4: Click on European Union • P5: Click on Japan • P6: Click on Compute Button • P7: Click on Clear Button • P8: Click on Quit Button

  35. Table 19.2 Port Output events (1) • Ports: Output Events • P9: Display U.S. dollar amount • P10: Display Brazilian Reals • P11: Display Canadian dollars • P12 :Display European Community Euros • P13: Display Japanese Yen • P14: Display ellipsis • P15: Indicate Brazil • P16: Indicate Canada • P17: Indicate European Community • P18: Indicate Japan

  36. Table 19.2 Port Output events (2) • Output Event • P19: Reset selected country • P20: Reset Brazil • P21: Reset Canada • P22: Reset European Community • P23: Reset Japan • P24: Reset equivalent currency amount • P25: End application • P26: Selected Currency Converted • P27:?

  37. Table 19.3 Atomic System Functions (transitions) • Atomic System Functions (or actions) maps to transition (s) • S1: Store US Dollar amount • S2: Sense Click on Brazil • S3: Sense click on Canada • S4: Sense click on EU • S5: Sense click on Japan • S6: Sense click on Compute button • S7: sense click on the Clear button • S8: Sense click on Quit button

  38. Table 19.4: Data places • D1: US dollar amount entered • D2: Country selected

  39. s1: Store US Dollar amount

  40. s4: Sense Click on E.U.

  41. S6: Sense Click on Compute button

  42. s7: Sense Click on Clear button

  43. EPDN Composition of four ASFs into a System Level Thread

  44. EDPN Composition of four ASFs • The composition is not defined • Place fusions? • Looks like it • Transition fusions?

  45. Graph showing the partial relationship between tasks (ASF) Corresponding to USE CASES

  46. Generate System Level Threads using ASF graph • Consider a set of threads T = {T1, T2, T3, T4} • Scenario 1: • 1) enter US amount, 2) select EU, 3) press compute button, 4) press quit • T1 = {s1, s4, s6, s8} • Scenerio2: • 1) User enters US$, 2) Click on Brazil, 3) Clicks on Compute, 4 Clicks on Clear button) • T2 = {s1, s2, s6, s7} • Scenario 3: • T3 = {s3, s1, s6, s7} • Scenario 4: • T4 = {s5, s1, s7, s8} • In general the set of T covers minimum set of usages for CCP, which means • Every action (or S) • Every port input (every input data and input event) • Every port output (every output data and output event)

  47. Generate System Level Threads:2 • Next-level user behavior • Common usage scenario • Converts amount to all 4 currencies, clears and quits • T5 = [s1, s2, s6, s3, s6, s4, s6, s5, s6, s7, s8]

  48. Examples of Scenarios related to unexpected behaviors (negative scenarios • Unexpected user behavior (negative scenario) • Neg-Sencario1: User changes his/her mind about currency • T6 = [s1, s2, s3, s4, s5, s6, s7, s8] • unexpected behavior (negative scenario) • Neg-Scenario2:User toggles between 2 countries, then quits • T7 = [s1, s2, s3, s2, s3, s2, s3, s8] • …

  49. GUI-based Testing: process • Identify the testing objective by defining a coverage criteria • Generate test cases from GUI structure, specification, model • Generate sequences of GUI events • Complete them with inputs and expected oracles • Define executable test cases • Run them and check the results

  50. GUI-based Regression Testing • GUI-based testing means to execute the GUI of a system exercising its GUI components; • A small changes in the GUI layout could make the existing GUI test cases useless; • Hence, GUI-based test suite need to be maintained and often changed  supporting tools are welcomed • Often, GUIs are realized by means of rapid prototyping or automatic framework. • requires an efficient approach to generate and maintain GUI test suite  supporting tools are welcomed

More Related