1 / 73

System testing chapter 14

System testing chapter 14. Csci 565 Spring 2007. objectives. System testing All about threads at system level How big a thread should be? How to identify them? The role of threads in Structural/functional testing SATM Test Threads. System level testing. System testing

kaelem
Download Presentation

System testing chapter 14

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. System testingchapter 14 Csci 565 Spring 2007

  2. objectives • System testing • All about threads at system level • How big a thread should be? • How to identify them? • The role of threads in Structural/functional testing • SATM Test Threads

  3. System level testing • System testing • Can be considered a final step in integration testing • A check of consistency between the software system and its specification • Encompasses system wide properties against a system specification • System, acceptance, and regression testing are all concerned with the behavior of a entire software system

  4. System, acceptance, and regression testing

  5. System test design • It is considered a good practice to design system test cases as early possible • Why? • Exposes critical behaviors • Avoids unpleasant surprises • Testing global properties may require extensive simulation of the execution environment

  6. Unit, integration, and system testing

  7. Threads: some interpretations • Primitives concepts with different meaning at different levels • Think of them as Use-cases/scenarios • Threads and level of abstractions • System level behavior corresponds to a sequence of Atomic System Functions (ASFs) • Integration level behavior corresponds to a sequence of MM-path • Unit level behavior corresponds to a DD-path

  8. Threads: some generic examples • Example of threads may include • A scenario of normal usage • A stimulus/response pair • A sequence of source instructions • A sequence of atomic system functions (ASFs) • A sequences of a state machine description of the system • …

  9. Threads at different levels: some specific examples (SATM) • Candidate threads for SATM • Not Ok (too concrete) • Digit Entry (concrete) • an ASF works with only one stimuli/response • PIN Entry (abstract) • ASFs work with a set of stimuli/response • max integration, min system level • A transaction (very abstract) • A system level thread because of end-to-end completion • <Card entry, menu selection, perform operation, report the results> • An ATM session or multiple transactions (very very abstract) • A sequence of threads • OK for system level • Good for interactions among threads

  10. System testing and requirement specification • System testing w.r.t. requirement specifications • Any system can be specified using • Data (ERD) • Actions (DFD) • Ports (source/destination of input-level system) • Events (occurrence of data or actions on the ports) • Threads (scenarios or paths in control models)

  11. Data-centric models • When the focus is on the persistent data, data-centric models must be used (e.g., ERD) • developed using CRUD (create, retrieve, Update, Delete) • In SATM, Data • PAN (different accounts) • PIN (personal identification number)

  12. Ports • Ports (devices) • The source and destination of system level inputs and outputs • the points at which an I/O device are connected to the system • Examples in SATM • The digits and cancel keys • Functions keys (buttons) • Display screen • Slots

  13. Events • refer to system level input (or output) • the translation point between physical (external) and logical (internal) • Event can be • Discrete (e.g., keystroke) • Continuous (e.g., pressure, temperature)

  14. actions • Process, or transformation point • DFD • work with • Inputs/outputs for both control and/or data

  15. Relationships among basis concepts

  16. Modeling relationship among basis constructs (fig. 14.2)

  17. Examples of Models • Models can be • Structural models • Used for development • Express the functional decomposition and decomposition • Contextual models • Emphasize system ports (or external environment) • Behavioral models • Emphasize the control aspects of the system • Identification of models? • Right models for right problems

  18. Model selection: Event partitioning view of function F

  19. Model selection: FSM of Function F

  20. Finding Threads from behavioral models • FSM models of the SATM is the appropriate staring point for the system testing threads • The model can be interpreted as follows • States (processing steps or the abstraction of the history) • Transitions (logical events)

  21. Top level SATM in FSM

  22. Top level SATM in FSM

  23. Low level SATM in FSM (PIN Entry state)

  24. Low level SATM in FSM (PIN Entry state)

  25. Low level SATM in FSM (PIN Entry state)

  26. Table 1: Events in the PIN entry FSM

  27. Very low level SATM ( PIN TRY) in SFM

  28. Very low level SATM ( PIN TRY) in SFM

  29. Very low level SATM ( PIN TRY) in SFM

  30. Very low level SATM ( PIN TRY) in SFM

  31. Very low level SATM ( PIN TRY) in SFM

  32. Very low level SATM ( PIN TRY) in SFM

  33. Table 2: Port Events in the PIN try FSM

  34. Table 3: An example of a path (or thread) with a correct PIN (1234) Logical event

  35. Table 4: An example of a thread (1235, 123C, 1234) Logical events

  36. Structural Strategies (or path testing) for thread testing • Generating thread test cases is easy • Too many threads (explosion problem) • Strategies • Bottom-up threads • 6 paths can be tested in the PIN Try FSM to test (see table 5) • Correct recognition of digits • Response to the cancellation • Matching expected and entered PINs

  37. Table 5: 6 thread in the PIN Try FSM correct incorrect cancellation

  38. Very low level SATM ( PIN TRY) in SFM

  39. Bottom-up (next upper level) • Once the lower level is tested, move to the upper level (the PIN Entry FSM) • Four paths tested to achieve these objectives • Correct recognition of PIN • Incorrect PIN • Response to the cancel key • Matching expected and actual PIN entered • `See table 6’

  40. Table 6: Thread Paths in the PIN Entry FSM

  41. Low level SATM in FSM (PIN Entry state)

  42. Low level SATM in FSM (PIN Entry state)

  43. Low level SATM in FSM (PIN Entry state)

  44. Low level SATM in FSM (PIN Entry state)

  45. Coverage Metrics for FSM • FSM are directed graphs • use a graph-based coverage metrics • Hierarchical relationship means • Super node (upper level node) • Sub-node (lower level) • Super nodes treat sub-nodes as a procedure with enter/exist semantics

  46. States and transitions Coverage matrices • Two Coverage (metrics) can be applied • State coverage • States correspond to nodes • Transition coverage • Transitions correspond to edges • Path Coverage? • ‘See table 14.7’

  47. Table 7: State/Transition coverage of the 3-try thread

  48. Table 8: Thread/State incidence

  49. Table 9: Thread/Transition Incidence

  50. Problems with path testing (FSM) • The main assumption revolves around path traversing to uncover faults • Can be counter-productive • E.g., CCC in table 6 causes a three-try to fail • Some path cannot be executed by themselves because of dependency between lower levels and upper levels • E.g., 1235 in table 5 causes a transition to Second PIN try • What happens if the user does not press any key?

More Related