1 / 19

Note 4. Organization Structure and Test Documents

Note 4. Organization Structure and Test Documents. Yan Shi Lecture Notes for SE 3730 / CS 5730. Outline. QA organizational structure Test Documents Test plan Test specifications Context driven test planning Good requirements to test.

seamus
Download Presentation

Note 4. Organization Structure and Test Documents

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Note 4. Organization Structure and Test Documents Yan Shi Lecture Notes for SE 3730 / CS 5730

  2. Outline • QA organizational structure • Test Documents • Test plan • Test specifications • Context driven test planning • Good requirements to test

  3. http://bobcesca.thedailybanter.com/files/2013/08/fox_henhouse_12.jpghttp://bobcesca.thedailybanter.com/files/2013/08/fox_henhouse_12.jpg • “The guardians of the hen house can’t be in the employment of the fox.”

  4. Independent QA Organization http://ibagroupit.com/about/structure/

  5. Independent QA Organization • QA organizations are generally separate from organizations such as Development, Marketing, and Sales and report to different VPs. • This maintains independence and allows specialization of skills. • This independence supports: • Decision making that primarily reflect a company’s software quality goals rather than development, financial, marketing, … goals.

  6. QA Roles in the Lifecycle (1) • Process cops: • Is the process followed? PDCA? • Requirements and/or use cases: • current, validated and formally documented? • Test Cases: • generated and reviewed after requirements are confirmed! • Configuration Management: • QA owns the code once it’s baselined! • Build Control: • clean and controlled process and environment. • documented and repeatable?

  7. QA Roles in the Lifecycle (2) • During the testing process: • Unit testing (white-box): done by developers, audited by QA • TDD: assist customer to produce automated acceptance tests. • Simulators/Test Harness: build and maintain. • STE(special test equipment). • Integration testing • System testing • Stress and soak testing • Acceptance testing: written by the customer.

  8. Test Document Types • Test plan • Test goals, overall plans, schedules, special equipment and resources • But not the test cases! • Completed during INCEPTION or early ELABORATION phase • Test specification • Detailed test cases • Created during ELABORATION phase after requirement is detailed. • Completed during early CONSTRUCTION phase. • Evolve during CONSTRUCTION and early TRANSITION phases. • Test plan and specification are often merged into one test document.

  9. Test Document Styles • Inclusive test documents: • chapters for everything from unit tests to acceptance tests. • Individual test documents: • Each software phase has its own test plan and specification • Perhaps an overview document to link all phases together • Unit test documents: • Sometimes it is a part of a Detailed Design document.

  10. Test Document Template • Introduction: • Purpose • Test approach • Definition of test environment • Definition of the build environment and version control • Test cases • Test setup and preconditions • Test inputs and steps • Expected result • Pass/Fail • Estimated and actual test time • Teardown procedure: cleanup after testing so that the running of this test does not contaminate subsequent tests • Traceability to requirements • Traceability to software • IEEE Std 829-2008: Standard for Software Test Documentation

  11. Context Driven Test Planning How to Evolve a Context Driven Development Plan Kaner, Bach and Pettichord, “Lessons Learned in Software Testing”

  12. Test Plan for IPhone Calling

  13. Good Requirements to Test

  14. Examples of Requirements • A1: The system shall have a wonderful users’ interface. • A2: The user interface shall require four or fewer mouse clicks to initiate any transaction and follow Company XYZ’s user interface standards. • A3: It would be nice to have white lettering on a blue background. • B1: The system shall deliver rapid responses to user queries. • B2: The system shall deliver 98% of all users queries within 3 seconds while sustaining transaction rates of 10,000 transactions per second on the target hardware. • B3: The system shall maintain a maximum transaction rate of 9,000 transactions per second. • C1: The control mechanism shall use E.W.M.A. (with a lambda of 0.3), and L.C.L and U.C.L. of 99.98 and 101.23, respectively.

  15. Characteristics of Good Requirements • Valid – meets user expectations and needs • Testable – the system can test and have very little uncertainty whether the requirement was satisfied or not satisfied. (A1 or A2) and (B1 or B2) • Non-conflicting – one requirement should not conflict with another requirement. (B2 or B3) • Complete and non-ambiguous – where interpretation is necessary, there will be different interpretations. • “I saw the man in the park with a telescope.” • Technical level–User technical terms, algorithms, business logic need to be clearly defined (generally in a glossary). C1 • Actually needed – not just wishes • Well organized – cross reference related requirement.

  16. SMART Requirement • SMART Requirements • Specific • Measurable • Attainable (Achievable, Actionable, Appropriate) • Realistic • Time-bound (Timely, Traceable)

  17. What Should Requirements Cover? • Function • what must the system do and how must the user utilize it to perform the function. • Logistics • what must the user do to be able to use the system and maintain it (backups, wires to other systems, upgrade procedures, conversions, etc.) • Environment and interfaces • are there existing systems that this system needs to play well with. • Performance • how fast must the system perform. • Quality of Service (QoS) • how well must the system work and last (fault tolerance, redundancy, recovery, …)

  18. Forms of Requirement Spec. • Raw individual statements • Requirement Specifications: • Hierarchical structure starting from the top: system  subsystems  major components • Use Cases: • Specify a “sequence of transitions performed by a system, which yields an observable result or value for a particular actor” • More natural; easier to understand by all parties • Easier to validate, to write test cases for

  19. Summary • Independent QA organization • QA roles in the software life cycle • Test Documents: • Test plan • Test specification • Other styles • IEEE 829-1998 standard • Context driven test planning • Good requirements to test • Valid, testable, non-conflicting, non-ambiguous, low technical level, necessary, cross-reference

More Related