Data Analysis Assignment

1. Quality AssuranceIS 101Y/CMSC 101YNovember 12, 2013Carolyn SeamanUniversity of Maryland Baltimore County

2. Data Analysis Assignment • Stacked bar charts

3. Data Analysis Assignment • Aggregate and focus

4. Quality Assurance • Refers to any activity whose goal is to make sure that the system or application being built is free from error and of high quality • That it works • That it solves a problem • That it’s usable

5. A process view of QA Problem Identification CMSC, CMPE, IS Analysis Design SDLC: Systems Development Lifecycle Implementation QA Installation Maintenance

6. QA includes: • Testing • Exercises a system or part of a system to make sure it produces the correct output or behaves in expected ways • Cannot happen until some part of the system is implemented • Reviews and inspections • Visual “reading” of some artifact • Can be done early by reviewing early documents, e.g. design

7. Testing Terms • Coverage • ideally, testing will exercise the system in all possible ways • not possible, so we use different criteria to judge how well our testing strategy “covers” the system • Test case • consists of data, procedure, and expected result • represents just one situation under which the system (or some part of it) might run

8. Test Planning • A test plan includes: • test objectives • schedule and logistics • test strategies • test cases • procedure • data • expected result • procedures for handling problems

9. Testing Phases • Unit testing- does this piece work by itself? • Integration testing- do these two pieces work together? • System testing- do all the pieces work together? • Alpha acceptance testing- try it out with in-house “users” • Installation testing- can users install it and does it work in their environment? • Beta acceptance testing- try it out with real users In development/ maintenance organization In user organization

10. Testing Techniques • Structural testing techniques • “white box” testing • based on statements in the code or individual hardware and connections • coverage criteria related to physical parts of the system • tests how a program/system does something • Functional testing techniques • “black box” testing • based on input and output • coverage criteria based on behavior aspects • tests the behavior of a system or program

11. System Testing Techniques • Goal is to evaluate the system as a whole, not its parts • Techniques can be structural or functional • Techniques can be used in any stage that tests the system as a whole (acceptance, installation, etc.) • Techniques not mutually exclusive

12. System Testing Techniques (cont.) stress testing- test larger-than-normal capacity in terms of transactions, data, users, speed, etc. execution testing- test performance in terms of speed, precision, etc. recovery testing- test how the system recovers from a disaster, how it handles corrupted data, etc. operations testing- test how the system fits in with existing operations and procedures in the user organization compliance testing- test adherence to standards security testing- test security requirements

13. System Testing Techniques (cont.) requirements testing- fundamental form of testing - makes sure the system does what it’s required to do regression testing- make sure unchanged functionality remains unchanged error-handling testing- test required error-handling functions (usually user error) manual-support testing- test that the system can be used properly - includes user documentation historical test data- tests until the number of defects found approaches the average number of defects in the products produced under similar circumstances

14. Unit Testing • Goal is to evaluate some piece (file, program, module, component, etc.) in isolation • Techniques can be structural or functional • In practice, it’s usually ad-hoc and looks a lot like debugging • More structured approaches exist

15. Unit Testing Techniques input domain testing- pick test cases representative of the range of allowable input, including high, low, and average values equivalence partitioning- partition the range of allowable input so that the program is expected to behave similarly for all inputs in a given partition, then pick a test case from each partition boundary value- choose test cases with input values at the boundary (both inside and outside) of the allowable range

16. Unit Testing Techniques (cont.) statement testing- ensure the set of test cases exercises every statement at least once branch testing- each branch of an if/then statement is exercised path testing- every path is exercised (impossible in practice) fault seeding- put a certain number of known faults into the code, then test until they are all found

17. Reviews and Inspection:Goal and Motivation By detecting defects early, and preventing their leakage downstream, the higher cost of later detection and rework is eliminated.

18. Basic Steps • Using a reading technique, • Perform a visual examination of the document • Detect and correct: • Defects • Violation of design standards • Other problems

19. Examples of artifacts that can be reviewed Include: • Contracts • Installation plans • Progress reports • Design descriptions • Release notes • Requirements specifications • Source code

20. What Is a Defect? • Any occurrence in a work product that is determined to be incomplete, incorrect, or missing • Any instance which a requirement is not satisfied (Fagan, 1986) • Informal synonyms: bug, fault, issue, problem, anomaly

21. Common Attributes of Systematic Reviews and Inspections Systematic reviews and inspections have these attributes in common: • Team participation • Documented results of the review • Documented procedures for conducting the review

22. Management Reviews • Performed by those directly responsible for the system • Monitor progress • Determine status of plans and schedules • Confirm requirements and their system allocation • Or, evaluate management approaches used to achieve fitness or purpose

23. Technical Reviews Confirms that a product • Conforms to specifications • Adheres to regulations, standards, guidelines, plans • Changes are properly implemented • Changes affect only those system areas identified by the change specification

24. Inspections (Formal Peer Reviews) Confirms that the software product satisfies • Specifications • Specified quality attributes • Regulations, standards, guidelines, plans • Identifies deviations from standards and specification • results in logging a defect

25. Walk-throughs • Evaluate a product or artifact or document • Sometimes used for educating an audience • Major objectives: • Find anomalies • Improve the product • Consider alternative implementations • Evaluate performance to standards and specs

26. Audits The purpose of an audit is to provide an independent evaluation of conformance of products and processes to applicable • Regulations • Standards • Guidelines • Plans • Procedures

27. Inspection Process • Most popular is the Fagan method • Inspection is separated into 5/6 phases • (Planning) • Overview • Preparation • Inspection Meeting • Rework • Follow-up

28. Review and Inspection Pitfalls • Insufficient Preparation • Moderator Domination • Incorrect Review Rate • Ego-involvement and Personality Conflict • Issue Resolution and Meeting Digression • Recording Difficulties and Clerical Overhead

29. Other QA Activities • Process conformance • Making sure that quality procedures are followed • Risk analysis • Planning for adverse events • Making the unexpected expected • Hiring and training • Developing guidelines and standards • Identifying new training needs

30. Project demos • This Thursday!!! • No slides • Everyone must participate and answer questions • Show your program • Running • Code • Talk about what it does and what it doesn’t do YET • 10 minutes, including questions • Turn in .pde file and document BEFORE class on Blackboard

31. Peer Evaluations • Take a look at the Peer Evaluation form and think about the following two options: 1. Each student gets the actual sheets filled out by their teammates 2. Each student gets a summary of the feedback written by the instructor • Evaluations due in class on THURSDAY (rewards to those who bring them to class)