1 / 27

Non-Functional Testing

Non-Functional Testing. Philippe CHARMAN charman@fr.ibm.com http://users.polytech.unice.fr/~charman/. Last update: 21-05-2014. Non exhaustive list of non-functional testing. Performance Testing Load testing Stress Testing Endurance Testing Usability Testing Accessibility Testing

sylviac
Download Presentation

Non-Functional Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Non-Functional Testing Philippe CHARMAN charman@fr.ibm.com http://users.polytech.unice.fr/~charman/ Last update: 21-05-2014

  2. Non exhaustive list of non-functional testing • Performance Testing • Load testing • Stress Testing • Endurance Testing • Usability Testing • Accessibility Testing • Compatibility Testing • Compliance Testing • Internationalization and localization Testing • Doc & Training Testing • Recovery Testing • Etc.

  3. Load Testing • Load testing is the process of putting demand on a system or device and measuring its response • Some examples: • simulating multiple users accessing a web server • giving a huge document to a word processor • subjecting a mail server to a large amount of e-mail traffic • writing and reading data to and from a hard disk continuously • It helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation • Synonyms: volume testing, scalability testing, etc.

  4. Load Testing

  5. Stress Testing • Testing used to determine the stability of a system • Testing beyond normal operational capacity, often to a breaking point, in order to observe the results • The goals may be to ensure the software does not crash in conditions of insufficient computational resources (such as memory or disk space),

  6. Endurance Testing • Kind of load testing over a longer period of time • During endurance tests, memory utilization is monitored to detect potential leaks • Synonyms: reliability testing, soak testing

  7. Performance Tests

  8. Usability Evaluation Methods • Usability : how well users can learn and use a product to achieve their goals and how satisfied they are with that process • How to evaluate usability: • Ease of learning • Efficiency of use • Memorability • Error frequency and severity • Subjective satisfaction • Some evaluation methods: • Inspection • Testing • Inquiry • Rapid prototyping

  9. Usability Inspection • Usability inspection is aimed at finding usability problems in the design, • An evaluator inspects a user interface • Some inspection methods: • Cognitive walkthrough • Heuristic evaluation

  10. Usability Testing • Technique used to evaluate a product by testing it on real users • How many users to test ? • Eye Tracking

  11. Eye Tracking Demo

  12. Accessibility Testing • Software accessibility is a general term used to describe the degree to which an application is available to as many people as possible • Access issues: • Visual impairments • Motor skills • Hearing impairments • Photosensitive epilepsy

  13. Accessibility Testing • Typical test cases: • Make sure that all functions are available via keyboard only (do not use mouse) • Make sure that information is visible when display setting is changed to High Contrast modes • Make sure that screen reading tools can read all the text available and every picture/Image have corresponding alternate text associated with it • Make sure that product defined keyboard actions do not affect accessibility keyboard shortcuts • Make sure that Web pages do not contain anything that flashes more than two times in any one second period • etc ...

  14. Accessibility Testing • Do a simple test: • Start menu > Control Panel > Accessibility Options > Display > • Check the checkbox "Use High Contrast" • Are your main applications (web browser, email reader, etc.) still usable ? • Another test: • Install the tool Color Oracle from http://colororacle.org/ • Is it still easy to distinguish colors differences with your main applications?

  15. Color impaired vision

  16. Compatibility Testing • Testing conducted on the application to evaluate the application's compatibility with the computing environment that can be composed of: • Hardware platform : x86, x64, UltraSPARC-I, POWER7 etc. • OS: Windows XP 32-bit, Debian 4.0, Solaris 11, etc. • Database: Oracle, Sybase, DB2, etc. • Browser: IE, Firefox, Chrome, • Peripheral: printer, scanner, • Etc. • Compatibility Matrix • Pairwise Testing methods

  17. Compatibility Matrix

  18. Backward & Forward Compatibility Testing • Backward compatibility • Forward compatibility

  19. Compliance Testing • Some exemples: • Blue Wash d’IBM • HTLM5 Conformance Tests • Coding rules in C/C++ from MISRA, JSF, etc. • Etc.

  20. C/C++ Standards • Code should be: • Portable • Reliable • Maintenable • Testable • Reutilisable • Extensible • Readable

  21. C/C++ Standards • Some examples • Avoid long and complex functions. • Rationale: long functions are difficult to understand and to test • Do not use #define to use a constant • Use "const«  instead • Use #include to include .h only and not .cpp files

  22. Compliance Tests • More: • MISRA web site (Motor Industry Software Reliability Association) http://www.misra.org.uk/ • JSF web site (Joint Strike Fighter) http://www.jsf.mil/downloads/down_documentation.htm • Web site of the tool Understand http://www.scitools.com/

  23. Internationalization& Localization Testing • Globalisation = i18n + l10n • I18N & l10n • OS • Keyboard • Display text • Date format • Number format • Etc.

  24. Internationalization& Localization Testing • To be performed once functional testing has been completed and all the functionality related defects are identified and removed • 2 phases: First, to make sure that application’s GUI or functionality will not be broken with the translated text Second to make sure that translation of all the strings have happened properly

  25. Recovery Testing • Testing how well an application is able to recover from crashes, hardware failures and other similar problems. • Recovery testing != reliability testing

  26. References • Srinivasan Desikan & Gopalaswamy Ramesh, Software Testing: Principles and Practice, Pearson Education • J. Nielsen, "Heuristic Evaluation". In Jakob Nielsen and Robert L. Mack, editors, "Usability Inspection Methods". John Wiley and Sons, Inc. 1994. • Mauro, C.L.: Professional usability testing and return on investment as it applies to user interface design for web-based products and services

  27. More information … • http://www.usability.gov/ • http://www.usabilityhome.com/ • http://www.testinggeek.com/ • http://www.misra.org.uk/

More Related