Testing Web Applications

1. Testing Web Applications Willem VisserRW334

2. Overview • Testing Theory • Testing Web Apps • Tools

3. The Basics • A test consists of • Input • Expected output (also called the Oracle) • White Box Testing • Considering the code when coming up with tests • Black Box Testing • Only consider the input-output interface • Automated Testing is the Holy Grail

4. Types of Testing • Unit Testing • Integration Testing • Systems Testing • Regression Testing • Acceptance Testing • Alpha/Beta Testing • Usability Testing • Performance Testing • Robustness Testing • Security Testing

5. Lower Level Code Testing • Unit Testing • Test individual components of the code • Can be as little as one function • White box • Integration Testing • Interface between components • White Box • Can be as much as the whole system (which case it becomes more black box) • Code Coverage as Test Adequacy Measure

6. Functional Testing • Testing that the systems meets the requirements • Systems Testing • In the deployed environment • Unlike Integration testing • Always black box

7. Cross Cutting • Regression Testing • When code changes (some) old tests must still pass • Could be at unit, integration or systems level • Very high cost involved in regression testing • Regression failure has high cost to fix

8. Customer Facing • Acceptance Testing • Is the customer happy with the product? • Alpha/Beta Testing • Let the system out to a few of your customers and see how they feel about it • Usability Testing • Typically for GUI/Web/Mobile to not just check that the system is correct but also easy to use • Harder to create an Oracle

9. Non-Functional Testing • Performance • Load • See how the system behaves at peak load • Stress • Push the system beyond its limits to see how far it will survive • Security Testing • Check whether there are vulnerabilities that might lead to loss of privacy or other security issues

10. Web App Testing • Full Gambit of Features • Front End GUI • Usability issues • Hard to test • Browser Compatibility issues • Server Side (including storage) • Performance issues • Security issues

11. Unit Testing • Utterly important! • Finding bugs early saves money • Makes regression testing much more effective • Write once, run often • What do you test? • Business logic! • No need to test simple code or interactions with 3rd party libraries • Until they fail of course! • Use Coverage tools to help you decide if you have tested enough

12. Stubs or Mocks • In unit testing you are interested in local behavior and assume other things you might be using behave correctly • Most unit testing frameworks provides stubs for these 3rd party components • Good example is datastore and memcache stubs provided by GAE • Only problem is that sometimes these stubs don’t respect the behavior of the real thing!

13. Unit Testing Frameworks • Java – JUnit • The most famous of them all • PHP – PHPUnit • Python – PyUnit • Actually just “import unittest” • And many more XUnit tools for language X • For example GAEUnit, but it looks “dead” now • Use coverage tools along side, examples… • Coverage.py for Python • (Ecl)Emma for (Eclipse) Java

14. Unit Testing and GUIs • When doing Unit Testing you try and stay well clear of the GUI • Clicking and entering text through a GUI is not automated, although we will see later that with Replay Technology it can also be done • Isolating the GUI to allow more efficient testing is the reason to use Model-View-Presenter rather than Model-View-Controller design pattern

15. Webapp2 example import webapp2 class HelloHandler(webapp2.RequestHandler): def get(self): self.response.write('Hello, world!') app = webapp2.WSGIapplication([('/', HelloHandler)]) Code to Test import unittest import webapp2 # from the app main.py import main class TestHandlers(unittest.TestCase): def test_hello(self): #input request = webapp2.Request.blank('/’) #output response = request.get_response(main.app) #oracle self.assertEqual(response.status_int, 200) self.assertEqual(response.body, 'Hello, world!’) Test

16. Integration Testing • Thin line between unit and integration • Strictly speaking when more than one component is used you are doing integration testing • For example if your web app uses a datastore then the test on the previous slide that came in via a GET request could be an integration test not a unit test • This is not worth worrying about • Unit/Integration Testing is fine

17. System Testing • Now the full round-trip is being tested, including the Browser component • Unit/Integration Testing can be done on a local environment, but System Testing need to be in the deployed environment • State-of-the-Practice • Record and replay tests

18. Seleniumhttp://docs.seleniumhq.org/ • The #1 tool in Web System Testing • “automates browsers. That’s it!” • Much like the webapp2 example earlier, just many more options, including browser specific drivers • IDE for record and replay • With scripts that can be edited • WebDriver that allows one to run without a browser at all

19. Selenium Example fromseleniumimport webdriver fromselenium.common.exceptionsimport TimeoutException fromselenium.webdriver.support.uiimport WebDriverWait # available since 2.4.0 fromselenium.webdriver.supportimport expected_conditions as EC # available since 2.26.0 # Create a new instance of the Firefox driver driver = webdriver.Firefox() # go to the google home page driver.get("http://www.google.com") # find the element that's name attribute is q (the google search box) inputElement = driver.find_element_by_name("q") # type in the search inputElement.send_keys("cheese!") inputElement.submit() try: # we have to wait for the page to refresh, the last thing that seems to be updated is the title WebDriverWait(driver, 10).until(EC.title_contains("cheese!")) # You should see "cheese! - Google Search" print driver.title finally: driver.quit()

20. Java Example importcom.thoughtworks.selenium.*; // This is the driver's import. You'll use this for instantiating a // browser and making it do what you need. publicclassNewTestextends SeleneseTestCase { publicvoidsetUp() throws Exception { setUp("http://www.google.com/", "*firefox"); } publicvoidtestNew() throws Exception { selenium.open("/"); selenium.type("q", "selenium rc"); selenium.click("btnG"); selenium.waitForPageToLoad("30000"); assertTrue(selenium.isTextPresent("Results * for selenium rc")); } }

21. Usability Testing • Based on user opinions • Manual according to a list of tasks • Observer records behavior • Could use things like eye-tracking for more precise results • Automated Usability Testing is still a research topic

22. Robustness Testing • Try anything and see if something breaks • Most famous of these is Android Monkey • Sends random keystrokes to your Android app • gremlins.js does the same thing for web apps <script src="path/to/gremlins.min.js"></script> <script> gremlins.createHorde().unleash(); </script>

23. Performance Testing • System performance under specific loads • Concurrent users doing a certain number of transactions for a certain duration • This is a very hard kind of testing to do • Needs lots of infrastructure • Often the performance bottleneck is your testing framework and not the system under test • In the end we know where the problem is • THE DATABASE! • More generally where something has to wait for something else to finish

24. Web Performance Testing • Measure throughput or transaction rate • Server response time • Rendering, but that might need additional scripts on the client side • You might not know what is the expected performance, so often you profile the performance • You will quickly notice bad performance

25. Web front-end Performance Tools • Google Pagespeed tools • www.webpagetest.org • Google Chrome Developer Tools • Nice summary of tools related to performance can be found at http://samsaffron.com/archive/2012/03/23/sam-s-ultimate-web-performance-tools-and-resources

26. JMeter • Load and Performance Testing • Server side • Not a browser, but can simulate some actions typically done by a browser • HTTP, etc. • No javascript execution

27. Security Testing • This is HUGE! • A whole course or even degree can be devoted to it • Nice checklist of things to do at https://www.owasp.org/index.php/Web_Application_Security_Testing_Cheat_Sheet • Large part of security testing is manual • Penetration Testing • Find security vulnerabilities that can be exploited • Tools to find buffer overflows is one of the most effective automatic approaches • Fuzz Testing • Semi-structured random testing

28. Zed Attack Proxy (ZAP) • Automated Penetration Testing • Point it to URL and it does the rest • Includes Fuzzing and many more • Find it here https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project