1 / 36

200107RadView - Omaha SPIN

paul
Download Presentation

200107RadView - Omaha SPIN

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. 1

    3. 3 Topology of a Web Site

    4. 4 General Considerations

    5. 5 The NEED to Test (Are we wondering…) Is the site content meaningful? Is this application easy to use? How about browser compatible? Are we using the right tools? How reliable is our technology? Do the Servers have enough power? How many visitors are we expecting? Are the machines fast enough? How much activity can the site handle?

    6. 6 What and How to test? (Fundamental Considerations) Test Planning Browser Compatibility Design compliance Usability Efficiency Scalability Best suited OS platform Network bandwidth Hardware capacity Site availability Monitoring Security

    7. 7 Testing a Web Site Understanding what types of testing are needed for each application, is essential for several reasons: Ensures you that the testing effort, will support the business and technical requirements. The activities required for each type of test and its associated phases, are included within the master test plan (and schedule). Helps identifying and plan for the environments and resources that are needed to prepare and execute each test type. Assures the achievement of any Testing Goals.

    8. 8 Testing a Web Site (continued) There are mainly two essential types of testing that should be accomplished. Functionality Performance

    9. 9 Functionality Testing What is it? Why is so important? What is: Unit Test Integrated Test Functional Test System Test Regression Test

    10. 10 What is Functionality Testing? Is a set of skilled approaches and procedures which systematically verify that the application’s components and process modules, are available and working as expected. Compliance with Business rules, Technical requirements, System Documentation, and User manuals is of course… Mandatory!

    11. 11 Why is Functionality Testing important?

    12. 12 Functionality Testing -Unit Test Unit Test Unit tests perform basic tests at component level and test a specific business process, application, and/or system configuration. Unit tests ensure that each unique path of a business process performs accurately to the documented specifications and contains clearly defined inputs and expected results Valid Username + Valid Password + Login (Click) = Get IN site Valid Username + Valid Password + Login (Click) = Get IN site

    13. 13 Functionality Testing -Integration Test Integration Test Integration test is testing a specific feature together with other newly developed features Testing the interface of two components explores how components interact with each other Integration testing inspects the variables passed not only between two components, but also the global variables. Integration testing is specifically aimed at exposing the problems that arise from the combination of components Previously captured unit test scripts can be combined to create a variety of integration test cases This test phase assumes the components and the objects they manipulate have all passed their local unit tests A valid login + selecting one of the option buttons will take you to that section of the web site. A valid login + selecting one of the option buttons will take you to that section of the web site.

    14. 14 Functionality Testing - Functional Test Functional Test Functional testing is centered around the following items: Valid Input - identified classes of valid input must be accepted. Invalid Input - identified classes of invalid input must be rejected Functions - identified functions must be exercised Output - identified classes of application outputs must be exercised. Systems/Procedures - interfacing systems or procedures must be invoked.

    15. 15 Functionality Testing – System & Regression Test Regression Test Regression tests are a series of tests to ensure that no adverse changes are introduced to the application during maintenance changes, upgrades, or other modifications. Regression testing is composed of a suite of components that are rerun when any one-application component has been modified This provides confidence that the system will function with newly implemented requirements. Regression testing is often an iterative process during a testing cycle

    16. 16 Performance Testing What is it? Why is so important? What are the types of Performance Tests? What is the difference between Load, Performance, and Stress Testing ?

    17. 17 What is Performance Testing ? Performance Tests determine the runtime “behavior” of the application and its supporting infrastructure, under certain conditions. Performance Testing is used to measure several system characteristics, such as processing speed, response time, resource consumption, throughput, and efficiency.

    18. 18 Why is Performance Testing important? The week of Feb 6, 2000: Hackers delivered over 1-Billion transactions concurrently to each one of these sites. Yahoo eBay Buy.com Amazon eTrade

    19. 19 Performance Testing - Load Test Load Test Load Tests determine the applications behavior under load, up to and including its limits (not just at its limits). Load tests specifically refer to the load size (number of concurrent users) and related values.

    20. 20 Performance Testing - Stress Test Stress Test Stress testing determines the applications ability to handle large amounts of data. Stress Testing can be much more successful with a full load applied to the server. Running addition applications on the server in conjunction with the client side tests is an additional form of stress testing.

    21. 21 Performance, Load, and Stress Testing ? They all have different meanings, although they seem to be used almost interchangeably sometimes. To better understand these differences; let’s apply these terms while testing a car: Performance Testing - a car does 0-60mph in a certain number of seconds under normal conditions Load Testing – The car can pull a trailer weighing x - tons before it can no longer move Stress Testing - The car continues to work fine driving 60 mph for two hours, in 120 degree temperatures, with only 1 gallon of coolant in the radiator

    22. 22 Automated Web Testing

    23. 23 Why Automate Testing? Is it necessary to repeat the sequence of actions many times? Do you need to run the same tests on multiple hardware configurations? Need to test with several concurrent users? Exercise multiple options, components, and configurations Save time and money

    24. 24 What are we testing for? Where should I start? How do we proceed? What tool do I select?

    25. 25 Simulating the Real World The MAIN purpose of running automated tests is to emulate the REAL world as close as possible while saving time and effort.

    26. 26 Tips for selecting an Automated Testing Tool

    27. 27

    28. 28 What are we Testing for? Everyone involved should be clear on what exactly we are testing for and the goals for the outcome of the test Testing For: How many Total Concurrent Users ? At what point does the performance degrade ? Which server will max out first ? (Web, DB, App, etc.) How about Data integrity ? What is the Total Throughput ? (Bandwidth, pipeline) What are the total TPS ? Goals: Round time =< 15 sec Total Concurrent Users => 300 / Min

    29. 29 Automation Web Testing Methodology

    30. 30 The Application Describe the application to test using a application model Describe the Hardware environment Create a Benchmark (Agenda) to be recorded in Phase 2. Define what tasks each user will perform Define (or estimate) the percentage of users per task. Benchmark - A benchmark procedure uses a predefined set of data and measures the results returned by the Application Being Tested (ABT). Benchmarks are intended to make comparisons easy!

    31. 31 Testing Types and Metrics needed Load Increasing Workload Start / End with how many users? Increment by how many users? How often to increment (Seconds)? Steady-State How many users? How long (Days/Hours/Minutes/Seconds)? Performance What are the reasonable response times in seconds? (each task/page) Stress What are the expected limits? How many Peak users ?

    32. 32 Record Record the defined testing activities that will be used as a foundation for your load test scripts. One activity per task or multiple activities depending on user task definition

    33. 33 Performance Test scenario

    34. 34 Performance Test scenario Test Date: 07/17/2001 Virtual Users (VUs): Test Goals Start: 5 Max Cycle Time <= 40 Sec Incremented by: 5 Maximum: 200 Initial Cycle Time = 35 Sec Sleep = 32 Seconds Test Script(s): One typical user from logon through completion (continuously) This would be an average user

    35. 35 Performance Test scenario What Type of Test is This ?What Type of Test is This ?

    36. 36 Performance Test scenario Memory is OK at around 27%, Notice the Connections start to drop offMemory is OK at around 27%, Notice the Connections start to drop off

    37. 37 Performance Test scenario

    38. 38 Performance Test scenario

    39. 39 Conclusions

    40. 40 Selecting the right Tool Our Premiere Testing and Analysis Solutions A testing tool is only as good as it’s reporting capabilities. WebLoad gives you the ability to create any number of reports and graphs for any part of the test. As an example, let’s assume you’re testing the performance and scalability of your application for an upcoming marketing campaign. This campaign will bring an exceptionally large number of visitors to your site at very specific times. What will happen to your registered customers that also happen to be on line at that time? WebLoad is the only product that will enable you to reproduce the test, focusing on the metrics involving your registered users. You can zoom in directly on those metrics and analyze the data to determine if there will be any problems. As you make changes to your site to compensate for any issues you have found, you can use WebLoad to test the changes, monitoring the results in real time to ensure the changes had the desired effect. Some examples of WebLoad’s reporting capabilities include: Automated Timers Timers and Counters for Every HTTP Transaction performed are created automatically, allowing you to analyze download times and hits. User Defined Transactions Ability To Create Custom Statistics. This feature enables you to create a timer for any business process for performance analysis. System Metrics Comparing System with Performance. How are the servers performing? WebLoad can incorporate any data from Windows NT Performance Monitor or UNIX RSTATD directly into the test metrics for comprehensive analysis. Regression Analysis How has the application performed over time? Has the performance improved or degraded? WebLoad allows you to view a current running test vs. yesterday’s test. Perhaps you want to see the progression of performance over time. WebLoad allows you to graph any number of test results for comparison. Data Drilling WebLoad’s unique Data Drilling feature enables you to view any occurrences on an individual virtual client level. This feature is extremely useful in capturing intermittent errors. Table Compare WebLoad’s Table Compare Wizard allows you to compare any HTML table for form and content on an individual virtual client level, automatically. Any differences are automatically recorded and visually displayed during data analysis. Custom Reports Competitive offerings only supply canned reports, forcing you to use what the tool manufacturer thought was important. WebLoad’s flexibility in data analysis allows you to create any number of graphs and reports, focusing on specific areas of interest to you. A testing tool is only as good as it’s reporting capabilities. WebLoad gives you the ability to create any number of reports and graphs for any part of the test. As an example, let’s assume you’re testing the performance and scalability of your application for an upcoming marketing campaign. This campaign will bring an exceptionally large number of visitors to your site at very specific times. What will happen to your registered customers that also happen to be on line at that time? WebLoad is the only product that will enable you to reproduce the test, focusing on the metrics involving your registered users. You can zoom in directly on those metrics and analyze the data to determine if there will be any problems. As you make changes to your site to compensate for any issues you have found, you can use WebLoad to test the changes, monitoring the results in real time to ensure the changes had the desired effect. Some examples of WebLoad’s reporting capabilities include: Automated Timers Timers and Counters for Every HTTP Transaction performed are created automatically, allowing you to analyze download times and hits. User Defined Transactions Ability To Create Custom Statistics. This feature enables you to create a timer for any business process for performance analysis. System Metrics Comparing System with Performance. How are the servers performing? WebLoad can incorporate any data from Windows NT Performance Monitor or UNIX RSTATD directly into the test metrics for comprehensive analysis. Regression Analysis How has the application performed over time? Has the performance improved or degraded? WebLoad allows you to view a current running test vs. yesterday’s test. Perhaps you want to see the progression of performance over time. WebLoad allows you to graph any number of test results for comparison. Data Drilling WebLoad’s unique Data Drilling feature enables you to view any occurrences on an individual virtual client level. This feature is extremely useful in capturing intermittent errors. Table Compare WebLoad’s Table Compare Wizard allows you to compare any HTML table for form and content on an individual virtual client level, automatically. Any differences are automatically recorded and visually displayed during data analysis. Custom Reports Competitive offerings only supply canned reports, forcing you to use what the tool manufacturer thought was important. WebLoad’s flexibility in data analysis allows you to create any number of graphs and reports, focusing on specific areas of interest to you.

    41. 41

    42. 42 Key Features

    43. 43 That’s All

More Related