1 / 18

Generic API Test tool

Generic API Test tool. By Moshe Sapir Almog Masika. Agenda. Testing practice in the industry The Automatic API test tool concept Attributes supplied to the developer Test coverage Design and implementation Tool demonstration. Testing practice in the industry.

honora
Download Presentation

Generic API Test tool

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Generic API Test tool By Moshe Sapir Almog Masika

  2. Agenda • Testing practice in the industry • The Automatic API test tool concept • Attributes supplied to the developer • Test coverage • Design and implementation • Tool demonstration

  3. Testing practice in the industry • Testing software module requires deep understanding of the module’s inner logic and its interfaces’ semantics • Currently, dedicated tests are written to each software component • Usually the tests do not try to retrieve functional information from the module’s code • Changes in the software logic are resulted in changes in the test automation logic

  4. Automatic API Test Tool Concept • Enable the developer to add metadata to the modules code • Use .NET attributes to represent the metadata • The metadata should contain information which will enable automatic testing of the module • Create automatic test tool that given the module and attributes can generate tests

  5. Attributes supplied to the developer – Function level attributes • Input-Output Assertion: • Assertion is defined by three expressions <P, Q, L> • If P is true before the function is run, then Q is true after the function is run, or one of the exceptions in L is thrown during execution • For double Sqrt(double x) the developer will define: • P: x > 0 • Q: ret ^ 2 = x • L = {DivisionByZero }

  6. Attributes supplied to the developer – Function level attributes (cont) • Input Parameters attribute • Provide set of interesting assignments to the methods parameters • For double tan(double x) the developer will define the following assignments: • X = pi • X = 0 • X = pi/2 • X = -1234.45*pi

  7. Attributes supplied to the developer – Class level attributes • Class Instances • Provide set of interesting instances of the class • For class Int32 the developer will define the following instances: • 0 • 1 • -1 • 2^31 • (-2^31) + 1 • 12345 • -780

  8. Attributes supplied to the developer – Class level attributes (cont) • Class Invariants • Define set of Boolean expressions that should be true after and before executing any member function on an instance of the class • For Account class: class Account { string Owner; int Balance; } The developer will define: • Owner != string.Empty • Balance >= 0

  9. Test Coverage – Method level coverage • General test • Run the method as many times as possible, using parameters supplied by the developer. If no parameters were supplied, try to generate the them • Input – Output Assertion test • For each <P, Q, L> defined on the method, try to find an assignment that satisfies P, run the method and validate Q and L.

  10. A B C D M1 M2 M3 Test Coverage – Assembly level coverage • Function call composition • Try to build a call graph of the functions in the assembly and generate the call paths. Validate input-output assertions and class invariants during the calls: • For the classes: A, B, C, D And the functions: • M1: A x B -> A • M2: B x C -> D • M3: A x D -> A Build the following graph:

  11. Design and implementation – Functional decomposition • Test Attributes- Attributes that are available to the assembly developer. • User Interface- Retrieve from the user the input required to run the test and to present him the test results. • Test Engine- Responsible to run the tests and produce the test results. • Results Reporting: Responsible to save the results in persistent file and redirect them to the graphic user interface.

  12. Design and implementation – Functional decomposition • Data flow between the different components:

  13. Test Engine Design: The Algorithm design pattern

  14. Test Attributes Hierarchy

  15. Design and implementation – User Interface Structure

  16. Test execution sequence diagram

  17. Demonstration . . .

  18. Conclusions • .NET attributes can be used to keep information that will enable automatic testing of software • The technique we have shown is limited to platforms that allow introduction of Meta data in the binary code (.NET, Java) • The Syntax of defining expressive test attributes can be error prone (mainly due to the use of strings as function identifiers) • The ability of such tools to verify large scale software is still to be investigated. Currently is seems can be used as first tire testing (BVT).

More Related