1 / 38

Pegasus Technical Workshop - Testing -

7/16/2012. Pegasus Technical Workshop, April 2003. 2. Discussion. Unit testsSBLIM Test Suite Trace componentsData Display DebuggerValgrindOther type debuggingInteroperability. 7/16/2012. Pegasus Technical Workshop, April 2003. 3. Unit tests. . 7/16/2012. Pegasus Technical Workshop, April 2003

lucita
Download Presentation

Pegasus Technical Workshop - Testing -

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. 7/17/2012 Pegasus Technical Workshop, April 2003 1 Pegasus Technical Workshop - Testing - 30 April 2003 Open Group Members Meeting

    2. 7/17/2012 Pegasus Technical Workshop, April 2003 2 Discussion Unit tests SBLIM Test Suite Trace components Data Display Debugger Valgrind Other type debugging Interoperability

    3. 7/17/2012 Pegasus Technical Workshop, April 2003 3 Unit tests

    4. 7/17/2012 Pegasus Technical Workshop, April 2003 4 Unit Tests Unit testing components individually Eg. Common Classes Test elements without CIMOM running Unit testing functionality. eg. Providers (Requires CIMOM operating) Still difficult on all platforms because of startup Developer Guidelines Provide Unit tests for everything possible. Integrate the Unit tests into the Make tests environment

    5. 7/17/2012 Pegasus Technical Workshop, April 2003 5 Unit tests Each of the C++ modules is individually tested through small test programs. TestDataTime TestConfig TestString TestQualifier … etc .. Run the ‘Test*’ programs with the environment variable PEGASUS_TEST_VERBOSE=yo set. Tests boundary conditions, illegal conditions, etc

    6. 7/17/2012 Pegasus Technical Workshop, April 2003 6 End-End Testing TestClient Minimal fixed test TestMakefile or use the BuildMakefile (which does the checkout and runs TestMakefile) make –f TestMakefile <x> prestarttests (functional tests) poststarttests (no security, no SSL) tests (both above) standardtests Our framework is Make. Test design guidelines. Use Assert in the tests. Philosophy – problems stop the test.

    7. 7/17/2012 Pegasus Technical Workshop, April 2003 7 Wbemexec usage Wbemexec based test suite (line test): wbemexec –d 2 test.xml wbemexec -h (examples .xml are in test/wetest/)

    8. 7/17/2012 Pegasus Technical Workshop, April 2003 8 Wbemexec example

    9. 7/17/2012 Pegasus Technical Workshop, April 2003 9 What we have not done well in tests Overall test framework Command line test tool Graphics test/browser tool Ocular organ needed Stress tests.

    10. 7/17/2012 Pegasus Technical Workshop, April 2003 10 SBLIM Test Suite

    11. 7/17/2012 Pegasus Technical Workshop, April 2003 11 SBLIM Test Suite Specification http://www-124.ibm.com/sblim/ A test suite for any CIMOM Role matching engine Performs three major tests: Interface test Consistency test Specification test NEW! April this year

    12. 7/17/2012 Pegasus Technical Workshop, April 2003 12 Interface test Verify that all required provider interfaces are properly implemented, by executing each operation of the Instance and Association Interfaces and interpret the returned results. Checks the implementation for inappropriate return codes. Further types (Property, Method and Indication (or Event) Interfaces) are known.

    13. 7/17/2012 Pegasus Technical Workshop, April 2003 13 Consistency Test Class and property level. Number of proper instances. Correctness of returned values. Observance of threshold values. Checking returned values against system values.

    14. 7/17/2012 Pegasus Technical Workshop, April 2003 14 Specification test The specification test makes use of the meta-information about the model. Within this test type, it is possible to figure out, if the provider implements the class definition in the required manner. For example, the definition of a class marks a certain property as required. But the provider does not return a value for this property. This is a violation of the CIM specification.

    15. 7/17/2012 Pegasus Technical Workshop, April 2003 15 Trace Components

    16. 7/17/2012 Pegasus Technical Workshop, April 2003 16 Trace Components Tracing is done per component (not per file). Different levels per trace: 1 - Function Entry/Exit 2 - Basic flow trace messages, low data detail 3 - Inter-function logic flow, medium data detail 4 - High data detail

    17. 7/17/2012 Pegasus Technical Workshop, April 2003 17 src/Pegasus/Common/TraceComponents.h List of components: Channel XmlWriter CimData ProvManager Authorization Authentication WQL Thread .. etc .. ALL

    18. 7/17/2012 Pegasus Technical Workshop, April 2003 18 Use it? Change the trace component: bin/cimconfig –s traceComponents=Thread,ProvManager Logs the data in cimserver.trc file Change the trace level: bin/cimconfig –s traceLevel=4 See also mak/Buildmakefile for typical trace configurations.

    19. 7/17/2012 Pegasus Technical Workshop, April 2003 19 Limitations High volume. Traces all functions. Developer oriented, not a production tool

    20. 7/17/2012 Pegasus Technical Workshop, April 2003 20 Logging Flow and error logging being installed now. Oriented towards production. How do we use and control ??? Chuck

    21. 7/17/2012 Pegasus Technical Workshop, April 2003 21 Display Data Debugger

    22. 7/17/2012 Pegasus Technical Workshop, April 2003 22 Display Data Debugger http://www.gnu.org/software/ddd/ GNU DDD is a graphical front-end for command-line debuggers such as GDB, DBX, WDB, Ladebug, JDB, XDB, the Perl debugger, or the Python debugger. Besides ``usual'' front-end features such as viewing source texts, DDD has become famous through its interactive graphical data display, where data structures are displayed as graphs.

    23. 7/17/2012 Pegasus Technical Workshop, April 2003 23 How to debug Pegasus Providers? Edit the src/Pegasus/Config/DefaultPropertyTable.h and change “daemon”, “true” to “daemon”, “false” Compile your Pegasus and your provider with the environment PEGASUS_DEBUG=1 present. make clean; make; make repository Install DDD and GDB Run ddd

    24. 7/17/2012 Pegasus Technical Workshop, April 2003 24 Data Display Debugger … File -> Open Program. Select cimserver ddd shows the cimserver.cpp source code in the main window Set a break at the first execution line in the main function in the cimserver.cpp Type “run” in the command window (a window under the source code). When it stops, select File -> Open Source. Select ProviderModule.cpp Set the breakpoint in the function CIMBaseProvider *ProviderModule::load(const String & providerName)

    25. 7/17/2012 Pegasus Technical Workshop, April 2003 25

    26. 7/17/2012 Pegasus Technical Workshop, April 2003 26 DDD Type “continue” in the command window Use your favorite CIM Browser, access the CIM Provider you want to debug. In the DDD it will halt. Move a mouse pointer on a “_fileName” variable and check the library name. File-> Open Source. Select your source code (OrangeProvider.cpp and OrangeProviderMain.cpp). Set a breakpoint in where you want. Type “continue” in the command window.

    27. 7/17/2012 Pegasus Technical Workshop, April 2003 27 valgrind Programmers! Make your software Valgrind-clean. Test it with Valgrind and fix all problems Valgrind reports. This will give you some assurance that your code is free of a broad class of memory management errors. You may well find undiscovered bugs, and your code will probably be more stable as a result. It's good for your code, good for you and especially it's good for the people who use your code. Don't delay -- Valgrind today.

    28. 7/17/2012 Pegasus Technical Workshop, April 2003 28 The Gate of death http://developer.kde.org/~sewardj/ Valgrind is a GPL'd tool to help you find memory-management problems in your programs. When a program is run under Valgrind's supervision, all reads and writes of memory are checked, and calls to malloc/new/free/delete are intercepted.

    29. 7/17/2012 Pegasus Technical Workshop, April 2003 29 From FAQ: Valgrind can detect problems such as: Use of uninitialised memory Reading/writing memory after it has been free'd Reading/writing off the end of malloc'd blocks Reading/writing inappropriate areas on the stack Memory leaks -- where pointers to malloc'd blocks are lost forever Passing of uninitialised and/or unaddressible memory to system calls Mismatched use of malloc/new/new [] vs free/delete/delete [] Some misuses of the POSIX pthreads API

    30. 7/17/2012 Pegasus Technical Workshop, April 2003 30 Commands valgrind -? valgrind –v /usr/bin/cimserver

    31. 7/17/2012 Pegasus Technical Workshop, April 2003 31 Notes We have some problems with Pegasus threading. Local fixes are probably available if you are interested.

    32. 7/17/2012 Pegasus Technical Workshop, April 2003 32 Other type debugging

    33. 7/17/2012 Pegasus Technical Workshop, April 2003 33 Vanilla type debugging Old way of doing it: #define DEBUG(x) cerr << “Provider: “ << X << endl A list of GNU tools for debugging http://www.gnu.org/directory/devel/debug/ And more (commercial)

    34. 7/17/2012 Pegasus Technical Workshop, April 2003 34 Interoperability Testing Server Operation visualization Use trace or logger Pegasus Client Operation visualization (CIMCLient.cpp) Considering logging calls (2.2) Today, compliable display to console of message flow

    35. 7/17/2012 Pegasus Technical Workshop, April 2003 35 Issues Visual Studio and our Make Debugging Providers for other platforms

    36. 7/17/2012 Pegasus Technical Workshop, April 2003 36 Interoperability

    37. 7/17/2012 Pegasus Technical Workshop, April 2003 37 Issues Data types adhere to Common Language Specification CIM/HTTP issues between different clients and servers. SNIA CIMbrowser works. WBEMcli works.

    38. 7/17/2012 Pegasus Technical Workshop, April 2003 38 Interoperability testing Largely through SNIA and informal tests today No Interoperability test suite exists SNIA has demonstrated Largely SNIA, Pegasus, WBEMServices are interoperable at HTTP, XML level Minor exceptions for boundary conditions Some problems remain in operation definition E.g. what does include qualifiers mean?

More Related