1 / 22

SciDAC SSS Quarterly Report

SciDAC SSS Quarterly Report. Sandia Labs January 25, 2005 William McLendon. Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

sierra
Download Presentation

SciDAC SSS Quarterly Report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SciDAC SSS Quarterly Report Sandia Labs January 25, 2005 William McLendon Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

  2. APItest - Overview • Current Release : v0.2.5 • Test Driver tool for testing software • Ordered tests • API Testing • Available for download: • ftp://ftp.sandia.gov/outgoing/apitest/

  3. Work Since Previous Meeting • October : Development meeting at ORNL • Imported APItest into OSCAR • Started developing some APItests • Fixing bugs • Finishing up User Guide

  4. Scripted Test Driver XML based tests Object Oriented Easy to define new test types. Conditional test ordering Batch Scripts Can call other batches Built In Test Types Shell Script SSS Fully scripted in Python Portable Can set environment variables for tests. SUID Test Capability CAUTION! Timeouts User Interface Types Web Browser Command Line Archived Results Browsing Test Results Export Save Test results to an XML file. Open Source LGPL APItest - Features

  5. APItest - Requirements • Packages required to run APItest: • Python 2.3 or greater • lower versions can work but install is not as easy. • ElementTree (www.effbot.org) • Twisted (www.twistedmatrix.com) • SSSlib (ftp://ftp.mcs.anl.gov/pub/sss/) • optional, for SSSlib tests.

  6. APItest - Execution • There are two ways to execute APItest • Command Line Only • $> apitest -f test_file.[apt|apb] • Web browser GUI (Interactive) • $> apitest httpd • -i [--iroot] : root directory to look for tests • -p [--port] : port number (default = 2112) • -h [--host] : host (default = localhost) • Running with --help will display command line usage.

  7. APItest - GUI (Top Page) • Top level index page for APItest • APItest recurses from the [testroot] to find tests. • File Extensions: • TEST : .apt • BATCH : .apb • Batch tests shown in bold • Organized by directory

  8. APItest - GUI (Test Run Execution) • Quick status view of test execution. • This screen loads when the EXECUTE button is selected. • Color coded for easy viewing. • Currently running test is indicated. • Refreshes every 15 seconds until tests complete.

  9. APItest - GUI (Test Run Summary) • Summary of test results • Color coding • Batch Features • Cumulative summaries • Total # of tests • # that passed (green) • # that failed (red) • Hyperlinks to see more details about a test or batch.

  10. APItest - GUI (Test Details - PASSED) • Example of a test that was executed and passed. • Summary Data • Dependency Data • Only shown if test was run as part of a batch. • Output • Side-by-side comparison • Dependent on type of test.

  11. APItest - GUI (Batch Summary) • Clicking a hyperlink into a batch test gives us a summary view. • In this example, one test failed a dependency • FAILDEP in the status column indicates this. • Clicking on the scriptTest2.apt link will show us more data about this test.

  12. APItest - GUI (Test Details - FAILDEP) • Detailed view of scriptTest2.apt from previous slide. • Failed a dependency on cmdTest2.apt • Expected : FAIL • Actual : PASS

  13. APItest - GUI (List of Test Runs) • Clicking the View All Runs button gives this screen. • Test Runs are sorted by date and time. • Click links to view details of a particular run.

  14. PREP TEST SDOFF SSS START TEST SDON RM EMNG SDSTAT SSS STOP TEST SDOFF SSS - Service Directory Startup Test

  15. APItest - Ongoing & TODO • User Guide • Validation via file matching • Bug fixing • Configuration file • More SSS tests • Test Developer GUI • Remote test execution • Additional native test types • HTTP, TCP/IP, XMLRPC • Session management and security (Cookies?) • DB Connectivity

  16. APItest - Test Types • CMD • Execute a command via the command line. • SCRIPT • Execute a script. • SSS • Execute a ssslib communication.

  17. APItest - Example CMD Test <testDef> <info> Example test. Runs ls -lr from /tmp, expects 0 exit status and any input. </info> <test type="cmd” wdir="/tmp"> <command interpreter="/bin/ksh">ls</command> <input name="argument" format="literal">-l</input> <input name="argument" format="literal">-r</input> <inputname="stdin" format="literal" /> <output name="stdout" format="regexp">.*</output> <outputname="stderr" format="literal" /> <output name="status" format="literal">0</output> </test> </testDef>

  18. APItest - Example Script Test <testDef> <info>Timeout test</info> <test name="timestamp1" type="script" timeout="4"> <command interpreter="/bin/bash" wdir="/tmp"> for x in 1 2 3 4 5 6 7 8 9 10 do echo "timeout_test :${x}: `date`” sleep 1 done </command> <output name="stdout" format="regexp">.*timeout.*</output> <outputname="stderr" format="literal" /> <output name="status" format="literal">0</output> </test> </testDef>

  19. APItest - Example ENVVAR Test <testDef> <info> This script executes but adds some stuff to the environment. NOTE: setting envvar OVERRIDES any inherited environment variables. </info> <test type="script"> <command interpreter="/bin/csh">#!/bin/csh env</command> <input name="envvar" format="literal" key="test_envvar_1">foobar</input> <input name="envvar" format="literal" key="test_envvar_2">foo</input> <output name="stdout" format="regexp"> .*test_envvar_1=foobar\n.*test_envvar_2=foo\n.*</output> <outputname="stderr" format="literal" /> <output name="status" format="literal">0</output> </test> </testDef>

  20. APItest - Example SUID Test <testDef> <test type="script"> <command interpreter="python" uname="wcmclen" gid="500"> #!/usr/bin/env python import os print os.getuid() print os.getgid() </command> <input name="argument">/tmp/testuser/</input> <input name="argument">-l</input> <output name="stdout" format="regexp">^27030$\n^500$</output> <outputname="stderr" format="literal" /> <output name="status" format="literal">0</output> </test> </testDef>

  21. APItest - Example SSS Test <testDef> <info>Sample sss query of the service directory</info> <test type="sss" destination="service-directory"> <input name="sendbuf"> &lt;get-location>&lt;location component='service-directory' host='*' port='*' protocol='*' schema_version='*' tier='*'/>&lt;/get-location></input> <output name="recvbuf" format="regexp">.*</output> </test> </testDef>

More Related