Scidac sss quarterly report
Download
1 / 21

SciDAC SSS Quarterly Report - PowerPoint PPT Presentation


  • 369 Views
  • Uploaded on

SciDAC SSS Quarterly Report. Sandia Labs August 27, 2004 William McLendon. Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SciDAC SSS Quarterly Report' - flora


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Scidac sss quarterly report l.jpg

SciDAC SSS Quarterly Report

Sandia Labs

August 27, 2004

William McLendon

Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.


Apitest overview l.jpg
APItest - Overview

  • Current Release : v0.2.0

  • Test Driver tool for testing software

    • Ordered tests

    • API Testing

  • Available for download:

    • ftp://ftp.sandia.gov/outgoing/apitest/


Apitest features l.jpg

Scripted Test Driver

XML based tests

Object Oriented

Easy to define new test types.

Conditional test ordering

Batch Scripts

Can call other batches

Built In Test Types

Shell

Script

SSS

Can set environment variables for tests.

SUID Test Capability

CAUTION!

Timeouts

User Interface Types

Web Browser

Command Line

Test Results Export

Save Test results to an XML file.

Open Source

LGPL

Fully scripted in Python

Portable

APItest - Features


Apitest requirements l.jpg
APItest - Requirements

  • Packages required to run APItest:

    • Python 2.3 or greater

    • ElementTree (www.effbot.org)

    • Twisted (www.twistedmatrix.com)

    • ssslib (ftp://ftp.mcs.anl.gov/pub/sss/)

      • optional, for SSSlib tests.


Apitest execution l.jpg
APItest - Execution

  • There are two ways to execute APItest

  • Command Line Only

    • $> apitest -f test_file.[apt|apb]

  • Web browser GUI (Interactive)

    • $> apitest httpd

      • -i [--iroot] : root directory to look for tests

      • -p [--port] : port number (default = 2112)

      • -h [--host] : host (default = localhost)

  • --help will display command line usage.


Apitest gui top page l.jpg
APItest - GUI (Top Page)

  • Top level index page for APItest

  • APItest recurses from the [testroot] to find tests.

  • File Extensions:

    • TEST : .apt

    • BATCH : .apb

  • Batch tests shown in bold

  • Organized by directory


Apitest gui test run execution l.jpg
APItest - GUI (Test Run Execution)

  • Quick status view of test execution.

  • This screen loads when the EXECUTE button is selected.

  • Color coded for easy viewing.

  • Currently running test is indicated.

  • Refreshes every 15 secs.


Apitest gui test run summary l.jpg
APItest - GUI (Test Run Summary)

  • Summary of test results

  • Color coding

  • Batch Features

    • Cumulative summaries

      • Total # of tests

      • # that passed (green)

      • # that failed (red)

  • Can click the hyperlinks to see more details about a test or batch.


Apitest gui batch summary l.jpg
APItest - GUI (Batch Summary)

  • Clicking a hyperlink into a batch test gives us a summary view.

  • In this example, one test failed a dependency

    • FAILDEP in the status column indicates this.

  • Clicking on the scriptTest2.apt link will show us more data about this test.


Apitest gui test details faildep l.jpg
APItest - GUI (Test Details - FAILDEP)

  • Detailed view of scriptTest2.apt from previous slide.

  • Failed a dependency on cmdTest2.apt

    • Expected : FAIL

    • Actual : PASS


Apitest gui list of test runs l.jpg
APItest - GUI (List of Test Runs)

  • Clicking the View All Runs button gives this screen.

  • Test Runs are sorted by date and time.

  • Click links to view details of a particular run.


Apitest gui test details passed l.jpg
APItest - GUI (Test Details - PASSED)

  • Example of a test that was executed and passed.

  • Summary Data

  • Dependency Data

    • Only shown if test was run as part of a batch.

  • Output

    • Side-by-side comparison

    • Dependent on type of test.


Sss service directory startup test l.jpg

PREP

TEST SDOFF

SSS START

TEST SDON

RM EMNG

SDSTAT

SSS STOP

TEST SDOFF

SSS - Service Directory Startup Test


Apitest ongoing work l.jpg
APItest - Ongoing Work

  • Develop more tests for SSS components!

  • Validation via file matching.

  • Configuration file.

  • Test Developer GUI.

  • Results browser.

  • Session management and security (Cookies?)

  • Additional native test types.

    • HTTP, TCP/IP, XMLRPC

  • DB Connectivity.

  • User Guide.

  • Bug fixing.


Apitest test types l.jpg
APItest - Test Types

  • CMD

    • Execute a command via the command line.

  • SCRIPT

    • Execute a script.

  • SSS

    • Execute a ssslib communication.


Apitest example cmd test l.jpg
APItest - Example CMD Test

<testDef>

<info>

Example test. Runs ls -lr from /tmp, expects 0 exit status and any input.

</info>

<test type="cmd” wdir="/tmp">

<command interpreter="/bin/ksh">ls</command>

<input name="argument" format="literal">-l</input>

<input name="argument" format="literal">-r</input>

<inputname="stdin" format="literal" />

<output name="stdout" format="regexp">.*</output>

<outputname="stderr" format="literal" />

<output name="status" format="literal">0</output>

</test>

</testDef>


Apitest example script test l.jpg
APItest - Example Script Test

<testDef>

<info>Timeout test</info>

<test name="timestamp1" type="script" timeout="4">

<command interpreter="/bin/bash" wdir="/tmp">

for x in 1 2 3 4 5 6 7 8 9 10

do

echo "timeout_test :${x}: `date`”

sleep 1

done

</command>

<output name="stdout" format="regexp">.*timeout.*</output>

<outputname="stderr" format="literal" />

<output name="status" format="literal">0</output>

</test>

</testDef>


Apitest example envvar test l.jpg
APItest - Example ENVVAR Test

<testDef>

<info>

This script executes but adds some stuff to the environment.

NOTE: setting envvar OVERRIDES any inherited environment variables.

</info>

<test type="script">

<command interpreter="/bin/csh">#!/bin/csh env</command>

<input name="envvar" format="literal" key="test_envvar_1">foobar</input>

<input name="envvar" format="literal" key="test_envvar_2">foo</input>

<output name="stdout" format="regexp">

.*test_envvar_1=foobar\n.*test_envvar_2=foo\n.*</output>

<outputname="stderr" format="literal" />

<output name="status" format="literal">0</output>

</test>

</testDef>


Apitest example suid test l.jpg
APItest - Example SUID Test

<testDef>

<test type="script">

<command interpreter="python" uname="wcmclen" gid="500">

#!/usr/bin/env python

import os

print os.getuid()

print os.getgid()

</command>

<input name="argument">/tmp/testuser/</input>

<input name="argument">-l</input>

<output name="stdout" format="regexp">^27030$\n^500$</output>

<outputname="stderr" format="literal" />

<output name="status" format="literal">0</output>

</test>

</testDef>


Apitest example sss test l.jpg
APItest - Example SSS Test

<testDef>

<info>Sample sss query of the service directory</info>

<test type="sss" destination="service-directory">

<input name="sendbuf">

&lt;get-location>&lt;location component='service-directory' host='*' port='*'

protocol='*' schema_version='*' tier='*'/>&lt;/get-location></input>

<output name="recvbuf" format="regexp">.*</output>

</test>

</testDef>