TEAM Engine

Overview

TEAM Engine (Test, Evaluation, And Measurement Engine) is an engine for testing web services and other resources written in JAVA. It executes test scripts written in Compliance Test Language (CTL), TestNG and other languages. It is lightweight and easy to run as a command line or to setup as a service. It can be used to test any type of service or encoding. It is also the official tool used by the Open Geospatial Consortium (OGC) for compliance testing.

Installation

  1. Install JRE 1.6.0 - You will need a copy of the Java Runtime Environment (JRE), version 1.6.0 or greater (also called JRE 6.0). If not already on your system, download this from http://java.sun.com and install it.
  2. Unpack the archive - Download the TEAM binary distribution archive from the SourceForge project page and unpack it to the location of your choice.
  3. Set environment variables - The TEAM commands use the JAVA_HOME environment variable to determine the location of the JRE. If it is not already set to the location of your copy of JRE, edit the file setenv.bat (Windows) or setenv.sh (Unix/Linux) to set the JAVA_HOME variable. It should be set to the JRE directory, which should contain a bin directory that contains the java executable. You may also set a JAVA_OPTS variable to pass other arguments to the Java Virtual Machine (JVM), such as proxy parameters.

Test Tool

Synopsis

Use the test tool (test.bat for Windows or test.sh for Unix/Linux) to execute CTL scripts. Executing the tool with no command-line parameters will provide a synopsis as shown below:

Test mode:
  Use to start a test session.

  test [-mode=test] -source={ctlfile|dir} [-source={ctlfile|dir}] ...
    [-suite=[{namespace_uri,|prefix:}]suite_name] [-logdir=dir] [-session=session]

Resume mode:
  Use to resume a test session that was interrupted before completion.

  test -mode=resume -source={ctlfile|dir} [-source={ctlfile|dir}] ...
    -logdir=dir -session=session

Retest mode:
  Use to reexecute individual tests.

  test -mode=retest -source={ctlfile|dir} [-source={ctlfile|dir}] ...
    -logdir=dir test1 [test2] ...

Doc mode:
  Use to generate a list of assertions.

  test -mode=doc -source={ctlfile|dir} [-source={ctlfile|dir}] ...
    [-suite=[{namespace_uri,|prefix:}]suite_name]

In this synopsis, optional items are enclosed in brackets ([]). Braces ({}) are used to indicate options when several values are allowed, and a vertical bar (|) is used to separate the options. Option names and values are case sensitive.

Test Mode

Use test mode to start a new test session. You may specify test mode using parameter -mode=test, or you may omit the -mode parameter since it defaults to test mode.

Supply the source CTL file(s) to execute using a -source parameter. Its value may specify a CTL file, or may specify a directory containing multiple CTL files. If it specifies a directory, each file in the directory with a .ctl or a .xml extension is assumed to be a CTL file. Be aware that for each of these files the test tool will create intermediate stylesheet files in the source directory with either .txsl (test, resume, and retest modes) or .dxsl (doc mode) extensions.

You may supply more than one -source parameter. If there are tests in multiple sources with the same qualified name, the tests in the later source location will take precedence.

In test mode, execution begins with one of the test suite elements in the source CTL. If there is just one suite in the CTL sources, the test tool will start there, but if there are multiple suites, you will need to supply a -suite parameter. The suite name may be specified using the suite element.s unqualified name, its prefix qualified name, or its namespace qualified name. For the prefix qualified form, use the same namespace prefix used in the CTL file, followed by a colon, followed by the base suite name. For the namespace qualified form, use the full namespace URI, followed by a comma, followed by the base suite name.

It is not necessary to log your test session, but doing so will allow you to use the log viewer to see details on any tests that failed, recover if the session is interrupted, or reexecute failed tests. To log the session, supply a -logdir parameter. Its value should be a directory on the local file system. If the directory does not exist, it is created.

Inside the log directory, the system will create a session directory where it will store the log files for the new test session. You may supply a name for this directory using the -session parameter or the system will create one automatically. If the session directory already exists, its contents will be overridden.

Resume Mode

Occasionally, a test session may get interrupted. This could happen if the JVM process dies or a test produces a Java exception that halts the test process. If this happens to a test session that is being logged, you can run the test tool again using parameter -mode=resume to continue the process where it left off.

In resume mode, you must supply the same -source parameter(s) and -logdir parameter that were used when the session was started in test mode. You must also identify the session to be resumed with the -session parameter.

Retest Mode

If for any reason you would like to reexecute any of the tests in a logged test session, run the test tool again using parameter -mode=retest. Supply the same -source parameter(s) and -logdir parameter that were used when the session was started in test mode.

Following the named parameters, supply a list of the test(s) to be reexecuted. The tests should be identified by their logged test path, which starts with the session identifier. This is the value shown in parenthesis following the test name in the test results or test summary log.

Doc Mode

Doc mode does not actually execute any of the CTL code. It produces a list of the assertions in a test suite for documentation purposes.

To run the tool in doc mode, specify parameter -mode=doc. Supply -source parameter(s) and, if necessary, a -suite parameter as in test mode.

Log Viewer

Use the log viewer (viewlog.bat for Windows or viewlog.sh for Unix/Linux) to view test results for logged sessions. Executing the tool with no command-line parameters will provide a synopsis as shown below:

To list sessions in a log directory:
  viewlog -logdir=dir

To list tests in a session:
  viewlog -logdir=dir -session=session

To view detailed results for tests:
  viewlog -logdir=dir test1 [test2] ...

The parameters for the log viewer are similar to the parameters for the test tool. For a list the sessions that exist in a log directory, supply the -logdir parameter. For a hierarchical list the tests in a test session and their results, supply both the -logdir parameter and the - session parameter. For a detailed result log, including HTTP requests and their results and any other logged messages, supply the -logdir parameter and a list of the test(s) to view, identified by their test path.

Developer

TEAM engine was originally developed by Northrop Grumman Corporation, jointly with The National Technology Alliance and is Copyright (C) 2005-2006, Northrop Grumman Corporation. All Rights Reserved. It is currently maintained by the OGC Community

License

The TEAM Engine is open source software provided to you under the terms and conditions of the Mozilla Public License, version 1.1. A copy of the license is included in the distribution archive in the text file license.txt. An HTML version of the license is available online at http://www.mozilla.org/MPL/MPL-1.1.html.

TEAM Engine depends on several other open source software packages, which are included in the distribution archive. These dependant packages and their licenses are listed below.

Saxon 8.6.1   Mozilla Public License, Version 1.0
Xerces J 2.8.0   Apache Software License, Version 2.0

Acknowledgements

Thanks to SourceForge.net Logo for project hosting.