Zero Install

the antidote to app-stores

0test

A Zero Install feed can specify a command to run to test the program automatically using the self-test attribute (or the "test" command). 0test runs this command. It can also test combinations of various versions of a program and its dependencies.

Name 0test
Maintainer Thomas Leonard
License GNU General Public License
Run it Zero Install feed
SCM GIT repository

Create a short-cut to it in the usual way:

$ 0alias 0test http://0install.net/2008/interfaces/0test.xml

Usage

Most simply, you can use it to run the tests for a program you have downloaded or checked out of version control. e.g.

$ git clone some-program
$ 0test some-program/program.xml

You can also use it to test released programs, e.g. to test the default version of 0compile:

$ 0test http://0install.net/2006/interfaces/0compile.xml
Passed
 - 0compile v0.21-post, ZeroInstall-Injector v0.51.1, 0publish v0.18-post, ROX-Lib2 v2.0.6-post

0test prints out the result and the versions of all components used in the test.

You can specify a particular set of versions to test:

$ 0test http://0install.net/2006/interfaces/0compile.xml 0.26 0.27
Passed
 - ZeroInstall-Injector v1.7 0compile v0.26, ROX-Lib2 v2.0.6, python v2.7.3, 0publish v0.20
 - ZeroInstall-Injector v1.7 0compile v0.27, ROX-Lib2 v2.0.6, python v2.7.3, 0publish v0.20
None skipped
None failed

A summary is printed at the end showing the result of each combination:

Passed
the tests all passed successfully (the command returned an exit status of zero)
Skipped
this combination of versions can't be used together (or no tests are defined)
Failed
some tests failed (the self-test command returned a non-zero exit status)

You can specify versions of libraries the program depends on too. 0test will try all combinations. This tests three versions of 0compile against three versions of Zero Install (a total of 3 x 3 = 9 tests):

$ 0test --html results.html \
  http://0install.net/2006/interfaces/0compile.xml 0.19 0.20 0.21 \
  http://0install.net/2007/interfaces/ZeroInstall.xml 0.47 0.48 0.49
Passed
 - 0compile v0.20, ZeroInstall-Injector v0.48, 0publish v0.18-post, ROX-Lib2 v2.0.6-post
 - 0compile v0.20, ZeroInstall-Injector v0.49, 0publish v0.18-post, ROX-Lib2 v2.0.6-post
 - 0compile v0.21, ZeroInstall-Injector v0.48, 0publish v0.18-post, ROX-Lib2 v2.0.6-post
 - 0compile v0.21, ZeroInstall-Injector v0.49, 0publish v0.18-post, ROX-Lib2 v2.0.6-post
Skipped
 - 0compile v0.21, ZeroInstall-Injector v0.47
Failed
 - 0compile v0.19, ZeroInstall-Injector v0.47, 0publish v0.18-post, ROX-Lib2 v2.0.6-post
 - 0compile v0.19, ZeroInstall-Injector v0.48, 0publish v0.18-post, ROX-Lib2 v2.0.6-post
 - 0compile v0.19, ZeroInstall-Injector v0.49, 0publish v0.18-post, ROX-Lib2 v2.0.6-post
 - 0compile v0.20, ZeroInstall-Injector v0.47, 0publish v0.18-post, ROX-Lib2 v2.0.6-post

The --html results.html option causes it to also generate this table:

ZeroInstall-Injector
0compile 0.47 0.48 0.49
0.19 0publish 0.18-post ROX-Lib2 2.0.6-post 0publish 0.18-post ROX-Lib2 2.0.6-post 0publish 0.18-post ROX-Lib2 2.0.6-post
0.20 0publish 0.18-post ROX-Lib2 2.0.6-post 0publish 0.18-post ROX-Lib2 2.0.6-post 0publish 0.18-post ROX-Lib2 2.0.6-post
0.21 skipped 0publish 0.18-post ROX-Lib2 2.0.6-post 0publish 0.18-post ROX-Lib2 2.0.6-post

Green = Passed, Red = Failed, Yellow = Skipped. The contents of the cells show the versions of components used in the tests where you didn't specify the version (so 0test selected one for you).

We can see in this example that the unit-tests for 0compile 0.19 no longer pass, regardless of the version of Zero Install used. This problem was caused by a change to the GNU-Hello test program it uses within its own tests. Ideally, programs should depend on their test data using <requires> (see below), but 0compile actually needed to test downloading of the test code itself. 0compile 0.20 fails with older version of Zero Install due to a bug in the way "0launch --get-selections" generated the XML. 0compile 0.21 depends on a later version explicitly, so the broken combination is skipped.

If you specify a test matrix with more than two dimensions, 0test will generate a series of tables.

Other ways to specify versions

As well as passing simple version numbers, you can also allow 0test to choose a suitable version given some constraints. This is useful in test scripts:

Ranges
You can use "," to give a range of possible versions. This is useful with native packages where you don't know the exact version. For example, to test against any Python version 2.6.x (2.6 <= version < 2.7):
$ 0test prog.xml http://repo.roscidus.com/python/python 2.6,2.7
Passed
 - prog 0.1, python v2.6.8-0.2
%nonlocal
Don't select a local (i.e. unreleased) version to test against:
$ 0test prog.xml
Passed
 - prog 0.1, lib v1.11-post

$ 0test prog.xml http://example.com/lib %nonlocal
Passed
 - prog 0.1, lib v1.11

Passing test arguments

You can pass extra arguments to the test command, but you must put them after '--' to stop 0test interpreting them as arguments or versions. e.g. to run 0test on itself with verbose output:

  $ 0test alias:0test -- -v

Specifying a test command

The --test-command option can be used to run the shell command of your choice, rather than the test command given in the feed's self-test attribute. The command that would be executed if the implemention were run normally is available as $*.

For example, to check that "rox --version" worked in versions 2.8 and 2.9 of ROX-Filer:

$ 0test -t '$* --version > /dev/null' \
    http://rox.sourceforge.net/2005/interfaces/ROX-Filer 2.8 2.9
Passed
 - ROX-Filer v2.8
 - ROX-Filer v2.9
None skipped
None failed

You can use --command to select a different <command>. By default, the command used is "test" when run normally, or "run" when using -t. You can also set this to the empty string to select no command (useful with libraries). In that case, $1 is the directory itself rather than the command.

Test-only dependencies

Starting from 0test 0.4, you can also use the <command> syntax instead of the self-test attribute. This allows passing extra arguments and specifying test-specific dependencies. For example:

  <group>
    <requires interface="http://example.com/somelib.xml">
      ...
    </requires>
    <command name="test" path="tests/testall.py">
      <requires interface="http://testing.com/testframework.xml">
	<environment insert="" mode="replace" name="TEST_FRAMEWORK"/>
      </requires>
    </command>
    <implementation id="." version="0.1-pre"/>
  </group>