Running and writing tests

The following guidelines apply to tests in both the Java and Python test components. However, some of the presented options apply to only one or the other.

Running unit tests

The unit testing framework is fairly simple. Only methods which contain logic written within OMERO are tested. This means that framework functionality like remoting or the Hibernate layer is not tested. This is a part of integration testing (see below).

You can run the unit tests for any component from its directory by entering:

./build.py -f components/<component>/build.xml test

The same can be done for all components using:

./build.py test-unit

Note that for tests written in Python the package pytest must be installed, see Writing Python tests.

Running integration tests

Integration testing is a bit more complex because of the reliance on a database, which is not easily mockable. All Hibernate-related classes are tested in integration mode.

The tests require a fast computer. Running all the integration tests places several restrictions on the environment:

• There must be a running OMERO database.
• An OMERO.server instance must be running.

Integration tests assume that:

• ICE_CONFIG has been properly set. The contents of the etc/ice.config file should be enough to configure a running server for integration testing. This means that code creating a client connection as outlined in developers/GettingStarted/AdvancedClientDevelopment.html should execute without errors.
• An OMERO.server instance is running on the host and port specified in the ICE_CONFIG file.

If any of the tests fail with a user authentication exception (or omero.client throws an exception), a new ice.config file can be created and pointed to by the ICE_CONFIG environment variable. Most likely the first settings that will have to be put there will be omero.user and omero.pass.

Running all tests

To run all the integration tests, use

./build.py test-integration

Component tests

Running an integration test suite for an individual component can be done explicitly via:

./build.py -f components/<component>/build.xml integration

Results are placed in components/<component>/target/reports.

Individual test groups

To run individual OmeroJava test groups (or comma-separated sets of groups) of tests, the -DGROUPS parameter can be used together with the test target

./build.py -f components/tools/OmeroJava/build.xml test -DGROUPS=integration

Individual tests

Alternatively, you can run individual tests which you may currently be working on. This can be done by using the test option. For example:

./build.py -f components/tools/OmeroJava/build.xml test -DTEST=integration/AdminTest
./build.py -f components/tools/OmeroPy/build.xml test -DTEST=test/integration/test_admin.py

Individual test class methods

Individual OmeroJava test class methods (or a comma-separated list of methods) can be run using the -DMETHODS parameter together with the test target. The test method must be provided in the fully qualified name form (package.class.method).

./build.py -f components/tools/OmeroJava/build.xml test -DMETHODS=integration.chgrp.AnnotationMoveTest.testMoveTaggedImage

Using markers in OmeroPy tests

Tests under OmeroPy can be included or excluded according to markers defined in the tests. This can be done by using the -DMARK option. For example:

./build.py -f components/tools/OmeroPy/build.xml integration -DMARK=long_running

See Writing Python tests for more information on this.

Failing tests

The test.with.fail ant property is set to false by default, which prevents test failures from failing the build. However, it can instead be set to true to allow test failures to fail the build. For example:

./build.py -Dtest.with.fail=true integration

Some components might provide individual targets for specific tests (e.g. OmeroJava provides the broken target for running broken tests). The build.xml file is the reference in each component.

Writing Java tests

For more information on writing tests in general see http://testng.org. For a test to be an “integration” test, place it in the “integration” TestNG group. If a test is temporarily broken, add it to the “broken” group:

@Test(groups = {"integration", "broken"}
public void testMyStuff() {

}

Tests should be of the Acceptance Test form. The ticket number for which a test is being written should be added in the TestNG annotation:

@Test(groups = "ticket:60")

This works at either the method level (see SetsAndLinksTest.java) or the class level (see UniqueResultTest.java).

OmeroJava

The tests under components/tools/OmeroJava/test will be the starting point for most Java-client developers coming to OMERO. An example skeleton for an integration test looks similar to

@Test(groups = "integration")
public class MyTest {

  omero.client client;

  @BeforeClass
  protected void setup() throws Exception {
    client = new omero.client();
    client.createSession();
  }

  @AfterClass
  protected void tearDown() throws Exception {
    client.closeSession();
  }

  @Test
  public void testSimple() throws Exception {
    client.getSession().getAdminService().getEventContext();
  }

}

Using Eclipse to run tests

To facilitate importing OMERO components into Eclipse, there are .project and .classpath-template files stored in each component directory (e.g. common’s .classpath and common’s .project).

There are also top-level .classpath and .project files which allow for importing all components as a single project, but this approach requires more memory and does not clearly differentiate the classpaths, and so can lead to confusion.

Before importing any component as a project into Eclipse, a successful build has to have taken place:

./build.py

This is for two reasons. Firstly, the Eclipse projects are not configured to perform the code generation needed. The build.py command creates the directory:

<component>/target

which will be missing from any Eclipse project you open before building the source.

Secondly, Ivy is used to copy all the jar dependencies from OMERO_SOURCE_PREFIX/lib/repository to <component>/target/libs, which is then used in the Eclipse .classpath files.

If Eclipse ever gets out of sync after the first build, ./build.py build-eclipse can be used to quickly synchronize.

TestNG plugin

A prerequisite of running unit and integration tests in the Eclipse UI is having the TestNG plug-in installed and working (help available on the TestNG site).

Unit tests

Running the unit tests under Eclipse requires no extra settings and is as easy as navigating to the package or class context menu Run As or Debug As, then selecting TestNG.

Integration tests

Integration tests require the ICE_CONFIG environment variable to be available for the Eclipse-controlled JVM. This can be done by editing Debug/Run configurations in Eclipse. After navigating to the Debug (or Run) Configurations window, the Environment tab needs to be selected. After clicking New, ICE_CONFIG can be defined as a path to the ice.config file. This setting needs to be defined per package, class or method.

Debugging a running OMERO instance

By using the “debug” target from templates.xml, it is possible to have OMERO listen on port 8787 for a debugging connection.

bin/omero admin stop
bin/omero admin start debug

Then in Eclipse, you can create a new “Debug” configuration by clicking on Remote Java Application, and setting the port to 8787. These values are arbitrary and can be changed locally.

Keep in mind:

• The server will not start up until you have connected with Eclipse. This is due to the “suspend=y” clause in templates.xml. If you would like the server to start without you connecting, use “suspend=n”.
• If you take too much time examining your threads, your calls may throw timeout exceptions.

Writing Python tests

To write and run Python tests you first need to install pytest:

pip install pytest

For more information on writing tests in general see http://pytest.org.

OmeroPy

Similar to the OmeroJava tests, the tests under components/tools/OmeroPy/test will be the starting point for most Python-client developers coming to OMERO. Integration tests should be placed under components/tools/OmeroPy/test/integration. The file names must begin with test_ for the tests to be found by pytest.

import omero

class TestExample(object)

  def setup_method(self, method):
    client = new omero.client()
    client.createSession()

  def teardown_method(self, method):
    client.closeSession()

  def testSimple():
    ec = client.getSession().getAdminService().getEventContext()
    assert ec, "No EventContext!"

Running tests directly

When writing tests it can be more convenient, flexible and powerful to run the tests from components/tools/OmeroPy using setup.py. Since Python is interpreted, tests can be written and then run without having to rebuild or restart the server. A few basic examples are shown below.

Run a single test file:

cd components/tools/OmeroPy
./setup.py test -s test/integration/test_admin.py

The -k option will run all integration tests containing the given string in their names. For example, to run all the tests under test/integration with permissions in their names:

./setup.py test -s test/integration -k permissions

This option can also be used to run a named test within a test module:

./setup.py test -s test/integration/test_admin.py -k testGetGroup

The -m option will run integration tests depending on the markers they are decorated with. For example, to run all integration tests excluding those decorated with the marker long_running:

./setup.py test -s test/integration -m "not long_running"

For more help with setup.py see:

./setup.py --help test

To make use of the more advanced options available in pytest that are not accessible using setup.py, the script py.test can be used directly. For example, to run the tests in a single package allowing standard output to be shown on the console:

py.test test/integration/test_admin.py -s

For a full list of options see:

py.test --help

and http://pytest.org/latest/usage.html for more help in running tests.