Build System¶

Overview

The page goes into details about how the build system is configured.

OMERO mostly uses an Ant-based build with dependency management provided by Ivy. C++ code is built using Cmake and Python uses the traditional distutils/setuptools tools.

Structure of the build¶

This is an (abbreviated) snapshot of the structure of the filesystem for OMERO:

OMERO_SOURCE_PREFIX
|
|-- build.xml .......................... Top-level build file
|
|-- build.py ........................... Python wrapper to handle OS-specific configuration
|
|-- omero.class ........................ Self-contained Ant launcher
|
|--etc ................................. Configuration folder
|   |-- grid ........................... Deployment files folder
|   |-- ivysettings.xml ................ Main Ivy configuration file
|   |-- hibernate.properties
|   |-- build.properties
|   |-- logback.xml
|   |-- omero.properties
|   \-- profiles
|
|-- examples ........................... User examples
|
\components
  |
  |--<component-name> .................. Each component has this same basic structure.
  |    |-- build.xml ................... Build file
  |    |-- ivy.xml ..................... Jar dependencies
  |    |-- test.xml .................... Test dependencies
  |    |-- src ......................... Source code
  |    |-- resources ................... Other files of interest
  |    |-- test ........................ Test source code and test resources
  |    \-- target ...................... Build output (deleted on clean)
  |
  |  NOTABLE COMPONENTS
  |
  |-- model ............................ The model component is special in that it produces
  |                                      a jar specific to your database choice: model-psql.jar
  |                                      The generated `ome.model.*` files contain Hibernate
  |                                      annotations for object-relational mapping.
  |
  |-- blitz ............................ The blitz component also performs code generation
  |                                      producing artifacts for Java, Python, and C++.
  |                                      and other build tools.
  |
  |--tools ............................. Other server-components with special build needs.
  |    |--build.xml .................... Build scripts
  |    |
  |    \--<tool-name>
  |         |--build.xml ............... Build file
  |         \--ivy.xml ................. Jar dependencies
  |
  \--antlib ............................ Special component which is not built, but referenced by the build
      |
      \--resources ..................... Build resources
          |--global.xml ................ Global build properties
          |--hibernate.xml
          |--lifecycle.xml ............. Ivy-related targets
          \--version.xml ............... Version properties

Note

User examples are explained under Working with OMERO

Unfortunately, just the above snapshot of the code repository omits some of the most important code. Many megabytes of source code is generated both by our own DSLTask as well as by the Ice slice2java, slice2cpp, and slice2py code generators. These take an intermediate representation of the OME-Model and generate our OME-Remote Objects. This code is not available in git, but once built, can be found in all the directories named “generated”.

Build tools¶

Ant¶

./build.py is a complete replacement for your local ant install. In many cases, you will be fine running ant. If you have any issues (for example OutOfMemory) , please use ./build.py instead. However, only use one or the other; do not mix calls between the two.

The main build targets are defined in the top-level build.xml file. All available targets can be listed using:

./build.py -p

Each of the component contains a build.xml and can be built directly using:

./build.py -f components/server/build.xml

This will call the default dist target for each component.

Ivy¶

The build system uses Ivy 2.3.0 as the dependency manager. The general Ivy configuration is defined in a settings file located under etc/ivysettings.xml.

In order to determine the transitive closure of all dependencies, Ivy resolves each ivy.xml file and stores the resolved artifacts in a cache to speed up other processes. The OMERO build system defines and uses two kinds of caches:

the local dependencies cache under lib/cache is used by most resolvers
Maven resolvers use the Maven cache under ~/.m2/repository

Note

When the Ivy configuration file or the version number is changed, the cache can become stale. Calling ./build.py clean from the top-level build will delete the content of the local cache.

Resolvers are key to how Ivy functions. Multiple dependency resolvers can be defined fine-grained enough to resolve an individual jar in order to pick up the latest version of any library from a repository, a generic URL or from the local file system. Since OMERO 5.1.3, the remote repository resolvers are set up to resolve transitive dependencies.

The OMERO build system uses by default a chain resolver called omero-resolver which resolves the following locations in order:

target/repository which contains most artifacts published by the build system in the install step of the lifecycle
the local dependency repository under lib/repository
the local Maven cache under ~/.m2/repository
the Maven central repository
the OME artifactory

Bio-Formats dependencies are resolved using a specific chain resolver called ome-resolver which resolves the following locations in order:

the local Maven cache under ~/.m2/repository
the OME artifactory

To define its dependencies, each component uses a top-level Ivy file, ivy.xml, for the build and optionally another Ivy file, test.xml, for the tests.

The OMERO build system defines and uses four types of Ivy configurations:

build: defines dependencies to be used for building
server: defines dependencies to be bundled under lib/server
client: defines dependencies to be bundled under lib/client
test: defines dependencies to be used for running the tests

While building, most Java components follow the same lifecycle define in lifecycle.xml. The default dist target for each component calls each of the following steps in order:

retrieve: retrieve the resolved dependencies and copy them under target/libs
prepare: prepare various resources (property files, lib/logback-build.xml)
generate: copy all resources from the previous step for compilation
compile: compile the source files into the destination repository
package-extra: package the sources and the Javadoc into Jar files for publication
package: package the compiled classes into a Jar file for publication
install: convert the component Ivy file into a pom file using makepom and publish the component artifacts

Individual components can override the content of this default lifecycle via their build.xml.

OmeroTools¶

The Ant build alone is not enough to describe all the products which get built. Namely, the builds for the non-Java components stored under components/tools are a bit more complex. Each tools component installs its artifacts to the tools/target directory which is copied on top of the dist top-level distribution directory.

Jenkins¶

The OME project currently uses Jenkins as a continuous integration server available here, so many binary packages can be downloaded without compiling them yourself. See the Continuous Integration documentation for further details.

Server build¶

The default ant target (build-default) will build the OMERO system and copy the necessary components for a binary distribution to the dist directory. Below is a comparison of what is taken from the build, where it is put, and what role it plays in the distribution.

OMERO_SOURCE_PREFIX	OMERO_SOURCE_PREFIX/dist	Comments
components/blitz/target/blitz.jar	`lib/server`	Primary Ice servants
components/blitz/target/server.jar	`lib/server`	Primary server logic
components/tools/OmeroCpp/lib*	`lib/`	Native shared libraries
components/tools/OmeroPy/build/lib	`lib/python`	Python libraries
lib/repository/<some>	`lib/client` & `lib/server`	Libraries needed for the build
etc/	`etc/`	Configuration
`sql/*.sql`	`sql/`	SQL scripts to prepare the database

Note

By default, OMERO C++ language bindings are not built. Use build-all for that.

These files are then zipped to OMERO.server-<version>.zip via release-zip

Coupled development¶

Since OMERO 5.1.3, Bio-Formats is decoupled from the OMERO build system which consumes Bio-Formats artifacts from the OME Maven repository via Ivy.

While this decoupling matches most of the development use cases, it is sometimes necessary to work on coupled Bio-Formats and OMERO branches especially during breaking changes of the OME Data Model or the Bio-Formats API.

The general rule for coupled branches is to build each component in their dependency order and use the local Maven repository under ~/.m2/repository to share artifacts.

Building Bio-Formats¶

From the top-level folder of the Bio-Formats repository,

if necessary, adjust the version of Bio-Formats which will be built, installed locally and consumed by OMERO e.g. for 5.2.0-SNAPSHOT:
```
$ ./tools/bump_maven_version.py 5.2.0-SNAPSHOT
```
run the Maven command allowing to build and install the artifacts under the local Maven cache:
```
$ mvn clean install
```

Building OMERO¶

From the top-level folder of the OMERO repository,

adjust the versions.bioformats property under etc/omero.properties to the version chosen for the Bio-Formats build
run the build system as usual:
```
$ ./build.py build-dev
```

Build System¶

Structure of the build¶

Build tools¶

Ant¶

Ivy¶

OmeroTools¶

Jenkins¶

Server build¶

Coupled development¶

Building Bio-Formats¶

Building OMERO¶

Page Contents

OMERO

Previous topic

Next topic

This Page