Build System
============
.. topic:: Overview
The page goes into details about how the build system is configured.
Creating binary distribution
----------------------------
The default ant target ("build-default") will build the OMERO system and
copy the necessary components for a binary distribution to the /dist
directory. Below is a comparison of what is taken from the build, where
it is put, and what role it plays in the distribution.
.. note::
By default, |OmeroCpp| is not built. Use ``build-all`` for that.
.. list-table::
:header-rows: 1
* - OMERO_SOURCE_PREFIX
- OMERO_SOURCE_PREFIX/dist
- Comments
* - components/blitz/target/blitz.jar
- lib/server
- Primary Ice servants
* - components/blitz/target/blitz.jar
- lib/server
- Primary Ice servants
* - components/blitz/target/server.jar
- lib/server
- Primary server logic
* - components/tools/OmeroCpp/lib*
- lib/
- Native shared libraries
* - components/tools/OmeroPy/build/lib
- lib/python
- Python libraries
* - lib/repository/<some>
- lib/client & lib/server
- Libraries needed for the build
* - etc/
- etc/
- Configuration
* - sql/\*.sql
- sql/
- SQL scripts to prepare the database
* - <javadoc/>
- docs/api
- (Optional) Javadocs produced with "java omero javadoc"
These files are then zipped to OMERO.server-<version>.zip via "java
omero release-zip"
Jenkins
~~~~~~~
The OME project currently uses Jenkins_ as
a continuous integration server available :jenkins:`here <>`, so many
binary packages can be downloaded without compiling them yourself.
OMERO.server is built by the :omerojob:`"OMERO" job <>`.
Hudson checks for git changes every 15 minutes and executes:
::
(cd docs/hudson; python launcher.py)
which invokes the "build-all", "javadoc" "findbugs", "coverage", and
"release-zip" targets.
The Javadocs are always made available :javadoc:`here <>`
as well as several build metrics.
Build tools
-----------
OMERO mostly uses an `ant <http://ant.apache.org>`_-based build with
dependency management provided by `Ivy <http://ant.apache.org/ivy>`_.
:doc:`Native code </developers/Cpp>` is built using
`SCons <http://scons.org>`__ and Python uses the traditional
distutils/setuptools tools.
Structure of the build
~~~~~~~~~~~~~~~~~~~~~~
This is an (abbreviated) snapshot of the structure of the filesystem for
OMERO:
::
OMERO_SOURCE_PREFIX
|
|-- build.xml .......................... Top-level build driver
|
|-- build.py ........................... Python wrapper to handle OS-specific configuration
|
|-- omero.class ........................ Self-contained Ant launcher
|
|--etc/ ................................ All configuration
| |-- grid/* ......................... Deployment files
| |-- ivysettings.xml
| |-- hibernate.properties
| |-- local.properties.example
| |-- log4j.xml
| |-- omero.properties
| \-- profiles
|
|-- examples ............................ User examples
|
\components
|
|
|--<component-name> ................... Each component has this same basic structure.
| |-- build.xml Main scripts
| |-- ivy.xml Jar dependencies
| |-- test.xml Test dependencies
| |-- src Source code
| |-- resources Other files of interest
| |-- test Test source code and test resources
| \-- target Output of build (deleted on clean)
|
|-- model ............................. The model component is special in that in produces
| a jar specific to your database choice: model-psql.jar
| The generated `ome.model.*` files contain Hibernate
| annotations for object-relational mapping.
|
|-- blitz ............................. The blitz component also performs code generation
| | producing artifacts for Java, Python, and C++.
| |
| \ blitz_tools.py ................. OMERO-specific SCons Environment definition
| and other build tools.
|
|--tools .............................. Other server-components with special build needs.
| |--build.xml
| \--<tool-name>
| |--build.xml
| `--ivy.xml
|
\--antlib ............................. Special component which is not built, but referenced by the build
|
\--resources
|--global.xml
|--directories.xml
|--lifecycle.xml
\--depdendencies.xml
.. note::
User examples are explained under :doc:`/developers/GettingStarted`
Each of the components can also be built directly. For example,
::
./build.py -f components/server/build.xml
Code generation
~~~~~~~~~~~~~~~
Unfortunately, just the above snapshot of the code repository omits some
of the most important code. Many MB of source code is generated both by
our own :sourcedir:`DSLTask <components/dsl>` as well as by
the Ice_ ``slice2java``, ``slice2cpp``, and
``slice2py`` code generators. These take an intermediate representation
of the :model_doc:`OME-Model <ome-xml/>` and generate our |OmeroModel|.
This code is not available in git, but once built, can be found in all the
directories named "generated".
.. _build#OmeroTools:
OmeroTools
~~~~~~~~~~~
Similarly, the ant build alone is not enough to describe all the
products which get built. Namely, the builds for the non-Java components
stored under :sourcedir:`components/tools`
are a bit more complex. Each tools component installs its artifacts to
the tools/target directory which is copied **on top of** the
OMERO\_HOME/dist top-level distribution directory. Current tools
include:
.. |OmeroFs| replace:: :doc:`/developers/Server/FS`
.. |ant| replace:: :term:`Ant-based builds`
.. |ice| replace:: :term:`Ice-based builds`
.. |scons| replace:: :term:`Scons-based builds`
+------------------+-------+-------+---------+
| | |ant| | |ice| | |scons| |
+------------------+-------+-------+---------+
| |OmeroCpp| | | | X |
+------------------+-------+-------+---------+
| |OmeroWeb| | X | | |
+------------------+-------+-------+---------+
| |OmeroFs| | | X | |
+------------------+-------+-------+---------+
| |OmeroPy| | | X | |
+------------------+-------+-------+---------+
| LicenseService | X | | X |
+------------------+-------+-------+---------+
.. glossary::
Ant-based builds
Some of the tools also contain Java code which imports files from ``antlib/resources`` and then proceeds like the other regular components.
Ice-based builds
An Ice-based build requires further invocations of ``slice2*`` code generation. Currently this
Scons-based builds
Builds which have C++ targets are based generally on `Â Scons <http://www.scons.org>`__. See |OmeroCpp| for more information.
Comments on Ivy
---------------
- Resolvers are key to how Ivy functions. Currently, the default
resolver is called "omero-resolver" and simply looks in our
repository (``./lib/repository``) for the jars which were downloaded
from git. Multi-resolvers can be defined (as granular as for an
individual jar) in order to pick up the latest version of whatever
library from HTTP, |SSH|, or from the local file system.
- OMERO\_HOME/lib/cache : in order to determine the transitive closure
of all dependencies, Ivy "resolves" each ivy.xml and stores the
resolved ivy.xml in its cache (in our build, ``./lib/cache``) to
speed up other processes. However, when changing the Ivy
configuration (``./etc/ivyconf.xml``) or version number
(``etc/omero.properties->omero.version``) the cache can become stale.
This should not happen, but currently does. It may be beneficial for
the time being to call ``ant clean`` from the top-level build which
will delete the cache.
Comments on build.py
--------------------
``./build.py`` is a complete replacement for your local ant install. In
many cases on and on most OS, you will be fine running ``ant``. If you
have any issues (for example ``OutOfMemory``) , please use
``./build.py`` instead. **\*However**\ \*, only use one or the other. Do
not mix calls between the two.