Scaling Omero ============= There are several ways that OMERO, or any server system, can scale. Optimizing your system for more than one of these factors is non-trivial, but we try to lay out some guidelines below for what has worked, what almost certainly will not work, and what -- under the right circumstances -- might be optimal. Concurrent invocations ---------------------- The bottlenecks for concurrent invocations are: - database connections - server threads - the router Database connections ~~~~~~~~~~~~~~~~~~~~ Database servers, in general, have a maximum number of allowed connections. In postgres, the default ``max_connections`` is 100, though in many cases this will be significantly lower due to the available shared memory (SHMMAX). If OMERO were to use direct connections to the database, after ``max_connections`` invocations, all further attempts to connect to the server would fail with "too many connection" exceptions. Instead, OMERO uses a **connection pool** in front of Postgres, which manages many more simultaneous attempts to connect to the database. OMERO's connection pool size as determined by :property:`omero.db.poolsize` should be lower than ``max_connections``. With the default ``max_connections`` set to 64, it is possible to execute 500 queries simultaneously without database exceptions. Instead, one receives server exceptions. Server threads ~~~~~~~~~~~~~~ In :doc:`/developers/server-blitz`, too many (500+ on the default configuration) simultaneous invocations will result in ``ConnectionLost`` exceptions. We are currently working on ways to extend the number of single invocations on one server, but a simpler solution is to start another :doc:`/developers/server-blitz` server. Total throughput ---------------- The bottlenecks for throughput are: - maximum message size - server memory - IO - network -------------- .. seealso:: :doc:`/sysadmins/unix/server-postgresql` Instructions about OMERO.server and PostgreSQL under UNIX and UNIX-like platforms. :doc:`/sysadmins/grid` :doc:`Clustering`