Using Bio-Formats in MATLAB
===========================
.. highlight:: matlab
This section assumes that you have installed the MATLAB toolbox as instructed
in the :doc:`MATLAB user information page </users/matlab/index>`. Note the
minimum recommended MATLAB version is R2017b.
As described in `Using Java Libraries <http://mathworks.com/help/matlab/matlab_external/product-overview.html>`_,
every installation of MATLAB includes a |JVM| allowing use of the Java API and
third-party Java libraries. All the helper functions included in the MATLAB
toolbox make use of the Bio-Formats Java API. Please refer to the
:javadoc:`Javadocs <>` for more information.
Increasing |JVM| memory settings
--------------------------------
The default |JVM| settings in MATLAB can result in
``java.lang.OutOfMemoryError: Java heap space`` exceptions when opening large
image files using Bio-Formats. Information about the Java heap space usage in
MATLAB can be retrieved using::
java.lang.Runtime.getRuntime.maxMemory
MATLAB Preferences (R2010a+)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Default |JVM| settings can be increased by `Java Heap Memory Preferences
<https://mathworks.com/help/matlab/matlab_external/java-heap-memory-preferences.html>`_
of MATLAB (R2010a onwards) using
:menuselection:`Preferences --> General --> Java Heap Memory`. We recommend to use 512 MB in general, but you don't need to decrease the value if the default value is bigger than 512 MB.
.. image:: ../images/matlab_memory_pref512MB.png
However, the maximum value allowed in the Preferences GUI (typically set to 25% of Physical Memory, whihc you can ask by the ``memory`` MATLAB command) can still be too small
for your purpose. In that case, you can directly edit :file:`matlab.prf` file in the
folder specified by the ``prefdir`` `MATLAB function <https://mathworks.com/help/matlab/ref/prefdir.html>`_
(eg. ``'C:\Users\xxxxxxx\AppData\Roaming\MathWorks\MATLAB\R2018b'``). Find the
parameter ``JavaMemHeapMax`` in the file and increase the number that follows
the capital ``I`` (in MB) to increase the maximum Java heap memory size. The
change will be reflected by the Preferences as above.
::
JavaMemHeapMax=I25000
.. image:: ../images/matlab_memory_pref25GB.png
You'll see a warning message as above.
java.opts
^^^^^^^^^
Alternatively, this can also be done by creating a :file:`java.opts` file in
`the startup directory <https://mathworks.com/help/matlab/matlab_env/matlab-startup-folder.html>`_
and overriding the default memory settings (see `this page <https://mathworks.com/help/matlab/matlab_env/java-opts-file.html>`_
for more information). We recommend using ``-Xmx512m`` (meaning 512 MB) in your :file:`java.opts`
file.
Calling::
bfCheckJavaMemory()
will also throw a warning if the runtime memory is lower than the recommended
value.
If errors of type ``java.lang.OutOfMemoryError: PermGen space`` are thrown
while using Bio-Formats with the Java bundled with MATLAB, you
may try to increase the default values of ``-XX:MaxPermSize`` and
``-XX:PermSize`` via the :file:`java.opts` file.
.. seealso::
`Java Heap Memory Preferences (R2010a onwards) <https://uk.mathworks.com/help/matlab/matlab_external/java-heap-memory-preferences.html>`_
`How do I increase the heap space for the Java VM in MATLAB 6.0 (R12)
and later versions? <https://uk.mathworks.com/matlabcentral/answers/92813-how-do-i-increase-the-heap-space-for-the-java-vm-in-matlab-6-0-r12-and-later-versions>`_
:mailinglist:`[ome-users] Release of OMERO & Bio-Formats 5.1.1 <ome-users/2015-April/005331.html>`
Opening an image file
---------------------
The first thing to do is initialize a file with the
:source:`bfopen <components/formats-gpl/matlab/bfopen.m>` function:
::
data = bfopen('/path/to/data/file');
This function returns an ``n``-by-4 cell array, where ``n`` is the number of
series in the dataset. If ``s`` is the series index between 1 and ``n``:
- The ``data{s, 1}`` element is an ``m``-by-2 cell array, where ``m`` is the
number of planes in the ``s``-th series. If ``t`` is the plane index
between 1 and ``m``:
- The ``data{s, 1}{t, 1}`` element contains the pixel data for the
``t``-th plane in the ``s``-th series.
- The ``data{s, 1}{t, 2}`` element contains the label for the ``t``-th
plane in the ``s``-th series.
- The ``data{s, 2}`` element contains original metadata key/value pairs that
apply to the ``s``-th series.
- The ``data{s, 3}`` element contains color lookup tables for each plane in
the ``s``-th series.
- The ``data{s, 4}`` element contains a standardized OME metadata structure,
which is the same regardless of the input file format, and contains common
metadata values such as physical pixel sizes - see
:ref:`ome-metadata` below for examples.
Accessing planes
^^^^^^^^^^^^^^^^
Here is an example of how to unwrap specific image planes for easy access:
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: accessing-planes-start
:end-before: accessing-planes-end
Displaying images
^^^^^^^^^^^^^^^^^
If you want to display one of the images, you can do so as follows:
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: displaying-images-start
:end-before: displaying-images-end
This will display the first image of the first series with its associated
color map (if present). If you would prefer not to apply the color maps
associated with each image, simply comment out the calls to ``colormap``.
If you have the image processing toolbox, you could instead use:
::
imshow(series1_plane1, []);
You can also create an animated movie (assumes 8-bit unsigned data):
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: animated-movie-start
:end-before: animated-movie-end
Retrieving metadata
^^^^^^^^^^^^^^^^^^^
There are two kinds of metadata:
- **Original metadata** is a set of key/value pairs specific to the input
format of the data. It is stored in the ``data{s, 2}`` element of the data
structure returned by ``bfopen``.
- **OME metadata** is a standardized metadata structure, which is the
same regardless of input file format. It is stored in the ``data{s, 4}``
element of the data structure returned by ``bfopen``, and contains common
metadata values such as physical pixel sizes, instrument settings,
and much more. See the :model_doc:`OME Model and Formats <>` documentation
for full details.
Original metadata
"""""""""""""""""
To retrieve the metadata value for specific keys:
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: read-original-metadata-by-key-start
:end-before: read-original-metadata-by-key-end
To print out all of the metadata key/value pairs for the first series:
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: read-original-metadata-start
:end-before: read-original-metadata-end
.. _ome-metadata:
OME metadata
""""""""""""
Conversion of metadata to the OME standard is one of Bio-Formats' primary
features. The OME metadata is always stored the same way, regardless of input
file format.
To access physical voxel and stack sizes of the data:
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: read-ome-metadata-start
:end-before: read-ome-metadata-end
For more information about the methods to retrieve the metadata, see the
:xml_javadoc:`MetadataRetrieve <ome/xml/meta/MetadataRetrieve.html>` Javadoc
page.
To convert the OME metadata into a string, use the ``dumpXML()`` method:
::
omeXML = char(omeMeta.dumpXML());
Changing the logging level
--------------------------
By default, ``bfopen`` uses ``bfInitLogging`` to initialize the logging system
at the `WARN` level. To change the root logging level, use the
:common_javadoc:`DebugTools <loci/common/DebugTools.html>` methods as
described in the :doc:`logging` section.
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: logging-start
:end-before: logging-end
Reading from an image file
--------------------------
The main inconvenience of the
:source:`bfopen.m <components/formats-gpl/matlab/bfopen.m>` function is that
it loads all the content of an image regardless of its size.
To access the file reader without loading all the data, use the low-level
:source:`bfGetReader.m <components/formats-gpl/matlab/bfGetReader.m>`
function:
::
reader = bfGetReader('path/to/data/file');
You can then access the OME metadata using the ``getMetadataStore()``
method:
::
omeMeta = reader.getMetadataStore();
Individual planes can be queried using the
:source:`bfGetPlane.m <components/formats-gpl/matlab/bfGetPlane.m>` function:
::
series1_plane1 = bfGetPlane(reader, 1);
To switch between series in a multi-image file, use the
:javadoc:`setSeries(int) <loci/formats/IFormatReader.html#setSeries-int->` method. To retrieve a plane given a set of
`(z, c, t)` coordinates, these coordinates must be linearized first using
:javadoc:`getIndex(int, int, int) <loci/formats/IFormatReader.html#getIndex-int-int-int->`
::
% Read plane from series iSeries at Z, C, T coordinates (iZ, iC, iT)
% All indices are expected to be 1-based
reader.setSeries(iSeries - 1);
iPlane = reader.getIndex(iZ - 1, iC -1, iT - 1) + 1;
I = bfGetPlane(reader, iPlane);
Saving files
------------
The basic code for saving a 5D array into an OME-TIFF file is located in the
:source:`bfsave.m <components/formats-gpl/matlab/bfsave.m>` function.
For instance, the following code will save a single image of 64 pixels by 64
pixels with 8 unsigned bits per pixels:
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: bfsave-plane-start
:end-before: bfsave-plane-end
And the following code snippet will produce an image of 64 pixels by
64 pixels with 2 channels and 2 timepoints:
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: bfsave-multiple-planes-start
:end-before: bfsave-multiple-planes-end
By default, ``bfsave`` will create a minimal OME-XML metadata object
containing basic information such as the pixel dimensions, the dimension order
and the pixel type.
To customize the OME metadata, it is possible to create a metadata object
from the input array using :source:`createMinimalOMEXMLMetadata.m <components/formats-gpl/matlab/createMinimalOMEXMLMetadata.m>`, add custom
metadata and pass this object directly to ``bfsave``:
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: bfsave-metadata-start
:end-before: bfsave-metadata-end
The dimension order of the file on disk can be specified in two ways:
- either by passing an OME-XML metadata option as a key/value pair as shown above
- or as an optional positional argument of ``bfsave``
If a metadata object is passed to ``bfsave``, its dimension order stored
internally will take precedence.
For more information about the methods to store the metadata, see the
:xml_javadoc:`MetadataStore <ome/xml/meta/MetadataStore.html>` Javadoc page.
.. _reader_performance:
Improving reading performance
-----------------------------
Initializing a Bio-Formats reader can consume substantial time and memory.
Most of the initialization time is spend in the
:javadoc:`setId(java.lang.String) <loci/formats/IFormatHandler.html#setId-java.lang.String->`
call. Various factors can impact the performance of this step including the
file size, the amount of metadata in the image and also the file format itself.
One solution to improve reading performance is to use Bio-Formats memoization
functionalities with the
:javadoc:`loci.formats.Memoizer <loci/formats/Memoizer.html>` reader wrapper.
By essence, the speedup gained from memoization will only happen after the
first initialization of the reader for a particular file.
The simplest way to make use the ``Memoizer`` functionalities in MATLAB is
illustrated by the following example:
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: memoizer-start
:end-before: memoizer-end
If the time required to call :javadoc:`setId(java.lang.String) <loci/formats/Memoizer.html#setId-java.lang.String->` method is larger
than :javadoc:`DEFAULT_MINIMUM_ELAPSED <loci/formats/Memoizer.html#DEFAULT_MINIMUM_ELAPSED>` or the minimum value
passed in the constructor, the initialized reader will be cached in a memo
file under the same folder as the input file. Any subsequent call to
``setId()`` with a reader decorated by the ``Memoizer`` on the same input file
will load the reader from the memo file instead of performing a full reader
initialization.
More constructors are described in the
:javadoc:`Memoizer javadocs <loci/formats/Memoizer.html>` allowing to control
the minimal initialization time required before caching the reader and/or to
define a root directory under which the reader should be cached.
As Bio-Formats is not thread-safe, reader memoization offers a new solution to
increase reading performance when doing parallel work. For instance, the
following example shows how to combine memoization and MATLAB parfor to do
work on a single file in a parallel loop:
.. literalinclude:: examples/bftest.m
:language: matlab
:start-after: memoizer-parfor-start
:end-before: memoizer-parfor-end