Writing a new file format reader
================================
This document is a brief guide to writing new Bio-Formats file format readers.
All format readers should extend either :javadoc:`loci.formats.FormatReader
<loci/formats/FormatReader.html>` or :javadoc:`an existing reader <loci/formats/in/package-summary.html>`.
Methods to override
-------------------
- :javadoc:`isSingleFile(java.lang.String) <loci/formats/IFormatReader.html#isSingleFile-java.lang.String->`
Whether or not the named file is expected to be the only file in the
dataset. This only needs to be overridden for formats whose datasets can
contain more than one file.
- :javadoc:`isThisType(loci.common.RandomAccessInputStream) <loci/formats/IFormatReader.html#isThisType-loci.common.RandomAccessInputStream->`
Check the first few bytes of a file to determine if the file can be read by
this reader. You can assume that index 0 in the stream corresponds to
the index 0 in the file. Return true if the file can be read; false if
not (or if there is no way of checking).
- :javadoc:`fileGroupOption(java.lang.String) <loci/formats/IFormatReader.html#fileGroupOption-java.lang.String->`
Returns an indication of whether or not the files in a multi-file dataset
can be handled individually. The return value should be one of the
following:
* :javadoc:`FormatTools.MUST_GROUP <loci/formats/FormatTools.html#MUST_GROUP>`:
the files cannot be handled separately
* :javadoc:`FormatTools.CAN_GROUP <loci/formats/FormatTools.html#CAN_GROUP>`:
the files may be handled separately or as a single unit
* :javadoc:`FormatTools.CANNOT_GROUP <loci/formats/FormatTools.html#CANNOT_GROUP>`:
the files must be handled separately
This method only needs to be overridden for formats whose datasets can
contain more than one file.
- :javadoc:`getSeriesUsedFiles(boolean) <loci/formats/IFormatReader.html#getSeriesUsedFiles-boolean->`
You only need to override this if your format uses multiple files in a
single dataset. This method should return a list of all files
associated with the given file name and the current series (i.e. every file
needed to display the current series). If the ``noPixels`` flag is set, then
none of the files returned should contain pixel data.
For an example of how this works, see
:bfreader:`loci.formats.in.PerkinElmerReader <PerkinElmerReader.java>`. It
is recommended that the first line of this method be
``FormatTools.assertId(currentId, true, 1)`` - this ensures that the file
name is non-null.
- :javadoc:`openBytes(int, byte[], int, int, int, int) <loci/formats/IFormatReader.html#openBytes-int-byte:A-int-int-int-int->`
Returns a byte array containing the pixel data for a specified subimage
from the given file. The dimensions of the subimage (upper left X
coordinate, upper left Y coordinate, width, and height) are specified in the
final four int parameters. This should throw a
:javadoc:`FormatException <loci/formats/FormatException.html>` if the image
number is invalid (less than 0 or >= the number of images). The ordering of
the array returned by openBytes should correspond to the values returned by
:javadoc:`isLittleEndian <loci/formats/IFormatReader.html#isLittleEndian-->`
and
:javadoc:`isInterleaved <loci/formats/IFormatReader.html#isInterleaved-->`.
Also, the length of the byte array should be [image width * image height *
bytes per pixel]. Extra bytes will generally be truncated. It is recommended
that the first line of this method be
``FormatTools.checkPlaneParameters(this, no, buf.length, x, y, w, h)`` -
this ensures that all of the parameters are valid.
- :javadoc:`initFile(java.lang.String) <loci/formats/FormatReader.html#initFile-java.lang.String->`
The majority of the file parsing logic should be placed in this method. The
idea is to call this method once (and only once!) when the file is first
opened. Generally, you will want to start by calling
``super.initFile(String)``. You will also need to set up the stream for
reading the file, as well as initializing any dimension information and
metadata.
Most of this logic is up to you; however, you should populate the
:javadoc:`core <loci/formats/FormatReader.html#core>` variable (see
:javadoc:`loci.formats.CoreMetadata <loci/formats/CoreMetadata.html>`).
Note that each variable is initialized to 0 or null when
``super.initFile(String)`` is called.
Also, ``super.initFile(String)`` constructs a Hashtable called
:javadoc:`metadata <loci/formats/FormatReader.html#metadata>`
where you should store any relevant metadata.
The most common way to set up the OME-XML metadata for the reader is to
initialize the MetadataStore using the
:javadoc:`makeFilterMetadata() <loci/formats/FormatReader.html#makeFilterMetadata-->` method and populate the
Pixels elements of the metadata store from the ``core`` variable using the
:javadoc:`MetadataTools.populatePixels(MetadataStore, FormatReader) <loci/formats/MetadataTools.html#populatePixels-loci.formats.meta.MetadataStore-loci.formats.IFormatReader->` method::
# Initialize the OME-XML metadata from the core variable
MetadataStore store = makeFilterMetadata();
MetadataTools.populatePixels(store, this);
If the reader includes metadata at the plane level, you can initialize the
Plane elements under the Pixels using
:javadoc:`MetadataTools.populatePixels(MetadataStore, FormatReader, doPlane) <loci/formats/MetadataTools.html#populatePixels-loci.formats.meta.MetadataStore-loci.formats.IFormatReader-boolean->`::
MetadataTools.populatePixels(store, this, true);
Once the metadatastore has been initialized with the core properties,
additional metadata can be added to it using the setter methods. Note that
for each of the model components, the ``setObjectID()`` method should be
called before any of the ``setObjectProperty()`` methods, e.g.::
# Add an oil immersion objective with achromat
String objectiveID = MetadataTools.createLSID("Objective", 0, 0);
store.setObjectiveID(objectiveID, 0, 0);
store.setObjectiveImmersion(getImmersion("Oil"), 0, 0);
- :javadoc:`close(boolean) <loci/formats/IFormatReader.html#close-boolean->`
Cleans up any resources used by the reader. Global variables should be
reset to their initial state, and any open files or delegate readers should
be closed.
Note that if the new format is a variant of a format currently supported by
Bio-Formats, it is more efficient to make the new reader a subclass of the
existing reader (rather than subclassing :javadoc:`loci.formats.FormatReader
<loci/formats/FormatReader.html>`). In this case, it is
usually sufficient to override
:javadoc:`initFile(java.lang.String) <loci/formats/FormatReader.html#initFile-java.lang.String->` and :javadoc:`isThisType(byte[]) <loci/formats/FormatReader.html#isThisType-byte:A->`.
Every reader also has an instance of :javadoc:`loci.formats.CoreMetadata
<loci/formats/CoreMetadata.html>`. All readers should populate the fields in
CoreMetadata, which are essential to reading image planes.
If you read from a file using something other than
:common_javadoc:`loci.common.RandomAccessInputStream <loci/common/RandomAccessInputStream.html>` or
:common_javadoc:`loci.common.Location <loci/common/Location.html>`,
you *must* use the file name returned by ``Location.getMappedId(String)``, not
the file name passed to the reader.
Thus, a stub for ``initFile(String)`` might look like this:
::
protected void initFile(String id) throws FormatException, IOException {
super.initFile(id);
RandomAccessInputStream in = new RandomAccessInputStream(id);
// alternatively,
//FileInputStream in = new FileInputStream(Location.getMappedId(id));
// read basic file structure and metadata from stream
}
For more details, see
:common_javadoc:`loci.common.Location.mapId(java.lang.String, java.lang.String) <loci/common/Location.html#mapId-java.lang.String-java.lang.String->`
and :common_javadoc:`loci.common.Location.getMappedId(java.lang.String) <loci/common/Location.html#getMappedId-java.lang.String->`.
Variables to populate
---------------------
There are a number of global variables defined in
:javadoc:`loci.formats.FormatReader <loci/formats/FormatReader.html>` that
should be populated in the constructor of any implemented reader.
These variables are:
- :javadoc:`suffixNecessary <loci/formats/FormatReader.html#suffixNecessary>`
Indicates whether or not a file name suffix is required; true by default
- :javadoc:`suffixSufficient <loci/formats/FormatReader.html#suffixSufficient>`
Indicates whether or not a specific file name suffix guarantees that this
reader can open a particular file; true by default
- :javadoc:`hasCompanionFiles <loci/formats/FormatReader.html#hasCompanionFiles>`
Indicates whether or not there is at least one file in a dataset of this
format that contains only metadata (no images); false by default
- :javadoc:`datasetDescription <loci/formats/FormatReader.html#datasetDescription>`
A brief description of the layout of files in datasets of this format; only
necessary for multi-file datasets
- :javadoc:`domains <loci/formats/FormatReader.html#domains>`
An array of imaging domains for which this format is used. Domains are
defined in
:javadoc:`loci.formats.FormatTools <loci/formats/FormatTools.html>`.
Other useful things
-------------------
- :common_javadoc:`loci.common.RandomAccessInputStream
<loci/common/RandomAccessInputStream.html>` is a
hybrid RandomAccessFile/InputStream
class that is generally more efficient than either RandomAccessFile or
InputStream, and implements the DataInput interface. It is recommended that
you use this for reading files.
- :common_javadoc:`loci.common.Location <loci/common/Location.html>` provides an API
similar to java.io.File, and supports
File-like operations on URLs. It is highly recommended that you use this
instead of File. See the :javadoc:`Javadocs <>` for additional information.
- :common_javadoc:`loci.common.DataTools <loci/common/DataTools.html>` provides a
number of methods for converting bytes to
shorts, ints, longs, etc. It also supports reading most primitive types
directly from a RandomAccessInputStream (or other DataInput implementation).
- :javadoc:`loci.formats.ImageTools <loci/formats/ImageTools.html>` provides several methods
for manipulating
primitive type arrays that represent images. Consult the source or Javadocs
for more information.
- If your reader relies on third-party code which may not be available to all
users, it is strongly suggested that you make a corresponding service class
that interfaces with the third-party code. Please see :doc:`service` for a
description of the service infrastructure, as well as the
:javadoc:`loci.formats.services package <loci/formats/services/package-summary.html>`.
- Several common image compression types are supported through subclasses of
:javadoc:`loci.formats.codec.BaseCodec <loci/formats/codec/BaseCodec.html>`. These include JPEG, LZW, LZO, Base64, ZIP and RLE (PackBits).
- If you wish to convert a file's metadata to OME-XML (strongly encouraged),
please see :doc:`Bio-Formats metadata processing </about/index>` for further information.
- Once you have written your file format reader, add a line to the
:source:`readers.txt <components/formats-api/src/loci/formats/readers.txt>`
file with the fully qualified name of the reader, followed by a '#' and the
file extensions associated with the file format. Note that
:javadoc:`loci.formats.ImageReader <loci/formats/ImageReader.html>`,
the master file format reader, tries to identify which format reader to use
according to the order given in :source:`readers.txt <components/formats-api/src/loci/formats/readers.txt>`, so be sure to place your
reader in an appropriate position within the list.
- The easiest way to test your new reader is by calling "java
loci.formats.tools.ImageInfo <file name>". If all goes well, you should see
all of the metadata and dimension information, along with a window showing
the images in the file. :javadoc:`loci.formats.ImageReader <loci/formats/ImageReader.html>` can take additional
parameters; a brief listing is provided below for reference, but it is
recommended that you take a look at the contents of
:source:`loci.formats.tools.ImageInfo <components/bio-formats-tools/src/loci/formats/tools/ImageInfo.java>` to see
exactly what each one does.
+--------------+----------------------------------------------------------+
| Argument | Action |
+==============+==========================================================+
| -version | print the library version and exit |
+--------------+----------------------------------------------------------+
| file | the image file to read |
+--------------+----------------------------------------------------------+
| -nopix | read metadata only, not pixels |
+--------------+----------------------------------------------------------+
| -nocore | do not output core metadata |
+--------------+----------------------------------------------------------+
| -nometa | do not parse format-specific metadata table |
+--------------+----------------------------------------------------------+
| -nofilter | do not filter metadata fields |
+--------------+----------------------------------------------------------+
| -thumbs | read thumbnails instead of normal pixels |
+--------------+----------------------------------------------------------+
| -minmax | compute min/max statistics |
+--------------+----------------------------------------------------------+
| -merge | combine separate channels into RGB image |
+--------------+----------------------------------------------------------+
| -nogroup | force multi-file datasets to be read as individual files |
+--------------+----------------------------------------------------------+
| -stitch | stitch files with similar names |
+--------------+----------------------------------------------------------+
| -separate | split RGB image into separate channels |
+--------------+----------------------------------------------------------+
| -expand | expand indexed color to RGB |
+--------------+----------------------------------------------------------+
| -omexml | populate OME-XML metadata |
+--------------+----------------------------------------------------------+
| -normalize | normalize floating point images* |
+--------------+----------------------------------------------------------+
| -fast | paint RGB images as quickly as possible* |
+--------------+----------------------------------------------------------+
| -debug | turn on debugging output |
+--------------+----------------------------------------------------------+
| -range | specify range of planes to read (inclusive) |
+--------------+----------------------------------------------------------+
| -series | specify which image series to read |
+--------------+----------------------------------------------------------+
| -swap | override the default input dimension order |
+--------------+----------------------------------------------------------+
| -shuffle | override the default output dimension order |
+--------------+----------------------------------------------------------+
| -map | specify file on disk to which name should be mapped |
+--------------+----------------------------------------------------------+
| -preload | pre-read entire file into a buffer; significantly |
| | reduces the time required to read the images, but |
| | requires more memory |
+--------------+----------------------------------------------------------+
| -crop | crop images before displaying; argument is 'x,y,w,h' |
+--------------+----------------------------------------------------------+
| -autoscale | used in combination with '-fast' to automatically adjust |
| | brightness and contrast |
+--------------+----------------------------------------------------------+
| -novalid | do not perform validation of OME-XML |
+--------------+----------------------------------------------------------+
| -omexml-only | only output the generated OME-XML |
+--------------+----------------------------------------------------------+
| -format | read file with a particular reader (e.g., ZeissZVI) |
+--------------+----------------------------------------------------------+
\* = may result in loss of precision
- If you wish to test using TestNG, ``loci.tests.testng.FormatReaderTest``
provides several basic tests that work with all Bio-Formats readers. See
the FormatReaderTest source code for additional information.
- For more details, please look at the source code and :javadoc:`Javadocs <>`.
Studying existing readers is probably the best way to get a feel for the
API; we would recommend first looking at
:bfreader:`loci.formats.in.ImarisReader <ImarisReader.java>` (this is the
most straightforward one). :bfreader:`loci.formats.in.LIFReader <LIFReader.java>` and :bfreader:`InCellReader <InCellReader.java>` are also
good references that show off some of the nicer features of Bio-Formats.
If you have questions about Bio-Formats, please contact :imagesc:`the forums <tags/bio-formats>`.