NAME

       dirfile — a filesystem-based database format for time-ordered binary data

DESCRIPTION

       The  dirfile  database format is designed to provide a fast, simple format for storing and
       reading binary time-ordered data.  Dirfiles can be read using the GetData  Library,  which
       provides a reference implementaiton of these Standards.

       The  dirfile  database  is  centred  around  one or more time-ordered data streams (a time
       stream).  Each time stream is written to the filesystem in  a  separate  file,  as  binary
       data.   The  name  of  these  binary  files  correspond  to  the time stream's field name.
       Dirfiles support binary data fields for signed and unsigned integer types of 8 to 64 bits,
       as well as single and double precision floating-point real or complex data types.

       Two  time  streams  may  have different constant sampling frequencies and mechanisms exist
       within the dirfile format to ensure these time streams remain properly sequenced in time.

       To do this, the time streams in the  dirfile  are  subdivided  into  frames.   Each  frame
       contains  a  fixed integer number of samples of each time stream.  Two time streams in the
       same dirfile may have different numbers of samples per frame, but the  number  of  samples
       per frame of any given time stream is fixed.

       When synchronous retrieval of data from more than one time stream is required, position in
       the dirfile can be specified in frames, which will ensure synchronicity.

       The binary files are all located in one ore more filesystem directories, rooted  around  a
       central directory, known as the dirfile directory.  The dirfile as a whole may be referred
       to by its dirfile directory path.

       Included in the dirfile along with the time streams is the dirfile  format  specification,
       which  is  one  or  more  ASCII  text files containing the dirfile database metadata.  The
       primary file is the file called format located in the dirfile directory.   This  file  and
       any  additional files that it names, fully specify the dirfile's metadata.  For the syntax
       of these files, see dirfile-format(5).

       Version 3 of the Dirfile Standards introduced the large dirfile extension.  This extension
       added  the  ability  to  distribute  the  dirfile  metadata  among  multiple files (called
       fragments) in addition to the format file, as well as the ability to house portions of the
       database  in  subdirfiles.   These  subdirfiles may be fully fledged dirfiles in their own
       right, but may also be contained within a larger, parent dirfile.   See  dirfile-format(5)
       for information on specifying these subdirfiles.

       In  addition  to the raw fields on disk, the dirfile format specification may also specify
       derived fields which are calculated by performing simple element-wise operations on one or
       more input fields.  Derived fields behave identically to raw fields when read via GetData.
       See dirfile-format(5) for a complete list of  derived  field  types.   Dirfiles  may  also
       contain  both numerical and character string constant scalar fields, also further outlined
       in dirfile-format(5).

       Dirfiles are designed to be written to and read simultaneously. The dirfile  specification
       dictates  that  one particular raw field (specified either explicitly or implicitly by the
       dirfile metadata) is to be used as the  reference  field:  all  other  vector  fields  are
       assumed  to  have  at  least  as  many frames as the reference field has, and the size (in
       frames) of the reference field is used as the size of the dirfile as a whole.

       Version 6 of the Dirfile Standards added the ability to encode the binary files  on  disk.
       Each  fragment  may  have  its  own encoding scheme.  Most commonly, encodings are used to
       compress the data files  to  same  space.   See  dirfile-encoding(5)  for  information  on
       encoding schemes.

   Complex Number Storage Format
       Version  7  of  the Dirfile Standards added support for complex valued data.  Two types of
       complex valued data are supported by the Dirfile Standards:

       •   A 64-bit complex number consisting of a  IEEE-754  standard  32-bit  single  precision
           floating  point  real  part  and  a IEEE-754 standard 32-bit single precision floating
           point imaginary part, and

       •   A 128-bit complex number consisting of a IEEE-754  standard  64-bit  double  precision
           floating  point  real  part  and  a IEEE-754 standard 64-bit double precision floating
           point imaginary part.

       No integer-type complex numbers are supported.

       Unencoded complex numbers are stored on disk in "Fortran order", that is with the IEEE-754
       real  part  followed  by the IEEE-754 imaginary part.  The specified endianness of the two
       components follows that of purely real floating point numbers.  Endianness does not affect
       the  ordering  of  the real and imaginary parts.  This format also conforms to the C99 and
       C++11 standards.

       To aid in using complex valued data, dirfile field  codes  may  contain  a  representation
       suffix  which  specifies  a  function  to  apply to the complex valued data to map it into
       purely real data.  See dirfile-format(5).

AUTHORS

       The Dirfile format was created by C. B. Netterfield  <netterfield@astro.utoronto.ca>.   It
       is now maintained by D. V. Wiebe <getdata@ketiltrout.net>.

SEE ALSO

       dirfile-encoding(5), dirfile-format(5)

       For  an  introduction  to  the GetData Library reference implementation, see gd_open(3),or
       gd_getdata(3), or visit the GetData Project website: <http://getdata.sourceforge.net/>.