Ubuntu Manpage: elektra-meta-data

name
rationale
implementation

Provided by: elektra-doc_0.8.14-5.1ubuntu2_all

NAME

       elektra-meta-data - meta data

       metadata  is  data  about data. Up to now, there has been a limited number of metadata entries suited for
       filesys. For filesys this was efficient, but it  was  of  limited  use  for  every  other  backend.  This
       situation has now changed fundamentally by introducing arbitrary metadata.

       In  Elektra,  metakeys  represent  metadata.  They  can  be  stored  in  a key database, often within the
       representation of the Key. Metakey is a Key that is part of the data structure Key. It can be  looked  up
       using  a  metaname. Metanames are unique within a Key. Metakeys can also carry a value, called metavalue.
       It is possible to iterate over all metakeys of a Key, but it is impossible for a metakey  to  hold  other
       metakeys  recursively.  The  purpose of metakeys next to keys is to distinguish between configuration and
       information about settings.

Rationale

Metadata has different purposes:

• Traditionally Elektra used metadata to carry file system semantics. The backend filesys stores file
metadata (File metadata in POSIX is returned by stat()) in a struct with the same name. It contains a
file type (directory, symbolic link,..) as well as other metadata like uid, gid, owner, mode, atime,
mtime and ctime. into the Key objects. This solution, however, only makes sense when each file
shelters only one Key object.

• The metaname binary shows if a Key object contains binary data. Otherwise it has a null-terminated C
string.

• An application can set and get a flag in Key objects.

• Comments and owner, together with the items above, were the only metadata possible before arbitrary
metadata was introduced.

• Further metadata can hold information on how to check and validate keys using types or regular
expressions. Additional constraints concerning the validity of values can be convenient. Maximum
length, forbidden characters and a specified range are examples of further constraints.

• They can denote when the value has changed or can be informal comments about the content or the
character set being used.

• They can express the information the user has about the key, for example, comments in different
languages. Language specific information can be supported by simply adding a unique language code to
the metaname.

• They can represent information to be used by storage plugins. Information can be stored as syntactic,
semantic or additional information rather than text in the key database. This could be ordering or
version information.

• They can be interpreted by plugins, which is the most important purpose of metadata. Nearly all kinds
of metadata mentioned above can belong to this category.

• Metadata is used to pass error or warning information from plugins to the application. The
application can decide to present it to the user. The information is uniquely identified by numerical
codes. Metadata can also embed descriptive text specifying a reason for the error.

• Applications can remember something about keys in metadata. Such metadata generalises the
application-defined flag.

• A more advanced idea is to use metadata to generate forms in a programmatic way. While it is
certainly possible to store the necessary expressive metadata, it is plenty of work to define the
semantics needed to do that.

Implementation

       In  this  document,  we discuss the implementation of metadata. Metakey is implemented directly in a Key:
       Every metakey belongs to a key inseparable. Unlike normal key names there is no absolute path for  it  in
       the hierarchy, but a relative one only valid within the key.

       The advantage of embedding metadata into a key is that functions can operate on a key´s metadata if a key
       is passed as a parameter. Because of this,  keyNew()  directly  supports  adding  metadata.  A  key  with
       metadata  is self-contained. When the key is passed to a function, the metadata is always passed with it.
       Because of the tight integration into a Key, the metadata does not disturb the user.

       A disadvantage of this approach is that storage plugins  are  more  likely  to  ignore  metadata  because
       metakeys are distinct from keys and have to be handled separately. It is not possible to iterate over all
       keys and their metadata in a single loop. Instead only a nested loop provides  full  iteration  over  all
       keys and metakeys.

       The metakey itself is also represented by a Key. So the data structure Key is nested directly into a Key.
       The reason for this is to make the concept easier for the user who already knows how to work with a  Key.
       But even new users need to learn only one interface. During iteration the metakeys, represented through a
       Key object, contain both the metaname and the metavalue. The metaname is shorter than a key name  because
       the name is unique only in the Key and not for the whole global configuration.

       The  implementation  adds  no  significant  memory  overhead per Key if no metadata is used. For embedded
       systems it is useful to have keys without metadata. Special plugins can help for systems that  have  very
       limited  memory  capacity.  Also  for systems with enough memory we should consider that adding the first
       metadata to a key has some additional overhead. In the current implementation a new KeySet  is  allocated
       in this situation.

   Interface
       The interface to access metadata consists of the following functions:

       Interface of metadata:

           const Key *keyGetMeta(const Key *key, const char* metaName);
           ssize_t    keySetMeta(Key *key, const char* metaName,
               const char *newMetaString);

       Inside  a Key, metadata with a given metaname and a metavalue can be set using keySetMeta() and retrieved
       using keyGetMeta(). Iteration over metadata is possible with:

       Interface for iterating metadata:

           int keyRewindMeta(Key *key);
           const Key *keyNextMeta(Key *key);
           const Key *keyCurrentMeta(const Key *key);

       Rewinding and forwarding to the next key works as for  the  KeySet.  Programmers  used  to  Elektra  will
       immediately  be  familiar  with  the  interface.  Tiny  wrapper  functions still support the old metadata
       interface.

   Sharing of Metakey
       Usually substantial amounts of metadata are shared between keys. For example, many  keys  have  the  type
       int.  To avoid the problem that every key with this metadata occupies additional space, keyCopyMeta() was
       invented. It copies metadata from one key to another. Only one metakey resides in memory as long  as  the
       metadata is not changed with keySetMeta(). To copy metadata, the following functions can be used:

           int keyCopyMeta(Key *dest, const Key *source, const char *metaName);
           int keyCopyAllMeta(Key *dest, const Key *source);

       The  name copy is used because the information is copied from one key to another. It has the same meaning
       as in ksCopy(). In both cases it is a flat copy. keyCopyAllMeta() copies all metadata  from  one  key  to
       another. It is more efficient than a loop with the same effect.

       keyDup()  copies  all metadata as expected. Sharing metadata makes no difference from the user´s point of
       view. Whenever a metavalue is changed a new metakey is generated. It does not matter if the  old  metakey
       was  shared or not. This is the reason why a const pointer is always passed back. The metakey must not be
       changed because it can be used within another key.

                                                    July 2017                               ELEKTRA-META-DATA(7)