Provided by: elektra-doc_0.8.14-5_all 
NAME
doc_decisions_global_plugins_md - Global Plugins
Issue
• Checker plugins see only part of the configuration and cannot check constraints between
keys of different mountpoints
• Notification does not happen once after final commit, but for every plugin
• Some plugins are not implementable, e.g. global locks (that lock before any other plugin
and unlock after any other), but also journal plugins that keep track of actions to be
done and already done (latter is maybe out of scope for this decision)
Constraints
• Use arrays in configuration and allow arbitrary number of global plugins
• Plugin interface should be the same. Many plugins, e.g. dbus, should work as global
plugins w/o any significant change (e.g. only one more entry in contract)
Assumptions
• Global plugins to not depend on specific applications nor specific mountpoints.
• There is no plugin that works as global plugin but will not work as mounted plugin, too
(to investigate: locker plugins can lead to deadlocks?)
Considered Alternatives
• using different plugin interface (like hooks)
Decision
Configuration will be in arrays below the keys:
system/elektra/globalplugins/
prerollback
postrollback
preget
postget
preset
setstorage
precommit
postcommit
Plugins state in contract that they will work as global plugin, i.e. do not need to work
on individual config files, when following contract is present:
infos/global
The kdb-tool should have following list-like interface:
kdb global-plugin-add
kdb global-plugin-del
kdb global-plugin-list
Argument
Some nice features that will be implemented as global plugins.
Transformation
Transformation keys which are read and transformed to be usable by the application:
[dir/a]
transform=/x
transform/python=...upper()
/lua=..
(actually two plugins are involved: one that fetches transformation keys, the other that
executes the transformation code)
• preget: fetch all foreign keys (kdbGet)
• postget: run transformation for all foreign keys
Global lock
simplifies threading and process locking by not having to think about recursive cases.
Shell plugins
Run shell code at end of all plugins, e.g. especially doing
git add
git commit
Inference plugins
The globbing would be more natural (derived from specification). Or even more advanced
ways to copy information from specification to the keys, e.g. type inference
Journalling plugins
It should be possible to write plugins which need all file names of all resolver plugins.
E.g. journalling, global mmap.
Implications
Default global plugins
Its useful to have some important global plugins, e.g. locking by default. Internal list
to be used when no system/elektra/global_mountpoints/ exists.
State diagrams of plugins need to be redrawn to also include global plugin states.
Related decisions
Notes
Open Points
• How to test global plugins?
• locker plugins can lead to deadlocks? (must be avoided by contract?)
Implementation Hints
• add Plugin *globalPlugins [NR_OF_PLUGINS] to _KDB
• during kdbOpen, system/elektra/globalplugins/ is read and plugins are constructed and
placed into globalPlugins.
• In kdbGet and kdbSet hooks execute one of these plugins
• by default
• the plugins are all the same list plugins, and their subplugins are executed, when
system/elektra/globalplugins/_ states they should be executed
• a lock plugin that executes at begin and end of kdbGet and kdbSet, respective, i.e.
postrollback preget postget preset postcommit
• the lock plugin contains the code currently found in resolver
a plugin](http://libelektra.org/blob/master/src/plugins/lua/). In a similar way, someone
can write scripts, which are executed on every access to the
http://libelektra.org/blob/master/doc/help/elektra-glossary.md 'key database'.
To mount a lua based filter, you can use:
kdb mount file.ini /lua ini lua script=/path/to/lua/lua_filter.lua
Even though it works well, it is classified as technical preview.
Thanks to Manuel Mausz for this plugin!
Cryptography Plugin
In this technical preview, Peter Nirschl demonstrates how a plugin can encrypt Elektra's
values. In testcases it is already able to do so, but for the end user an easy way for key
derivation is missing.
A big thanks to Peter Nirschl!
Conditionals
Brings if inside Elektra. It lets you check if some keys have the values they should have.
kdb mount conditionals.dump /tmount/conditionals conditionals dump
kdb set user/tmount/conditionals/fkey 3.0
kdb set user/tmount/conditionals/hkey hello
kdb setmeta user/tmount/conditionals/key check/condition "(hkey == 'hello') ? (fkey == '3.0')" # success
kdb setmeta user/tmount/conditionals/key check/condition "(hkey == 'hello') ? (fkey == '5.0')" # fail
INI Plugin
The INI plugin got a near rewrite. Now it handles many situations better, has many more
options and features, including:
• preserving the order
• using keys as meta-data
• many new testcases
• fix escaping
Thanks to Thomas Waser for this work!
List Plugin
Currently, Elektra has some limitations on how many plugins can be added to certain
http://libelektra.org/blob/master/doc/help/elektra-plugins-ordering.md 'placements'.
Because of the rapidly growing number of plugins, some combinations are not possible
anymore.
This plugin tackles the issue, by delegating the work to an arbitrary number of
subplugins. As a bonus, it works lazily and thus might avoid the loading of some plugins
all together.
Thanks to Thomas Waser for this plugin!
Csvstorage Plugin
You can now mount csv-files. To mount test.csv simply use:
kdb mount test.csv /csv csvstorage
There are many options, e.g. changing the delimiter, use header for the key names or
predefine how the columns should be named. For details see the documentation.
Thanks to Thomas Waser!
Filecheck plugin
The also new plugin lineendings is already superseded by the filecheck plugin.
Thanks to Thomas Waser!
Enum plugin
The Enum plugin checks string values of Keys by comparing it against a list of valid
values.
Thanks to Thomas Waser!
Electrify Machinekit.io
We are proud that Machinekit starts using Elektra.
Alexander Rössler is digging into all details, and already enhanced the DBUS Plugin for
their needs. DBus now can emit a message for every changed key.
A big thanks to Alexander Rössler!
KDB Tools:
• fix kdb check return code (open fail)
Bugfixes
• libgetenv did not reinitalized its mutexes on forks
• add needSync also in C++ binding
• handle removed current working directories (fallback to /)
• avoid segfault on missing version keys (when doing kdb rm system/elektra/version)
• fix glob plugin + kdb mount with http://libelektra.org/blob/master/doc/help/elektra-
contracts.md 'config/needs usage'
• Mac OS X fix different handling of strerror_r (thanks to Daniel Bugl)
• do not change parentKey in early-error scenarios
• do not try to interpret some binary keys as function keys
Other Gems
• getenv example: do not link to elektra/elektratools, thanks to Pino Toscano
• fixes in other examples
• avoid useless UTF-8 chars and fix typos, thanks to Kurt Micheli
• pdf now also allows UTF-8 characters if added to elektraSpecialCharacters.sty, thanks to
Kurt Micheli
• libgetenv: lookup also used for layers
• handle wrong arguments of metals better, thanks to Ian Donnelly
• Improvement of error messages in the augeas plugin
• kdb set avoids fetching unnecessary namespaces
• verbose unmount
• logchange: small demonstration plugin to show how to log added, removed and changed keys
• setmeta will use spec as default
• libtools: avoid useless getName, add verbosity flag for findBackend
• Improve iconv error messages
• That mount needs permissions to /etc should now really be obvious with new error message
• many fixes in the template for new plugins
Get It!
You can download the release from here and now also here on github
• name: elektra-0.8.14.tar.gz
• TODO: hash sums missing
This release tarball now is also available signed by me using gpg
already built API-Docu can be found here
Stay tuned!
Subscribe to the RSS feed to always get the release notifications.
For any questions and comments, please contact the Mailing List the issue tracker on
github or by mail elektra@markus-raab.org.
Permalink to this NEWS entry
For more information, see http://libelektra.org
Best regards, Markus
0.8.13 Release
• guid: 3c00a5f1-c017-4555-92b5-a2cf6e0803e3
• author: Markus Raab
• pubDate: Thu, 17 Sep 2015 17:32:16 +0200
Again we managed to release with many new features, many fixes and also other
improvements.
Elektrify-getenv
getenv(3) is one of the most popular ways to retrieve configuration, even though it has
many known problems:
• no standard way to modify it
• relogin (or restart of shell) necessary
• names are flat (no hierarchical structure)
• cannot be set for individual applications
• different in at, cron and similar scripts
With elektrify-getenv we wrote a solution which solves most of the problems. We use the
LD_PRELOAD technique to additionally retrieve values from Elektra, and not only the
environment.
You simply can do:
```bash kdb set user/env/override/HTTP_PROXY 'http://my.proxy:8080' ```
This will set the HTTP_PROXY environment variable to http://my.proxy:8080. Configuration
can be retrieved with kdb get:
```bash kdb get user/env/override/HTTP_PROXY lynx # or start another www-browser with the
newly set HTTP_PROXY ```
Or using the man pages:
kdb elektrify-getenv man man --elektra:MANWIDTH=40
Will use MANWIDTH 40 for this invocation of man man. This feature is handy, if an option
is only available by environment, but not by command-line arguments, because sometimes
environment variables are not trivial to set (e.g. in Makefiles).
Some more examples:
kdb set user/env/override/MANOPT -- "--regex -LC"
kdb elektrify-getenv getenv MANOPT # to check if it is set as expected
kdb getenv MANOPT # if /etc/ld.so.preload is active
So is this the final solution for configuration and manual elektrification of applications
is not needed anymore?
The answer is: no and yes.
It is quite satisfactory for configuration that is inherently sharable (not different from
one application to another) and needs the environment semantics, i.e. some subprocesses
should have different configuration than others, e.g. in a specific terminal.
But it might not be a good solution for your own application, because libgetenv(3) implies
many architectural decision, that other elektrified applications would decide differently,
e.g.:
• it uses global variables (getenv(3) has no handle)
• it uses mutex for multi-threading safety
• the API getenv(3) only returns char* and has no support for other data types
For more information see http://git.libelektra.org/blob/master/src/libgetenv/README.md
'src/libgetenv/README.md'
Compatibility
As always, the API and API is fully forward-compatible, i.e. programs compiled against an
older 0.8 versions of Elektra will continue to work.
Because keyUnescapedName and keyGetUnescapedNameSize is added in this release, it is not
backward-compatible, i.e. programs compiled against 0.8.13, might not work with older 0.8
libraries.
The function keyUnescapedName provides access to an unescaped name, i.e. one where / and
\\ are literal symbols and do not have any special meaning. NULL characters are used as
path separators. This function makes it trivial and efficient to iterate over all path
names, as already exploited in all bindings:
• jna (java)
• lua
• python2
• python3
Other small changes/additions in bindings:
• fix key constructor, thanks to Manuel Mausz
• add copy and deepcopy in python (+examples,+testcases), thanks to Manuel Mausz
• dup() in python3 returned wrong type (SWIG wrapper), thanks to Toscano Pino for
reporting, thanks to Manuel Mausz for fixing it
Doxygen 1.8.8 is preferred and the configfile was updated to this version.
The symbols of nickel (for the ni plugin) do not longer leak from the Elektra library. As
such, old versions of testmod_ni won't work with Elektra 0.8.13. A version-script is now
in use to only export following symbols:
• kdb*
• key*
• ks*
• libelektra* for module loading system
• elektra* for proposed and other functions (no ABI/API compatibility here!)
In this release, ENABLE_CXX11 was changed to ON by default.
Note that in the next release 0.8.14 there will be two changes:
• According to issue #262, we plan to remove the option ENABLE_CXX11 and require the
compiler to be C++11 compatible. If you have any system you are not able to build
Elektra with -DENABLE_CXX11=ON (which is the default for 0.8.13) please report that
immediately.
• the python3 bindings will be renamed to python
By not having to care for pre-C++11 compilers, we hope to attract more developers. The
core part is still in C99 so that Elektra can be used on systems where libc++ is not
available. Many new plugins are still written in C99, also with the purpose of not
depending on C++.
Python Plugins
A technical preview of python3 and python2 plugins has been added.
With them its possible to write any plugin with python scripts.
Note, they are a technical preview. They might have severe bugs and the API might change
in the future. Nevertheless, it is already possible to, e.g. develop storage plugins with
it.
They are not included in ALL plugins. To use it, you have to specify it:
-PLUGINS="ALL;python;python2"
Thanks to Manuel Mausz for to this work on the plugins and the patience in all the last
minute fixes!
Qt-gui 0.0.8
The GUI was improved and the most annoying bugs are fixed:
• only reload and write config files if something has changed
• use merging in a way that only a conflict free merge will be written, thanks to Felix
Berlakovich
• made sure keys can only be renamed if the new name/value/metadata is different from the
existing ones
• fixed 1) and 2) of #233
• fixed #235
• fixed qml warning when deleting key
• fixed qml typerror when accepting an edit
A big thanks to Raffael Pancheri!
KDB Tool
The commandline tool kdb also got some improvements. Most noteworthy is that kdb get -v
now gives a complete trace for every key that was tried. This is very handy if you have a
complex specification with many fallback and override links.
It also shows default values and warnings in the case of context-oriented features.
Furthermore:
• Add -v for setmeta
• Copy will warn when it won't overwrite another key (behaviour did not change)
• improve help text, thanks to Ian Donnelly
Documentation Initiative
As Michael Haberler from machinekit pointed out it was certainly not easy for someone to
get started with Elektra. With the documentation initiative we are going to change that.
• The discussion in github issues should clarify many things
• We start writing man pages in ronn-format(7), thanks to Ian Donnelly for current work
• Kurt Micheli is woring on improved doxygen docu + pdf generation
• Daniel Bugl already restructed the main page
• Daniel Bugl also improved formatting
• doc: use
Return values:
more,thanks to Pino Toscano
• doxygen: fix template to use @ and not \\.
• SVG logo is preferred, thanks to Daniel Bugl
• doc: use
Return values:
more,thanks to Pino Toscano
• many typo fixes, thanks to Pino Toscano
• fix broken links, thanks to Manuel Mausz, Daniel Bugl and Michael Haberler
Any further help is very welcome! This call is especially addressed to beginners in
Elektra because they obviously know best which documentation is lacking and what they
would need.
Portability
kdb-full and kdb-static work fine now for Windows 64bit, thanks to Manuel Mausz. The
wresolver is now more relaxed with unset environment.
All issues for Mac OS X were resolved. With the exception of elektrify-getenv everything
should work now, thanks to Mihael Pranjic:
• fix mktemp
• testscripts
• recursive mutex simplification
• clearenv ifdef
and thanks to Daniel Bugl:
• RPATH fixed, so that kdb works
furthermore:
• fix __FUNCTION__ to __func__ (C99), thanks to Pino Toscano
• avoid compilation error when JNI_VERSION_1_8 is missing
• fix (twice, because of an accidental revert) the TARGET_CMAKE_FOLDER, thanks to Pino
Toscano
Thanks to Manuel Mausz for to testing and improving portability!
Packaging and Build System
• 0.8.12 packaged+migrated to testing, thanks to Pino Toscano
• fix build with external gtest, thanks to Pino Toscano
• switch from FindElektra.cmake to ElektraConfig.cmake, thanks to Pino Toscano
• use cmake_parse_arguments instead of parse_arguments, thanks to Manuel Mausz
Further Fixes
• Key::release() will also work when Key holds a null-pointer
• Key::getName() avoids std::string exception
• support for copy module was introduced, thanks to Manuel Mausz
• be more POSIX compatible in shell scripts (type to command -v and avoid echo -e) thanks
to Pino Toscano
• fix vararg type for KEY_FLAGS, thanks to Pino Toscano
• fix crash of example, thanks to Pino Toscano
• add proper licence file for Modules (COPYING-CMAKE-SCRIPTS), thanks to Pino Toscano
• fix XDG resolver issue when no given path in XDG_CONFIG_DIRS is valid
• make dbus example work again
• fix compiler warnings for gcc and clang
• fix valgrind suppressions
• Installation of GI binding is fixed, thanks to Dāvis
• make uninstall is fixed and docu improved
Notes
There are some misconceptions about Elektra and semi structured data (like XML, JSON).
Elektra is a key/value storage, that internally represents everything with key and values.
Even though, Elektra can use XML and JSON files elegantly, there are limitations what XML
and JSON can represent. XML, e.g., cannot have holes within its structure, while this is
obviously easily possible with key/value. And JSON, e.g., cannot have non-array entries
within an array. This is a more general issue of that configuration files in general are
constrained in what they are able to express. The solution to this problem is validation,
i.e. keys that does not fit in the underlying format are rejected. Note there is no issue
the other way round: special characteristics of configuration files can always be captured
in Elektra's metadata.
Get It!
You can download the release from here and now also here on github
• name: elektra-0.8.13.tar.gz
• size: 2141758
• md5sum: 6e7640338f440e67aba91bd64b64f613
• sha1: ca58524d78e5d39a540a4db83ad527354524db5e
• sha256: f5c672ef9f7826023a577ca8643d0dcf20c3ad85720f36e39f98fe61ffe74637
This release tarball now is also available signed by me using gpg
already built API-Docu can be found here
Stay tuned!
Subscribe to the RSS feed to always get the release notifications.
For any questions and comments, please contact the Mailing List the issue tracker on
github or by mail elektra@markus-raab.org.
Permalink to this NEWS entry
For more information, see http://libelektra.org
Best regards, Markus
0.8.12 Release
• guid: 98770541-32a1-486a-98a1-d02f26afc81a
• author: Markus Raab
• pubDate: Sun, 12 Jul 2015 20:14:09 +0200
Again we managed to release with new features, many build system fixes and also other
improvements.
dir namespace
This namespace adds per-project or per-directory (hence the name) configurations. E.g.
think how git works: not only /etc and ~ are relevant sources for configuration but also
the nearest .git directory.
This technique is, however, much more widely useful than just for git repositories! Nearly
every application can benefit from such a per-dir configuration. Its almost certain that
you have already run into the problem that different projects have different guidelines
(e.g. coding conventions, languages, whitespace requirements, line breaks, ..). Obviously,
thats not a per-user configuration and its also not a per-file issue (thats how its
usually solved today). So in fact you want, e.g., your editor to have an additional per-
project layer to choose between such settings.
The technique is useful for nearly every other tool:
• different color palettes in gimp, inkscape,..
• different languages for libreoffice
• different security settings for media players, interpreters (e.g. when in Download
folder)
• per-folder .htaccess in apache or other web servers
• any other per-dir configuration you can imagine..
It is simple to use, also for the administrative side. First, change to the folder to your
folder (e.g. where a project is):
cd ~/projects/abc
Then add some user (or system or spec) configuration to have some default.
kdb set user/sw/editor/textwidth 72
Then verify that we get this value back when we do a cascading lookup:
kdb get /sw/editor/textwidth
The default configuration file for the dir-namespace is pwd/KDB_DB_DIR/filename:
kdb file dir/sw/editor/textwidth
• KDB_DB_DIR can be modified at compile-time and is .dir per default
• filename can be modified by mounting, see below, and is default.ecf by default
We assume, that the project abc has the policy to use textwidth 120, so we change the dir-
configuration:
kdb set dir/sw/editor/textwidth 120
Now we will get the value 120 in the folder ~/projects/abc and its subdirectories (!), but
everywhere else we still get 72:
kdb get /sw/editor/textwidth
Obviously, that does not only work with kdb, but with every elektrified tool.
mount files in dir namespaces
For cascading mountpoints, the dir name is also automatically mounted, e.g.:
kdb mount editor.ini /sw/editor ini
But its also possible to only mount for the namespace dir if no cascading mountpoint is
present already:
kdb mount app.ini dir/sw/app tcl
In both cases keys below dir/sw/editor would be in the INI file .dir/editor.ini and not in
the file .dir/default.ecf.
dir together with spec namespace
In the project P we had the following issue: We needed on a specific computer the
configuration in /etc to be used in favour of the dir config.
We could easily solve the problem using the specification:
kdb setmeta spec/sw/P/current/org/base override/#0 /sw/P/override/org/base
Hence, we could create system/sw/P/override/org/base which would be in favour of
dir/sw/P/current/org/base. So we get system/sw/P/override/org/base when we do:
kdb get /sw/P/current/org/base
Alternatively, one could also use the specification:
kdb setmeta spec/sw/P/current/org/base