All posts by Ian Donnelly

New Release: Elektra 0.8.8

Hi Everybody!

Great news! I am very happy to announce that we have reached a new milestone for Elektra and released a new version, 0.8.8! This release comes right on the tail of the 0.8.7 release and it might just be our biggest release yet! We already have a great article covering all the changes from the previous release on our News documentation on GitHub. I just wanted to focus on a few of those changes on this blog, especially the ones that pertain to my Google Summer of Code Project.

First of all, Felix has worked to greatly improve the ini plug-in. This is the plug-in I used in my technical demo for mounting Samba’s smb.conf file. It now works even better with complex ini files such as smb.conf which means the automatic merging of files like smb.conf is even better now! That really goes to show one of the greatest strengths of the design of Elektra. Just by improving plug-ins, all the functions of Elektra can improve as well. The merge code was not changed in this release, yet because of an updated plug-in, the merge has improved.

Secondly, there have been some good improvements to the kdb command-line tool. Many of these improvements were used in my technical demo, but now they are actually a part of release (and a little more refined from then). We added a new command called kdb remount which allows a user (or script) to mount a file to the Elektra Key Database using an existing backend. An example of this command is:
kdb remount [new filename] [new path] [existing mountpoint]

This command mounts the new file to the new path in the Key Database using an existing backend. This works with the conffile merging by allowing us to mount the various versions of the conffile without having to specify which backend to use (it will use the same backend as the currently used conffile). Additionally, the umount command was updated to allow users to umount using the current mountpath (allowing commands such as kdb umount system/smb.conf) as opposed to backends. Moreover, we added an option to the kdb import command to specify a merge strategy using thing -s option. Now you can import a file into the Key Database and merge the content of that file with the current Keys in the Database.

Thirdly, we added some new scripts to Elektra to help with the ucf integration. These scripts were used in my technical demo, but now they are part of the release. elektra-mount and elektra-umount are wrappers for the commands kdb mount and kdb umount respectively. They are designed to be used in debian package scripts and are adapted for easier use than the generic commands. For instance, running elektra-mount will check to see if a file is already mounted at that location in the Key Database. Similarly, elektra-umount will not produce an error if the file was already unmounted. This is because maintainer scripts can be run multiple times in a row and producing an error will stop dpkg even when it shouldn’t. Additionally, we added a script called elektra-merge which can be used as a method for ucf to merge configuration files. This script acts as a liaison between ucf and elektra allowing automatic merges to be done by ucf using Elektra’s merge features in a seamless manner. For information of how these scripts work, check out my tutorial on integrating elektra-merge into a debian package.

The last bit of news I would like to share is the great progress of the Debian package. Thanks to Pino Toscano, version 0.8.7-4 of Elektra is now available in the Debian testing repo! This is great news as we are now that much closer to replacing the outdated Elektra 0.7 versions that are currently the latest versions of Elektra in the stable repo. Once the 0.8.X versions of Elektra make it to stable it will be much easier for us to keep the latest versions of Elektra in Debian, and that’s key to allowing Elektra to help improve users lives.

You can download the release from:
http://www.markus-raab.org/ftp/elektra/releases/elektra-0.8.8.tar.gz

• size: 1644441
• md5sum: fe11c6704b0032bdde2d0c8fa5e1c7e3
• sha1: 16e43c63cd6d62b9fce82cb0a33288c390e39d12
• sha256: ae75873966f4b5b5300ef5e5de5816542af50f35809f602847136a8cb21104e2

And the API-Documentation can be found here:

http://doc.libelektra.org/api/0.8.8/html/

Hope you enjoy the new release!

Sincerely,
Ian S. Donnelly

Wrapping Up

Hi Everybody!

I have been keeping very busy on this blog the past few days with some very exciting updates! Everything is now in place for my Google Summer of Code project! Elektra now has the ability to merge KeySets, ucf has been patched to allow custom merge commands, we included a new elektra-merge script to use in conjunction with the new ucf command, and we wrote a great tutorial on how to use Elektra to merge configuration files in any package using ucf. There are few things I have missed updating you all on or are still wrapping up.

First of all, I would like to address the Debian packages. While Elektra includes everything needed for my Google Summer of Code Project, it must be built from source right now. Unfortunately, with the rapid development Elektra has seen the past few months, we did not pay enough attention to the Debian packages and they became dusty and riddles with bugs. Fortunately, we have a solution and his name is Pino Toscano. Pino has, very graciously, agreed to help us fix our Debian packages back into shape. If you wish to see the current progress of the packages, you can check his repo. Pino has already made fantastic progress creating the Debian packages. I will post here when packages are all fixed up and the latest versions of Elektra are into the Debian repo.

Some great news that we just received is that Elektra 0.8.7 has just been accepted into Debian unstable! This is huge progress for our team and means that users can now download our packages directly from Debian’s unstable repo and test out these new features. Obviously, the normal caveats apply for any packages in unstable but this is still an amazing piece of news and I would like to thank Pino again for all the support he has provided.

Another worthy piece of news that I unfortunately haven’t had time to cover is that thanks to Felix Berlakovich Elektra has a new plug-in called ini. In case you couldn’t guess, this new plug-in is used to mount ini files and the good news is that it is a vast improvement on our older simpleini plug-in. While it is still a work in progress, this new plug-in is much more powerful than simpleini. The new plug-in supports section and comments and works with many more ini files than the old plug-in. You may have noticed that I used this new plug-in to mount smb.conf in my technical demo of Samba using Elektra from configuration merging. Since smb.conf follows the same syntax as an ini file this new plug-in works great for mounting this file into the Elektra Key Database.

I hope my blogs so far have been very informative and helpful.

Sincerely,
Ian S. Donnelly

Technical Demo

Hi Everybody,

Today I wanted to talk a bit about our technical demo. We patched a version of Samba to use our elektra-merge script in order to handle it’s configuration file smb.conf. Using the steps from my previous tutorial, we patched Samba to use this new technique of config merging. This patched version of Samba mounts it’s configuration to system/samba/smb in the Elektra Key Database. Then, during package upgrades, it uses the new --threeway-merge-command command with elektra-merge as the specified command. The result is automatic handling of smb.conf that is conffile-like (thanks ucf!) and the ability to have a powerful, automatic, three-way merge solution on package upgrades.

The main thing I would like to discuss is how this project is an improvement upon the current implementation of three-way merges in ucf. Before this project, ucf could attempt three-way merges on files it controlled using the diff3 tool. The main limitation of tools like diff3 are that they are line based and don’t inherently understand the files they are dealing with. Elektra on the other hand allows for a powerful system of backends which use plug-ins to understand configuration files. Elektra doesn’t store configuration data on a line-by-line basis, but in a more abstract way that is tailored to each configuration file using backends. smb.conf is a great example of this because it uses the syntax of an ini file so Elektra can mount it in a way that is intuitive to an ini file. Since data is stored in a format of key=data within ini files, Elektra stores this data in a similar way. For each key in smb.conf store a Key in Elektra with the value of that key store in a string. Then, during a merge, we can compare Keys in each version of smb.conf and easily see which ones changed and how they need to be merged into the result. On the other hand, diff3 has no concept of ini files or keys, it just compares the different versions line by line which results in many more conflicts than using elektra-merge. Moreover, a traditional weakness of diff is moving lines or data around. While diff3 does a very good job at handling this, it’s not perfect. In Elektra, Keys are named in an intelligent way based on their backend, so for smb.conf the line workgroup = HOME would always be saved under system/samba/smb/workgroup. It doesn’t matter if the lines are changed between versions because Elektra just has to check for the Key and its value.

My favorite example is a shortcoming in the diff3 algorithm (at least in my opinion). If something is changed to the same value in ours and theirs, but they differ from base, diff3 reports a conflict. On the other hand elektra-merge can easily handle this problem. A simple example of this would be changing the max log size value in Samba. Here is that line in each version of smb.conf:
Base:
max log size = 1000
Ours:
max log size = 2000
Theirs:
max log size = 2000

Obviously, in the merged version, result, one would expect this line to be:
max log size = 2000

Let’s check the result from elektra-merge:
max log size = 2000

Great! How about diff3:
<<<<<<< smb.conf.base
max log size = 1000
=======
max log size = 2000
>>>>>>> smb.conf.theirs

Whoops! As I mentioned the diff3 algorithm can’t handle this type of change, it just results in a conflict. Note that smb.conf.base is just representative of the file used as base and that smb.conf.theirs is representative of the file used as theirs. The file names were changed for the sake of clarity.

There are many other examples of the benefits to storing configuration data in a database of Keys that can better conform to actual data as opposed to storing configuration parameters in files where they can only be compared on a line to line basis. With the help of storage plug-ins, Elektra can ‘understand’ the configurations stored in it’s Key Database. Since we store the data in a way that makes sense for configuration data, we can easily merge actual configuration data as opposed to just lines in a file. A good example of this is in-line comments. Many of our storage plug-ins understand the difference between comments and actual configuration data. So if a configuration file has an inline comment like so:
max log size = 10000 ; Controls the size of the log file (in KiB)
we can compare the actual Keys, value pairs between versions max log size = 10000 and deal with the comments separately.

As a result, if we have a base:
max log size = 1000 ; Size in KiB

Ours:
max log size = 10000 ; Size in KiB

Theirs:
max log size = 1000 ; Controls the size of the log file (in KiB)

The result using elektra-merge would be:
max log size = 10000 ; Controls the size of the log file (in KiB)

Obviously, this line would cause a conflict on any line-based merge algorithm such as diff3 or git. It is worth noting that the ability of elektra-merge is directly related to the quality of the storage plug-ins that Elektra relies on. elektra-merge only looks at the name, value, and any metadata affiliated with each key. As a result, using the line plug-in would result in a merge only as powerful as any other line-based merge. Yet by using the ini plug-in on an ini file we get a much more advanced merge like the one described above.

As you can tell, this new method offers clear advantages to the traditional method of using diff3 to handle configuration merges. Also, I hope this demo shows how easy it is to implement these great features into your Debian packages. The community can only benefit if maintainers take advantage of these great features. I am glad to say that my Google Summer of Code Project has been a success, even if we had to do a little change of plans. The ucf integration ended up working great and is really easy for maintainers to implement. Hope you enjoyed this demo and better understand the power of using Elektra.

Sincerely,
Ian S. Donnelly

How-To: Integrate elektra-merge Into a Debian Package

Hi Everybody,

So I already explained that we decided to go in a new direction and patch ucf to allow automatic configuration file merging with any custom command. Today I wanted to explain how to use ucf’s new `–three-way-merge-command` functionality in
conjunction with Elektra in order to ultilize Elektra’s powerful tools in order to allow automatic three-way merges of your package’s configuration during upgrades in a way that is more reliable than a diff3 merge. This guide assumes that you are fimilar with ucf already and are just trying to impliment the --three-way-merge-command option using Elektra.

The addition of the --three-way-merge-command option was a part of my Google Summer of Code Project. This option takes the form:

--three-way-merge-command command [New File] [Destination]

Where command is the command you would like to use for the merge. New File and Destination are the same as always.

We added a new script to Elektra called elektra-merge for use with this new option in ucf. This script acts as a liaison between ucf and Elektra, allowing a regular ucf command to run a kdb merge even though ucf commands only pass New File and Destination whereas kdb merge requires ourpath, theirpath, basepath, and resultpath. Since ucf already performs a three-way merge, it keeps track of all the necessary files to do so, even though it only takes in New File and Destination.

In order to use elektra-merge, the current configuration file must be mounted to KDB to serve as ours in the merge. The script automatically mounts theirs, base, and result using the kdb remount command in order to use the same backend as ours (since all versions of the same file should use the same backend anyway) and this way users don’t need to worry about specifying the backend for each version of the file. Then the script attempts a merge on the newly mounted KeySets. Once this is finished, either on success or not, the script finishes by unmouting all but our copy of the file to cleanup KDB. Then, if the merge was successful ucf will replace ours with the result providing the package with an automatically merged configuration which will also be updated in KDB itself.

Additionally, we added two other scripts, elektra-mount and elektra-umount which act as simple wrappers for kdb mount and kdb umount. That work identically but are more script friendly

The full command to use elektra-merge to perform a three-way merge on a file managed by ucf is:

ucf --three-way --threeway-merge-command elektra-merge [New File] [Destination]

Thats it! As described above, elektra-merge is smart enough to run the whole merge off of the information from that command and utilizes the new kdb remount command to do so.

Integrating elektra-merge into a package that already uses ucf is very easy! In postinst you should have a line similar to:

ucf [New File] [Destination]

or perhaps:

ucf --three-way [New File] [Destination]

All you must do is in postinst, when run with the configure option you must mount the config file to Elektra:

kdb elektra-mount [New File] [Mounting Destination] [Backend]

Next, you must update the line containing ucf with the options --three-way and --threeway-merge-command like so:

ucf --three-way --threeway-merge-command elektra-merge [New File] [Destination]

Then, in your postrm script, during a purge, you must umount the config file before deleting it:

kdb elktra-umount [Name]

That’s it! With those small changes you can use Elektra to perform automatic three-way merges on any files that your package uses ucf to handle!

I just wanted to show a quick example, below is a diff representing the changes we made to the samba-common package in order to allow automatic configuration merging for smb.conf using Elektra. We chose this package because it already
uses ucf to handle smb.conf but it frequently requires users to manually merge changes across versions. Here is the patch showing what we changed:

diff samba_orig/samba-3.6.6/debian/samba-common.postinst samba/samba-3.6.6/debian/samba-common.postinst
92c92,93
< ucf --three-way --debconf-ok "$NEWFILE" "$CONFIG"
---
> kdb elektra-mount "$CONFIG" system/samba/smb ini
> ucf --three-way --threeway-merge-command elektra-merge --debconf-ok "$NEWFILE" "$CONFIG"
Only in samba/samba-3.6.6/debian/: samba-common.postinst~
diff samba_orig/samba-3.6.6/debian/samba-common.postrm samba/samba-3.6.6/debian/samba-common.postrm
4a5
> kdb elektra-umount system/samba/smb

As you can see, all we had to do was add the line to mount smb.conf during install, update the ucf command to include the new --threeway-merge-command option, and unmount system/samba/smb during a purge. It really is that easy!

Sincerely,
Ian S. Donnelly

The New Deal: ucf Integration

 

Hi Everybody,

A few days ago I posted an entry on this blog called dpkg Woes where I explained that due to a lack of response, we were abandoning our plan to patch dpkg for my Google Summer of Code project, and I explained that we had a new solution. Well today I would like to tell you about that solution. Instead of patching dpkg, which would take a long time and seemed like it would never make it upstream, we have added some new features to ucf which will allow my Google Summer of Code project to be realized.

If you don’t know, ucf, which stands for Update Configuration File, is a popular Debian package whose goal is to “preserve user changes to config files.” It is meant to act as an alternative to considering a configuration file a conffile on systems that use dpkg. Instead, package maintainers can use ucf to handle these files in a conffile-like way. Where conffiles must work on all systems, because they are shipped with the package, configuration files that use ucf can be handled by maintainer scripts and can vary between systems. ucf exists as a script that allows conffile-like handling of non-conffile configuration files and allows much more flexibility than dpkg’s conffile system. In fact, ucf even includes an option to perform a three-way merge on files it manages, it currently only uses diff3 for the task though.

As you can see, ucf has a goal that while different than ours, seems naturally compatible to our goal of automatic conffile merging. Obviously, since ucf is a different tool than dpkg we had to re-think how we were going to integrate with ucf. Luckily, integration with ucf proved to be much more simple than integration with dpkg. All we had to do to integrate with ucf was to add a generic hook to attempt a three way merge using any tool created for the task such as Elektra and kdb merge. Felix submitted a pull request with the exact code almost a week ago and we have talked with Manoj Srivastava, the developer for ucf, and he seemed to really like the idea. The only changes we made are to add an option for a three-way merge command, and if one is present, the merge is attempted using the specified command. It’s all pretty simple really.

Since we decided to include a generic hook for a three-way merge command instead of an Elektra-specific one (which would be less open and would create a dependency on Elektra), we also had to add functionality to Elektra to work with this hook. We ended up writing a new script, called elektra-merge which is now included in our repository. All this script does is act as a liaison between the ucf --three-way-merge-command option and Elektra itself. The script automatically mounts the correct files for theirs and base and dest using the new remount command.

Since the only parameters that are passed to the ucf merge command are the paths of ours, theirs, base and result, we were missing vital information on how to mount these files. Our solution was to create the remount command which mirrors the backend configuration of an existing mountpoint to create a new mountpoint using a new file. So if ours is mounted to system/ours using ini, kdb remount /etc/theirs system/theirs system/ours will mount /etc/theirs to system/theirs using the same backend as ours. Since theirs, base, and result should all have the same backend as ours, we can use remount to mount these files even if all we know is their paths.

Now, package maintainers can edit their scripts to utilize this new feature. If they want, package maintainers can specify a command to use to merge files using ucf during package upgrades. I will soon be posting a tutorial about how to integrate this feature into a package and how to use Elektra in your scripts in order to allow for automatic three-way merges during package upgrade. I will post a link to the tutorial here once it is published.

Sincerely,
Ian S. Donnelly

How-To: kdb import

Hi everybody,

Today I wanted to go over what I think is a very useful command in the kdb tool, kdb import. As you know, the kdb tool allows users to interact with the Elektra Key Database (KDB) via the command line. Today I would like to explain the import function of kdb.

The command to use kdb import is:

kdb import [options] destination [format]

In this command, destination is where the imported Keys should be stored below. For instance, kdb import system/imported would store all the keys below system/imported. This command takes Keys from stdin to store them into KDB. Typically, this command is used with a pipe to read in the Keys from a file.

The format argument you see above can be a very powerful option to use with kdb import. The format argument allows a user to specify which plug-in is used to import the Keys into the Key Database. The user can specify any storage plug-in to serve as the format for the Keys to be imported. For instance, if a user wanted to import a /etc/hosts file into KDB without mounting it, they could use the command cat /etc/hosts | kdb import system/hosts hosts. This command would essentially copy the current hosts file into KDB, like mounting it. Unlike mounting it, changes to the Keys would not be reflected in the hosts file and vise versa.

If no format is specified, the format dump will be used instead. The dump format is the standard way of expressing Keys and all their relevant information. This format is intended to be used only within Elektra. The dump format is a good means of backing up Keys from the Key Database for use with Elektra later such as reimporting them later. As of this writing, dump is the only way to fully preserve all parts of the KeySet.

It is very important to note that the dump does not rename keys by design. If a user exports a KeySet using dump using a command such as kdb export system/backup > backup.ecf, they can only import that keyset back into system/backup using a command like cat backup.ecf | kdb import system/backup.

The kdb import command only takes one special option:

-s --strategy

which is used to specify a strategy to use if Keys already exist in the specified destination.
The current list of strategies are:

preserve any keys already in the destination will not be overwritten
overwrite any keys already in the destination will be overwritten if a new key has the same name
cut all keys already in the destination will be removed, then new keys will be imported

If no strategy is specified, the command defaults to the preserve strategy as to not be destructive to any previous keys.

An example of using the kdb import is as follows:

cat backup.ecf | kdb import system/restore

This command would import all keys stored in the file backup.ecf into the Key Database under system/restore.

In this example, backup.ecf was exported from the KeySet using the dump format by using the command:
kdb export system/backup > backup.ecf

backup.ecf contains all the information about the keys below system/backup:

$cat backup.ecf
kdbOpen 1
ksNew 3
keyNew 19 0
system/backup/key1
keyMeta 7 1
binary
keyEnd
keyNew 19 0
system/backup/key2
keyMeta 7 1
binary
keyEnd
keyNew 19 0
system/backup/key3
keyMeta 7 1
binary
keyEnd
ksEnd

Before the import command, system/backup does not exists and no keys are contained there.
After the import command, running the command kdb ls system/backup prints:

system/backup/key1
system/backup/key2
system/backup/key3

As you can see, the kdb import command is a very useful tool included as part of Elektra. I also wrote a tutorial on the kdb export command. Please go read that as well because those two commands go hand in hand and allow some very powerful usage of Elektra.

Sincerely,
Ian S. Donnelly

How-To: kdb export

Hi everybody,

I recently posted a tutorial on the kdb import command. Well I also wanted to go over it’s sibling function, kdb export. These two commands work very similarly, but there are some differences.

First of all, the command to use kdb export is:

kdb export [options] source [format]

In this command, source is the root key of which Keys should be exported. For instance, kdb export system/export would export all the keys below system/export. Additionally, this command exports keys under the system/elektra directory by default. It does this so that information about the keys stored under this directory will be included if the Keys are later imported into an Elektra Key Database. This command export Keys to stdout to store them into the Elektra Key Database. Typically, the export command is used with redirection to write the Keys to a file.

As we discussed already, the format argument can be a very powerful option to use with kdb export. Just like with kdb import the format argument allows a user to specify which plug-in is used to export the Keys from the Key Database. The user can specify any storage plug-in to serve as the format for the exported Keys. For instance, if a user mounted their hosts file to system/hosts using kdb mount /etc/hosts system/hosts hosts they would be able to export these Keys using the hosts format by using the command kdb export system/hosts hosts > hosts.ecf. This command would essentially create a backup of their current /etc/hosts file in a valid format for /etc/hosts.

If no format is specified, the format dump will be used instead. The dump format is the standard way of expressing Keys and all their relevant information. This format is intended to be used only within Elektra. The dump format is a good means of backing up Keys from the Key Database for use with Elektra later such as reimporting them later. As of this writing, dump is the only way to fully preserve all parts of the KeySet.

The kdb export command takes one special option, but it’s different than the one for kdb import, that option is:

-E --without-elektra which omits the system/elektra directory of keys

An example of using the kdb export is as follows:

kdb export system/backup > backup.ecf

This command would export all keys stored under system/backup, along with relevant Keys in system/elektra, into a file called backup.ecf.

As you can see, the kdb export command is a very useful tool just like it’s sibling, kdb import. If you haven’t yet, please go read the tutorial I wrote for kdb import because these two commands are best used together and can enable some really great features of Elektra.

Sincerely,
Ian S. Donnelly

dpkg Woes

Hi Everybody,

The original plan for my Google Summer of Code project involved creating a merge tool for Elektra including a kdb merge command, that part is all finished and integrated into the newest release of Elektra, 0.8.7 already. The next step was to patch dpkg to add an option for conffile merging. The basic overview of how this would work is:

  • Original versions of conffiles would be saved somewhere on the system to serve as a base file
  • The users current version of the conffile would serve as ours
  • The package maintainers version would be used as theirs
  • The result of a three-way merge would overwrite the user’s current version and be saved as a new base
  • Dpkg would need an option to perform a three way merge with a hook that allowed any tool to be used for the task, not just Elektra.

Obviously, all of these things require patching dpkg, although not in too large away. Luckily, we found a previous attempt to patch dpkg for a three-way merge hook, however it is a few years old and was never included in dpkg. Felix decided to update this patch to work with the new versions of dpkg and cleaned up a bit of redundancy, he then resubmitted it to see if the dpkg maintainers would be interested in such a patch, and to start a dialogue about including all the patches we need. Unfortunately there has been no response from the dpkg team, any many bugs on their tracker have been sitting unanswered for years. Additionally, dpkg’s source can be quite complex and it would require a ton of effort for us to make all these patches with no guarantee of integration. As a result, we have decided to go another route (which I will be posting about soon).

I just wanted to update anybody interested on the progress of my Google Summer of Code Project by letting them know the experience with dpkg. We have come up with a new solution that is already looking great and you will hear a lot more about it this final week of Google Summer of Code.

Stay Tuned!
- Ian S. Donnelly

How-To: kdb merge

Hi Everybody,

As you may know, part of my Google Summer of Code project involved the creation of merge tools for Elektra. The one I am going to focus on today is kdb merge. The kdb tool allows users to access and perform functions on the Elektra Key Database from the command line. We added a new command to this very useful tool, the merge command. This command allows a user to perform a three-way merge of KeySets from the kdb tool.
The command to use this tool is:
kdb merge [options] ourpath theirpath basepath resultpath

The standard naming scheme for a three-way merge consists of ours, theirs, and base:

  • ours refers to the local copy of a file
  • theirs refers to a remote copy
  • base refers to their common anscestor.

This works very similarly for KeySets, especially ones that consist of mounted conffiles.

For mounted conffiles:

  • ours should be the user’s copy
  • theirs would be the maintainers copy
  • base would be the previous version of the maintainer’s copy

If the user is just trying to accomplish is a three-way merge using any two arbitrary keysets that share a base,
it doesn’t matter which ones are defined as ours or theirs as long as they use the correct base KeySet.
In kdb merge, ourpath, theirpath, and basepath work just like ours, theirs, and base except each one represents the
root of a KeySet. Resultpath is pretty self-explanatory, it is just where you want the result of the merge to be saved under.
It’s worth noting, resultpath should be empty before attempting a merge, otherwise there can be unintended consquences.

As for the options, there are a few basic options:

-i --interactive which attempts the merge in an interactive way
-t --test which tests the proposed merge and informs you about possible conflicts
-f --force which overwrites any Keys in resultpath

Additionally there is an option to specify a merge strategy, which is very important.

The other option, strategy is:
s --strategy which is used to specify a strategy to use in case of a conflict

The current list of strategies are:
preserve the merge will fail if a conflict is detected
ours the merge will use our version during a conflict
theirs the merge will use their version during a conflict
base the merge will use the base version during a conflict

If no strategy is specified, the merge will default to the preserve strategy as to not risk making the wrong decision. If any of the other strategies are specified, when a conflict is detected, merge will use the Key specified by the strategy (ours, theirs, or base) for the resulting Key.

An example of using the kdb merge command:
kdb merge -s ours system/hosts/ours system/hosts/theirs system/hosts/base system/hosts/result

For more in depth examples, please see the full tutorial on our GitHub Documentation.

-Ian S. Donnelly

The Line Plug-In

Hi Everybody,

As you may have noticed I wrote a new plug-in for Elektra called “line“. I used it for a lot of examples in my tutorial, How-To: Write a Plug-In. The line plug-in is a very simple storage plug-in for Elektra. This plug-in stores files into the Elektra Key Database creating a new Key for each line and setting the string value of the Key to the string value of the line of that file. So if we have a file called “hello.txt”:

Hello
World!

And we mount it to kdb like so: kdb mount ~/line.txt system/hello_line line. The output of kdb ls system/hello_line the output would be:

system/hello_line/#1
system/hello_line/#
2

With the getString values of #1 and #2 being Hello and World! respectively. If this seems like a very simple plug-in, that’s because it is. Obviously, this plug-in isn’t a great showcase for the robustness of Elektra, any data structure could store a file line by line relatively easy, so why did we add a line plug-in at all?

The answer is that we included a line plug-in to allow any line-based file to use functions of Elektra, particularly the new Merge function. My Google Summer of Code project is to allow for automatic three-way merge of Debian conffiles during package upgrades as opposed to the current prompt and manual merging a user must do if a conffile is edited. Using Elektra and the new mergecode we can mount a conffile with the best plug-in for it, (the ini plug-in for Samba’s smb.conf for instance) and that allows for a very powerful merging ability with a lot more success than a simple diff merge. However, there are a lot of conffiles that don’t use any particular standard (such as ini, xml, or JSON) to store data. That is where the line plug-in comes in. We can still mount these files using the line plug-in and attempt a merge. Of course it is much more likely to have conflicts, and this type of merge is still susceptible to many of the same flaws as regular file merges (such as not being able to detect when a line has been moved), but it simple cases, the merge may succeed which would reduce the overall number of times a user would be prompted during an upgrade.

Basically, I wrote a line plug-in for Elektra as a fallback for conffile merges when we can’t mount the conffile in any more meaningful way. While merges using KeySets that were mounted using line are more likely to fail than other, more specialized plug-ins, there are cases that these merges will succeed and the user will not have to deal with a confusing prompt. The whole point of my Google Summer of Code project is to make upgrading packages and dealing with conffiles much smoother and easier than it is now by including a three-way mere and this line plug-in will help with this goal.

Sincerely,
Ian S. Donnelly