Good reasons to use dlopen

Inside our MailerQ C++ application we've frequently used calls to 'dlopen()' - instead of using shared of static linking. There are a couple of good reasons why we've done this, which I'll explain in this article.

Let's start with a quick reminder. If you write a program in C or C++ you often need (or want) to call functions from external libraries. There are three ways to use a library: with static linking, shared linking and by using the dlopen() system call. If you use static linking, a library is completely embedded in the final executable. This is a safe solution if you want to distribute your software in binary format only, because all the libraries that your program depends on, are already embedded and thus distributed with it.

However, most programs use shared linking instead of static linking. With shared linking, the library code is not embedded in the executable, but remains in a seperate .so file. This is not a problem, if the program only depends on very frequently used libraries (like OpenSSL). One can then still safely distribute the software in binary format, as it is extremely likely that the popular libraries that the software depends on will also be available on the target system. The executable will stay much smaller, because the library code is not embedded. An additional advantage of shared linking is that if multiple running programs all use the same shared library at the same time, they can share the same memory space for it – and it thus safes memory on the system as well.

There is however a problem if a program depends on a library that is not so popular. If one distributes a program that depends on a certain library, and that library turns out not to be installed on the target system, the program will fail to start. Or if the library is installed, but with a different version number, the program will fail to start too. For MailerQ we for example depend on the OpenSSL library, but at the same time we distribute a single executable program that is compatible with a whole range of Linux distributions: Ubuntu 10.04, Ubuntu 12.04, Fedora, Debian and CentOS. However, not every distribution has the same version of OpenSSL installed. As a consequence, MailerQ wouldn't start if we had used shared linking.

Static linking is not a good alternative for us either. The OpenSSL library is available on most systems, so it would be a waste of resources if we had linked it statically. But more importantly: the OpenSSL library is frequently updated. Every time that a security issue is fixed in OpenSSL, a new version gets installed. We would have to bring a new version of MailerQ out too if we had used static linking.

Using dlopen to overcome version incompatibilities

Luckily there is a third alternative to static and shared linking: calling dlopen(). With this system call it is possible to open a shared library and use the functions from it, without having to link with it. Your program just starts, and when it finds out that it needs to use a function from a specific library, it calls dlopen() to open that library. If the library is not available on the system, the function returns NULL and it is up to you, the programmer, to handle that. You can let the program gracefully die, try an alternative (older) version of the same library, or use a completely different library that offers the same sort of features. The downside of dlopen() calls is that it is a little slower, and that it adds complexity to your program as you suddenly need to manage pointer-to-functions.

We've used dlopen() for loading the methods from the OpenSSL library. When MailerQ starts, it tries to locate a version of the OpenSSL library that is known to be sufficient for MailerQ, and it uses the functions from that library. By using dlopen() in such a way we've managed to create a single executable file that can be installed on many different Linux distributions, and that uses the version of OpenSSL that is available.

Using dlopen for optional libraries

MailerQ supports many different database and nosql engines. To connect to one of these storage solutions, MailerQ uses functions from external C libraries. But the functions from such a library are of course only used if MailerQ actually connects to one of those storage solutions. If MailerQ has been configured to connect to MySQL, there is no need to use the functions from the PostgreSQL or Sqlite libraries. However, if MailerQ had been compiled with a shared link dependency on the PostgreSQL and Sqlite libraries, MailerQ would fail to start if these libraries weren't available - while it does not even need the functions from it!

This also is a situation where calling dlopen() brings relief. If you configure MailerQ to connect to a Sqlite database, MailerQ will call dlopen() on the Sqlite library, and will report an error if this library turns out to not be available. It does not care if the other database libraries are available, because they are not used after all.

We also use a call to dlopen for the "uuid" library. This is a library that can be used to generate unique string identifiers. However, when this library is not available, MailerQ is still perfectly capably of running, and uses its own algorithm for generating ID's. If we had used shared linking, MailerQ would have crashed if the uuid library was unavailable.

Using dlopen for legal reasons

The GNU Public License (GPL) is an open source license that allows you to distribute and copy software. However, this comes at a price: if you use GPL software inside your software, and when you also distribute your software, it forces you to distribute your software under the conditions of the GPL as well.

In other words, if you take a piece of GPL'ed software, and you embed it in your own program so that your software is a derived work of it, your own program automatically also becomes GPL software. This is of course true for static linking (because then you literally embed the software covered by the GPL and you really distribute the GPL'ed software), but the GPL also mentions that this can be true for shared linking. It does not explicitly mention dlopen - but we can assume that they consider the license to be covering calls to dlopen as well.

I have a different view. Before you even start reading through the terms of a license, you must first ask yourself if you're doing something that is normally forbidden by copyright law, and for which copyright law states that you need explicit permission from the copyright holder. If this is true, you need a license, and satisfy the requirements from the license (which often means paying money, or in the case of the GPL: to release your own software as GPL). However, if you're not infringing on someone's copyrights in the first place, there is no need to even consider looking at a license.

Distributing software with a dlopen instruction in it is not a copyright infringement. You're not distributing someone else's software, nor are you copying or modifying it. This means that you do not need permission from any alleged copyright holder, and you certainly do not have to pay him money or meet other requirements. The only thing that a license can do is give permission to do something that would normally be forbidden - and require a certain compensation for that. It doesn't work in the other way: it can not forbid something that was already allowed. The fact that the GPL mentions something about shared linking is therefore not important. A license can not pronounce itself applicable to situations that did not require a license in the first place.

Conclusion

Thus, using dlopen has many different advantages. It is easier to create one executable that runs on many different systems, and you keep the advantages of shared linking: shared memory, smaller program size and no need to bring out new versions of your program if one of the libraries gets updated. With dlopen you can fall back on an alternative technology if a certain library is missing, and you do not have to meet any license requirements because you're not doing anything that is protected by copyright law.