M3AAWG general meeting
Copernica, the company behind MailerQ, visited the 36th general meeting of M3AAWG in February 2016. During this event we had the opportunity to meet some of our users and to get valuable feedback on our MailerQ product, as well as suggestions for future improvements.
In this article we give an overview of all the suggestions that were brought forward. Some of the suggestions have already been implemented and will be available soon in the upcoming MailerQ 4.0 release.
Spool directory for outgoing mail
Next to the current injection mechanisms (command line interface, the SMTP port or injecting messages directly into RabbitMQ), we received the suggestion to support spool directories as well. All mails that are dropped in such a spool directory should automatically be picked up, parsed and delivered. We consider this is a very good feature indeed, and it therefore has already been implemented and will be available in the upcoming MailerQ 4.0.
Possibility to drop or fail messages per domain
Some of our users want to be able to use the management console to mark a domain as being offline, so that all mails delivered to it are automatically dropped or failed. We will consider this for a future release.
MTA level send capacities
Currently, capacities can only be set on a per-domain basis. However, users also want to set capacities per MTA. This would allow users to set limits on the number of messages that get sent from a specific IP address, not matter to which IP or domain the messages are sent. This is a good suggestion and we're pleased to announce that it has already been implemented and will be available in the upcoming MailerQ 4.0 release.
Persistent paused deliveries
When deliveries are paused through the management console, these paused settings are only stored in main memory. This means that when MailerQ is restarted, the paused settings are gone and all deliveries are immediately resumed. From MailerQ 4.0 onwards the paused deliveries are going to be stored persistently, so that they survive a restart of MailerQ.
Command line utility or API
Not everyone likes the management console as much as we do. We've learned that some users like to be able to control MailerQ using command line tools or some kind of API. This is a good suggestion and will be available in one of our future versions.
Campaign tracking
We've heard that it would be greatly appreciated if statistics and filters could be applied on campaign identifiers. The input JSON should allow special properties (like "campaign" and/or "customer") and the management console should have tools that allows one to trace and control campaigns.
Log file improvements
The format of the log file should be fully configurable, so that MIME headers and JSON properties can be logged too. There should also be an option to create seperate log files for successes and for just the failures.
Although we originally designed MailerQ to be an application that does not do any disk I/O at all (to prevent possible blocking system calls), and we prefer RabbitMQ message queues for processing results instead of using log files, we do acknowledge that many users prefer the simplicity of such files over message queues. We will therefore improve the logging facilities in upcoming versions of the application.
Configuration in regular files
Most of the MailerQ configuration is stored in a relational database. However, some of our users prefer a configuration that is fully file based so that it is easier to make changes by hand. This has been noted and we will monitor whether more users would like this.
We prefer the database driven configuration, as it makes it easier to use the same configuration for many different MailerQ instances, and it makes it easier to read and modify the configuration from out of other applications.
Built-in sink
MailerQ can be started with a "smtp-sink" option, so that all outgoing mails are sent to a sink instead of to the actual recipient. This however requires an external SMTP server to run where these messages can be delivered to. It would be nice to have an option to have a built-in sink.
Overridable DNS lookups
A suggestion was received to make it possible to override DNS lookups via the management console. This would allow mails to be sent to special IP addresses instead of to the addresses that are advertised via DNS.
NoSQL improvements
Support for RIAK would be appreciated, as well as support for traditional directory based storage for messages instead of using a NoSQL engine. The upcoming MailerQ 4.0 release will indeed have a feature to use "dir:///path/to/dir" as a valid URL for a storage environment. Riak support will be implemented later on.
Multiple recipients per message
At Copernica we send out messages that are unique for each and every recipient. However, users who want to send out the same message to multiple recipients would like to include multiple recipients in the same JSON message, so that multiple "RCPT TO" commands are sent to a server, with only a single "DATA" message. This is not yet possible, but we acknowledge that MailerQ should indeed be able to send multiple "RCPT TO" instructions for a single message, and it should be able to track individual failures and deliveries for such messages.
Searching and delays
Queue delays should be made visible, and the option to search for messages inside the send queue.
Alternatives for RabbitMQ
MailerQ is tightly connected with RabbitMQ, using a couple of features that are specific to RabbitMQ. However, not everyone is as big a fan of RabbitMQ as we are, and the suggestion came in to use Apache Kafka as an alternative message queue system.
Because Apache Kafka is not fully compatible with RabbitMQ, this is not going to be an easy feature to implement, so we will monitor whether these suggestions come in more often.
Next meetings
The next M3AAWG meeting will be held in June in Philadelphia. Copernica is going to attend that meeting too, and we will be present at the next Certified Senders Alliance Summit in April in Germany. We hope to meet you there too.
