Comparing message injection methods: SMTP, AMQP, REST API & spool pick-up

There are various methods for submitting messages to your email delivery platform. The most popular and widely adopted is of course SMTP. However, injecting over an HTTP REST API is also gaining popularity as an alternative. In this post, we'll go over various injection methods and describe some of the pros and cons of each approach.

Method A: Simple Mail Transfer Protocol (SMTP)

Injecting your messages into your email delivery platform using SMTP is probably the most widely adopted injection method. Most if not all mail user agents (MUAs; the platform from which the message is initially constructed and sent out, such as email clients and marketing automation solutions) support message submission over SMTP. Similarly, most if not all email delivery platforms such as cloud-hosted SMTP relays and Mail Transfer Agents (MTAs) allow for SMTP injection.

Despite it being the most widely adopted injection method, there is a major downside to it. SMTP is a very chatty protocol. To send a message over a new connection between the sending MUA and the email delivery platform, a lot of communication goes back and forth such as for the initial connection greeting, authentication, the supported SMTP plugins, the from-address, recipient addresses, and MIME data. While you can send multiple messages over a single connection, this number is limited and if you're sending a mass mailing, the chattiness of SMTP can result in a lot of network overhead and thus delay. If your application or other MUA allows for a different injection method, that might be worth checking out.

MailerQ supports injection over SMTP. However because of the aforementioned reason, ideally we recommend using AMQP as the injection method. When injecting over SMTP, MailerQ will convert the incoming message to a JSON object and store it in the inbox queue in RabbitMQ.

Method B: Spool directory pick-up

Another long-standing message injection method is picking up messages from the spool directory. This is not so much supported by cloud-hosted SMTP relays, but mostly by MTAs. Injection using the spool directory is pretty straight forward: all you have to do is place a file containing all email headers and body in the respective directory. The MTA will continuously pick up messages from the spool directory and add them to the internal queue (either in memory or on disk).

While this process may seem very straightforward especially when you host your MTA within the same machine or network as your application or other MUA, there is one major downside. Writing messages to disks and reading from them is a very expensive I/O operation. Even when you have very fast SSDs (which can be very expensive), it will always be slower than accessing messages in RAM.

MailerQ supports injection through the spool directory. However because of the aforementioned reason, ideally we recommend using an injection method such as SMTP, or preferably AMQP. When injecting using the spool directory pick-up, MailerQ will convert the picked up message to a JSON object and store it in the inbox queue in RabbitMQ.

Method C: HTTP REST API

Besides SMTP, another injection method that is gaining popularity is message submission through a REST API. Because it's way less chatty than SMTP, the chances of running into higher than anticipated network latencies are much smaller. Most marketing automation platforms support message submission through REST API as well, as well as most of not all cloud-hosted SMTP relay platforms. Furthermore, some commercial enterprise-grade MTAs are supporting this message injection method more and more. In a hybrid setup that consists of either cloud-based relays and on-premise MTAs that support REST API injection or that consists of different on-premise MTAs that support REST API injection, this method would allow for easier and more flexible switching between these solutions. Furthermore, authentication and security methods for HTTP are arguable even more developed than for SMTP.

Probably the biggest downside of REST API injection, as is the case with the other injection methods, is that the messages are typically not injected into the message queue directly. While this extra step is not a slow operation like spool pick-up directory, it may still cause some overhead and at bigger volumes, that may impact performance.

MailerQ supports injection over REST API. However because of the aforementioned reason, ideally we recommend using AMQP as the injection method. When injecting over REST API, MailerQ will store the submitted JSON message objects in the inbox queue in RabbitMQ.

Method D: Advanced Message Queuing Protocol (AMQP)

Although not supported by most MTAs and other delivery platforms, for those that do support it (such as MTAs that rely on RabbitMQ or Kafka for message queueing) this sure is the fastest injection method. The main reason for this is that messages are directly inserted into the outbound mail queue, skipping any overhead caused by converting and queuing the message. AMQP is way less chatty than SMTP and also less than HTTP, whilst still supporting security and authentication methods.

We compared the injection speed using SMTP, spool directory pick-up and AMQP of 100.000 messages on a mid-end consumer laptop with 16GB DDR4 RAM, Intel i7 8th generation processor and SSD. The MUA, spool directory and RabbitMQ were all located on the same machine. For all methods, 100 processes were allocated, meaning that 1000 messages were injected per process. All outbound traffic was paused so bounces and network latency could not affect performance, messages were 75KB in size and the MTA used was MailerQ. The results are plotted and displayed below:

As expected, the spool directory pick-up is by far the slowest, taking more than twice as long as SMTP injection. We see that AMQP injection is the clear winner, beating SMTP injection by a whopping 13 seconds . After injecting all messages over AMQP, there are still more than 60.000 messages to be injected using SMTP and more than 87.000 using spool directory pick-up.

Interested in learning more about the various injection methods in MailerQ, or want to try it out yourself? Feel free to reach out to us at info@mailerq.com