MailerQ AmqpHttp

This is a tool to convert MailerQ output into the format understood by the httpcaller. It is typically used in a technology stack together with MailerQ and HttpCaller, where this tool is the middleware between these two applications.

MailerQ --> RabbitMQ --> AmqpHttp --> RabbitMQ --> HttpCaller --> Your REST API

AmqpHttp is responsible for converting MailerQ output into the format of HttpCaller. Both applications process JSON messages, but with different field names and a different structure. The input for httpcaller also requires some extra fields, like the URL that is to be called.

The application has the following features:

• It adds to each message the URL to which the HTTP POST call should be sent.
• It optionally adds a private key to have the calls signed.
• It can filter some fields from the call to hide information.

Installation

The application is stored in our APT repository. If you have already enabled our APT repository, you can install it with this command:

sudo apt install mailerq-amqphttp

Running

You typically run this program from the command line. It reads all its input from a config file stored in /etc/copernica/amqphttp.txt.

amqphttp

Configuration

All options can be supplied in the system wide config file, via environment variables and as command line options. The config file is stored in /etc/copernica/amqphttp.txt. Options from the config file can be overridden by passing them as command line arguments or environment variables. The "rabbitmq-address" setting, for example, can be provided in the following ways:

• the setting "rabbitmq-address" in the /etc/copernica/httpcaller.txt config file
• the command line argument "--rabbitmq-address"
• the environment variable "AMQPHTTP_RABBITMQ_ADDRESS"

We use "rabbitmq-address" as an example. All configuration settings can be supplied in a similar manner.

Supported options for RabbitMQ

The application reads out the MailerQ result queue from RabbitMQ, converts the messages, and publishes them back to a different queue, where they can be picked up by the httpcaller.

• rabbitmq-address: the address in amqp://user:password@hostname/vhost format.
• rabbitmq-queue: the name of the queue from which messages are read.
• rabbitmq-exchange: the exchange to which the converted messages are published.
• rabbitmq-routingkey: the name of the routing key for the converted messages.
• rabbitmq-qos: quality of service / how many messages to consume.

The queue name can also be formatted as "exchange:routingkey". If you use this format, all messages published to the specified exchange with the specified routing key are consumed.

Other options

• endpoint: the http://example.com url to which all calls will be sent.
• filter: optional filter to filter out specific fields.
• port: port number for the built-in metrics webserver.
• log: logfile location.
• config-file: alternative config file location.

Filtering messages

The "filter" option can be used to filter out certain fields from the input, for example because you do not want to send ALL data from the results queue to the HTTP endpoint. The filter option should be a space separated of JSON wildcard pointers with the fields that are filtered out from the messages.

filter:     results/*/recipient results/*/envelope

The above filter ensures that the recipient and envelope properties are filtered out of the results array. The filters are JSON pointers with added wildcard support. See https://datatracker.ietf.org/doc/html/rfc6901 for more information about JSON pointers.

See also

You normally want to setup a full RabbitMQ -> AmqpHttp -> HttpCaller stack. The mailerq-webhook can be used to set this all up.