Message broker is a set of components that allow supporting message exchange among other system components. This functionality allows you to handle processing transactions in real-time, such as notification processing and importing. Also, message queues are used when you are starting a Scheduled Script.

Key Terms

Message broker – a system component, the main function of which is to keep and share messages among other system components.

Producer – a program or a system module sending a message into the Queue (producing a message).

Consumer – a program or a system module that receives a message from the Queue and processes it.

Queue – a part of a Message Broker; a list implemented on the FIFO basis (First In First Out). It is intended to keep messages of similar type and allocate them among responsible Consumers.

Exchange – a part of a Message Broker responsible for the message allocation among Queues.

Routing Key – a key that is needed for message allocation among Queues.

Binding – a connection between Exchanges or between the Exchange and the Queue.

Message Broker operating modes

Message broker can operate with instant and delayed messages. The difference is that delayed messages are sent after some time specified in the $delay parameter.

Instant message processing

Instant message processing consists of several stages:

1. Message sending
2. Message storage in queue
3. Message processing.

Message sending

In brief, the component (program or module) producing a message (a Producer) generates a message and enhances it with a service data (such as Routing Key, User ID, Sys_ID referenced to the User Token (user_token) table). Before sending, the message is serialized.

Sending example:

$this->producer->publish($message = 'message', $routingKey = 'echo');

Message storage in queue

Once messages are queued up, they should then be processed.

The queue listens to the specified Consumer state (it is based on the Routing Key). If the Consumer gets the message, then it happens. If the queue has no consumers, then the messages line up in the queue but do not disappear.

By default, every queue has at least one consumer. If a queue has more than one consumer bound, then it allocates messages among them.

Message processing

Consumer is implemented as a background process running in a separate Docker container. It runs a shell app within the container; after that, the message is processed by a back-end interface with a specific class, which is chosen based on the Routing Key specified in the message.

Before processing, a consumer should perform user (or impersonator) authorization, which is the message sending initiator; necessary authorization information can be found in every message.

Delayed messages processing

In general, delayed message processing is similar to instant message processing. The difference is that the parameter responsible for delay specifying is implemented, allowing this message to be sent with a delay specified.

Logging

All message activities are logged to the standard output (Stdout).

Environment settings

Message broker system consists of two containers:

1. Within the first container, the message broker is running externally and available via TCP. Also, a specific plugin is configured there to create delayed messages.
2. Within the second container, the supervisor daemon runs the required workers amount (can be set by an environment variable). If one of these processes fails, the supervisor starts the new one.