Message broker is a set of components allowing supporting message exchanging among another system components. This functionality allows handle processing transactions in real-time, such as notification processing, importing, also message queue can be used when you are starting a Scheduled Script.
Key Terms
Message broker - a system component the main function of which is to keep and share messages among other system components.
Producer - a program or a system module sending a message into a Queue (producing a message).
Consumer - a program or a system module receiving a message from a Queue and processing it.
Queue - a part of a Message Broker; a list implemented on the FIFO basis (First In First Out). It is intended to keep messages of similar type and allocate them among responsible Consumers.
Exchange - a part of a Message Broker responsible for the message allocation among Queues.
Routing Key - a key that is needed for message allocation among Queues.
Binding - a connection between Exchanges or between an Exchange and a Queue.
Message Broker operation modes
Message broker can operate with instant messages and delayed messages. The difference is that delayed messages are sent after some time specified in the $delay parameter.
Instant messages processing
Instant messages processing consists of several stages:
- Message sending
- Message storage in queue
- Message processing
Message sending
In brief, the component (program or module) producing a message (a Producer) generates a message and enhances it with a service data (such as Routing Key, User ID, Sys_ID referenced to the User Token (user_token) table). Before sending, the message is serialized.
Sending example:
$this->producer->publish($message = 'message', $routingKey = 'echo'); |
Message storage in queue
After messages are queued up, then they should be processed.
The queue listens to the specified Consumer state (it is based on the Routing Key). When the Consumer can get the message, then it happens. If the queue has no consumers, then the messages line up in queue but do not disappear.
By default, every queue has at least one consumer. If queue has more than one consumer bound, then it allocates messages among them.
Message processing
Consumer is implemented as a background process running in a separate Docker container. It runs a shell app within the container; after that, message is processed by a back-end interface with a specific classes which are chosen based on the Routing Key specified in the message.
Before processing, a consumer should perform user (or impersonator) authorization which were the message sending initiators; necessary authorization information can be found in every message.
Delayed messages processing
In general, delayed messages processing is similar to instant messages processing; the difference is the parameter responsible for delay specifying is implemented allowing that this message should be sent with a delay specified.
Logging
All message exchanging activities are logged and can be found in the Queue Messages (sys_queue_message) table.
Queue Message fields description
| Field | Description |
|---|---|
| Message | Content of the message sending. This field is mandatory. |
| Routing Key | Message routing key. This field is mandatory. |
| Scheduled start time | Represents date and time of suggested message processing start (based on the value specified in the delay parameter); otherwise, contains NULL. |
| Start time | Contains a date and time when the message processing starts. |
| End time | Contains a date and time when the message processing ends |
| State | Message processing state. Available options:
|
| Log Exception ID | This field displays the ID of the thrown log exception if there was one; otherwise, contains NULL. |
| Memory usage | In there, total memory amount (in bytes) for message processing is displayed. |
Environment settings
Message broker system consists of two containers.
Within the first container, the message broker is running externally available via TCP. . Also, the specific plugin is configured there for creating delayed messages.
Within the second container, the supervisor daemon runs the required workers amount (can be set by an environment variable). In the case of one of this processes fails, the supervisor starts the new one.