search string side menu

RabbitMQ cluster

This article describes how to install RabbitMQ 3.12.0 and Erlang 25.3.2.2-1 for Ubuntu Linux 20.04 and 22.04.

Installing RabbitMQ version 4.0 is not supported in the current BRIX version. Check the supported versions of RabbitMQ in the System requirements for BRIX On-Premises article.

You can also refer to the guide in the official RabbitMQ documentation

Installation consists of six steps:

  1. Prepare nodes (servers).
  2. Install RabbitMQ.
  3. Prepare RabbitMQ cluster.
  4. Configure RabbitMQ.
  5. Configure HAproxy rabbitmq block).
  6. Connect to RabbitMQ.

Step 1: Prepare nodes (servers)

начало внимание

The minimum number of servers to organize a cluster is three.

конец внимание

  1. Create three nodes (servers) with sequentially numbered host names:
  • rabbitmq-server1.your_domain;
  • rabbitmq-server2.your_domain;
  • rabbitmq-server3.your_domain.
  1. Create the necessary host name mappings in DNS. If this is not possible, add the required entries to /etc/hosts.

Step 2: Install RabbitMQ

  1. Install the necessary packages:

sudo apt-get install curl gnupg apt-transport-https -y

  1. Import all necessary keys:

curl -1sLf "https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA" | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null
curl -1sLf https://ppa1.novemberain.com/gpg.E495BB49CC4BBE5B.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg > /dev/null
curl -1sLf https://ppa1.novemberain.com/gpg.9F4587F226208342.key | sudo gpg --dearmor | sudo tee /usr/share/keyrings/rabbitmq.9F4587F226208342.gpg > /dev/null

  1. Add RabbitMQ repositories:

sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
deb [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-erlang/deb/ubuntu $(lsb_release -cs) main
deb-src [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-erlang/deb/ubuntu $(lsb_release -cs) main
 
deb [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-server/deb/ubuntu $(lsb_release -cs) main
deb-src [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa1.novemberain.com/rabbitmq/rabbitmq-server/deb/ubuntu $(lsb_release -cs) main
EOF

  1. Update the package cache:

sudo apt-get update -y

  1. Install Erlang packages:

sudo apt-get install -y erlang-base=1:25.3.2.2-1 erlang-asn1=1:25.3.2.2-1 erlang-crypto=1:25.3.2.2-1 erlang-eldap=1:25.3.2.2-1 erlang-ftp=1:25.3.2.2-1 erlang-inets=1:25.3.2.2-1 erlang-mnesia=1:25.3.2.2-1 erlang-os-mon=1:25.3.2.2-1 erlang-parsetools=1:25.3.2.2-1 erlang-public-key=1:25.3.2.2-1 erlang-runtime-tools=1:25.3.2.2-1 erlang-snmp=1:25.3.2.2-1 erlang-ssl=1:25.3.2.2-1 erlang-syntax-tools=1:25.3.2.2-1 erlang-tftp=1:25.3.2.2-1 erlang-tools=1:25.3.2.2-1 erlang-xmerl=1:25.3.2.2-1

  1. Install rabbitmq-server and its dependencies:

sudo apt-get install rabbitmq-server=3.12.0-1 -y --fix-missing

  1. Start rabbitmq-server:

sudo systemctl enable --now rabbitmq-server

Step 3: Prepare RabbitMQ cluster

  1. On each rabbitmq node, create the file /etc/rabbitmq/rabbitmq-env.conf and add the environment variables RABBITMQ_NODENAME and RABBITMQ_USE_LONGNAME.

Example content for rabbitmq-server1.your_domain:

RABBITMQ_NODENAME=rabbit@rabbitmq-server1.your_domain
RABBITMQ_USE_LONGNAME=true

Example content for rabbitmq-server2.your_domain:

RABBITMQ_NODENAME=rabbit@rabbitmq-server2.your_domain
RABBITMQ_USE_LONGNAME=true

Example content for rabbitmq-server3.your_domain:

RABBITMQ_NODENAME=rabbit@rabbitmq-server3.your_domain
RABBITMQ_USE_LONGNAME=true

  1. Copy Cookie /var/lib/rabbitmq/.erlang.cookie from the first node rabbitmq-server1.your_domain to all the other nodes in the cluster.

Начало внимание

For the RabbitMQ cluster to work, all nodes participating in the cluster must have the same content in the /var/lib/rabbitmq/.erlang.cookie file.

Конец внимание

  1. Restart the RabbitMQ service on each node:

sudo systemctl restart rabbitmq-server

  1. Stop the application on nodes rabbitmq-server2.your_domain and rabbitmq-server3.your_domain for subsequent joining to the cluster:

sudo rabbitmqctl stop_app

  1. Reset rabbitmq on nodes rabbitmq-server2.your_domain and rabbitmq-server3.your_domain:

sudo rabbitmqctl reset

  1. Join nodes rabbitmq-server2.your_domain and rabbitmq-server3.your_domain to the cluster:

sudo rabbitmqctl join_cluster rabbit@rabbitmq-server1.your_domain

  1. Start the application on nodes rabbitmq-server2.your_domain and rabbitmq-server3.your_domain:

sudo rabbitmqctl start_app

  1. Check the cluster status:

sudo rabbitmqctl cluster_status

Step 4: Configure RabbitMQ

начало примечание

Note

For the password, the following characters are allowed:

  • Uppercase Latin letters: A to Z
  • Lowercase Latin letters: a to z
  • Digits: 0 to 9
  • Symbols: -_

Reserved (invalid) symbols:

! * ' ( ) ; : @ & = + $ , / ? % # [ ]

конец примечание

  1. On each node, enable the necessary plugins:

sudo rabbitmq-plugins enable \
rabbitmq_management

  1. Create vhost nd give the user access to it, executing commands on the first node rabbitmq-server1.your_domain:

sudo rabbitmqctl add_vhost elma365vhost
sudo rabbitmqctl add_user elma365user SecretPassword
sudo rabbitmqctl set_permissions -p elma365vhost elma365user ".*" ".*" ".*"
sudo rabbitmqctl set_user_tags elma365user administrator

  1. Create a policy that allows mirroring queues for all nodes in the cluster, executing the command on the first node rabbitmq-server1.your_domain:

sudo rabbitmqctl set_policy -p 'elma365vhost' MirrorAllQueues ".*" '{"ha-mode":"all"}'

If the High Availability Mode policy is not supported, use the Quorum policy:

rabbitmqctl set_policy –p 'elma365vhost' QuorumDefault “^.*” ‘{“queue-type”:“quorum”}’ --priority 0 --apply-to queues

To  view the configured policies, use the command:

sudo rabbitmqctl list_policies --vhost elma365vhost

Enabling TLS/SSL in RabbitMQ

To enable TLS/SSL support in RabbitMQ, you need to edit the configuration file /etc/rabbitmq/rabbitmq.conf:

  1. Add SSL configuration to the rabbit (port 5671) and management (port 15671) blocks.
  1. In the ssl_options.certfile and management.ssl.certfile parameters, specify the path to the server certificate file.
  1. In the ssl_options.keyfile и management.ssl.keyfile parameters, specify the path to the private key file.
  1. In the ssl_options.cacertfile и management.ssl.cacertfile parameters, specify the path to the root CA file.

listeners.tcp = none
listeners.ssl.default = 5671
ssl_options.cacertfile = /path/to/ca_certificate.pem
ssl_options.certfile = /path/to/server_certificate.pem
ssl_options.keyfile = /path/to/server_key.pem
ssl_options.verify = verify_peer
ssl_options.fail_if_no_peer_cert = true
 
management.ssl.port = 15671
management.ssl.cacertfile = /path/to/ca_certificate.pem
management.ssl.certfile = /path/to/server_certificate.pem
management.ssl.keyfile = /path/to/server_key.pem

  1. Restart the RabbitMQ service:

sudo systemctl restart rabbitmq-server

For more details on configuring TLS/SSL in RabbitMQ, refer to the official RabbitMQ documentation.
 
For enabling TLS/SSL between RabbitMQ nodes "Securing Cluster (Inter-node)," you need to:

  1. Combine the public and private keys of the node into one file, for example, in combined_keys.pem:

cat server_certificate.pem server_key.pem > combined_keys.pem

  1. Extract the path to the Erlang TLS library and export it to the environment variable ERL_SSL_PATH в файл /etc/rabbitmq/rabbitmq-env.conf:

erl -noinput -eval 'io:format("ERL_SSL_PATH=~s~n", [filename:dirname(code:which(inet_tls_dist))])' -s init stop > /tmp/ssl-path.txt
cat /tmp/ssl-path.txt /etc/rabbitmq/rabbitmq-env.conf > /tmp/new-rabbitmq-env.conf
mv -f /tmp/new-rabbitmq-env.conf /etc/rabbitmq/rabbitmq-env.conf

  1. In the /etc/rabbitmq/rabbitmq-env.conf file, add parameters for the environment variables SERVER_ADDITIONAL_ERL_ARGS and RABBITMQ_CTL_ERL_ARGS:

SERVER_ADDITIONAL_ERL_ARGS="-pa $ERL_SSL_PATH \
  -proto_dist inet_tls \
  -ssl_dist_opt server_certfile /path/to/combined_keys.pem \
  -ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"
RABBITMQ_CTL_ERL_ARGS="-pa $ERL_SSL_PATH \
  -proto_dist inet_tls \
  -ssl_dist_opt server_certfile /path/to/combined_keys.pem \
  -ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"

  1. Restart the RabbitMQ service on each node:

sudo systemctl restart rabbitmq-server

For more details on configuring TLS/SSL between RabbitMQ nodes, refer to the official RabbitMQ documentation.

Step 5: Configure HAProxy (rabbitmq block)

Load balancing between RabbitMQ cluster nodes will be performed using HAProxy. Configure it according to Configure HAProxy for RabbitMQ.

Step 6: Connect to RabbitMQ

Connection string to connect to the RabbitMQ cluster (via HAProxy):

amqp://elma365user:SecretPassword@haproxy-server.your_domain:5672/elma365vhost

Connection string to connect to the RabbitMQ cluster with TLS/SSL (via HAProxy with TLS/SSL):

amqps://elma365user:SecretPassword@haproxy-server.your_domain:5671/elma365vhost

configure-redis.html minio-cluster.html