Deployment to various CD/CD environments with alterating default user and password

[v-bulynkin]

Hello, please suggest me the sensible way to achieve the following:

There are 3 environments in Gitlab - DEV, STAGE and PROD. I have a definitions.json file where a lot of queues and bindings are set.
I know that $RABBITMQ_DEFAULT_USER and $RABBITMQ_DEFAULT_PASS variables are ignored when definitions.json is used and templating the definitons.json file is not supported.
However, I need to use different default user and password in each environment.

The obvious way is duplicating definitions.json to various Gitlab CI/CD file variables and mounting those files as secrets in appropriate containers. However, it's uncomfortable to edit something in definitions.json, moreover, it should be done several times if I need to add another queue.

Is there any way to use environment variables in this case?
Or perhaps split definitions.json into 2 parts - one will be in the project and contain queues and similar stuff and the second in Gitlab CI/CD file variables and contains only users/permissions/policies?
Maybe, there is another option I overlooked?

[lukebakken]

Is there any way to use environment variables in this case?

Yes, you can use environment variables in the rabbitmq.conf file: https://www.rabbitmq.com/configure.html#env-variable-interpolation

Let me know if that works for you.

[v-bulynkin]

Thank you! What to do with "users" and "permissions" sections in definitions.json? Simply remove them?

I tried the following:

definitions.json, a part (only vhosts, queues,exchanges and bindings are presented)

  "vhosts":[
    {"name": "/"}
  ],
  "queues": [
    {
      "name": "lk.catalog_template",
      "vhost": "/",
      "durable": true,
      "auto_delete": false,
      "arguments": {
        "x-ha-policy": "all"
      }
    },

rabbitmq.conf:

default_user = $(RABBITMQ_DEFAULT_USER)
default_pass = $(RABBITMQ_DEFAULT_PASS)
load_definitions = /etc/rabbitmq/definitions.json

Dockerfile:

FROM rabbitmq:management-alpine
COPY --chown=rabbitmq:rabbitmq rabbitmq.conf /etc/rabbitmq/
COPY --chown=rabbitmq:rabbitmq definitions.json /etc/rabbitmq/

docker-compose.yml ($RABBITMQ_USER and $RABBITMQ_PASS are in the Gitlab CI variables)

version: "3.9"
services:
  rabbitmq:
    image: ${IMAGE_NAME}:${VERSION}
    hostname: rabbitmq
    environment:
      RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASS}
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "-q", "ping"]
    ports:
      - ${RABBITMQ_PORT}:5672
      - ${RABBITMQ_ADMIN_PORT}:15672
    volumes:
      - ${RABBITMQ_STORAGE}:/var/lib/rabbitmq
    networks:
      - rabbitmq

networks:
  rabbitmq:
    external: true
    name: ${NETWORK}

And no luck:

2023-08-10 07:05:49.245505+00:00 [warning] <0.932.0> HTTP access denied: user 'admin' - invalid credentials

[lukebakken]

Great, thank you for the information. I will try to find time to look at this soon. I'm busy working on issues for RabbitMQ customers who have support contracts.

[lukebakken]

OK, when you are booting a fresh node, the definitions file import will cause RabbitMQ to ignore the default_user and default_pass settings:

https://www.rabbitmq.com/definitions.html#import-on-boot-nuances

Or perhaps split definitions.json into 2 parts - one will be in the project and contain queues and similar stuff and the second in Gitlab CI/CD file variables and contains only users/permissions/policies?

This is a good solution. Please see this repository for a full working example:

https://github.com/lukebakken/docker-library_rabbitmq-659

Running this command will start RabbitMQ with the admin-DEV user:

make BUILD_ENV=DEV up

...while this will create admin-PROD:

make BUILD_ENV=PROD up

Everything you need should be in the Makefile and other files in the project. Note that RabbitMQ supports setting load_definitions to a directory, and will import files in order. This is why I use the 10- and 20- prefixes, to ensure that the vhost is created prior to adding users.

Also note that rabbitmqctl hash_password exists to create hashes of strings to use as a password.

[v-bulynkin]

Thank you very much! You helped me to figure out a solution which I implemented. Here it is.

First of all, hash itself is not comfortable to deal with, so I put username and password in plain text to environment variables.

definitions.json is only one and there are keywords to replace:

  "users": [
    {
      "name": "RABBITMQ_USER",
      "password_hash": "RABBITMQ_PASS_HASH",
      "hashing_algorithm": "rabbit_password_hashing_sha256",
      "tags": "administrator",
      "limits": {}
    }
  ],
  "vhosts": [
    {
      "name": "/"
    }
  ],
  "permissions": [
    {
      "user": "RABBITMQ_USER",
      "vhost": "/",
      "configure": ".*",
      "write": ".*",
      "read": ".*"
    }
  ],
  "topic_permissions": [],
  "parameters": [],
  "global_parameters": [],
  "policies": [],

.gitlab-ci.yml (from build stage extend), where plain password transforms to hash and then substitutes the keyword in definitions.json:

  script:
    - RABBITMQ_PASS_HASH=`docker run --rm rabbitmq:management-alpine rabbitmqctl hash_password $RABBITMQ_PASS |grep -v $RABBITMQ_PASS`
    - sed -i "s|RABBITMQ_USER|$RABBITMQ_USER|g" definitions.json
    - sed -i "s|RABBITMQ_PASS_HASH|$RABBITMQ_PASS_HASH|" definitions.json
    - docker build -t $IMAGE_NAME:$VERSION .
    - docker push $IMAGE_NAME:$VERSION

sed is using pipes as delimiters to avoid errors with characters in hash:

sed: -e expression #1, char 30: unknown option to `s'