search string side menu

BRIX On-Premises / BRIX On-Premises Enterprise / Data migration from embedded to external databases in BRIX Enterprise

Data migration from embedded to external databases in BRIX Enterprise

You can migrate data from databases within a Kubernetes cluster to external storages using:

The main steps of data migration by external tools:

  1. Prepare to data dump creation.
  2. Create a data dump.
  3. Restore data on external storage.
  4. Update BRIX Enterprise application parameters.

After migrating databases to external storages, system reactivation is required.

Step 1: Prepare to data dump creation

  1. First, install the jq package.
  2. To access the databases within the cluster, forward the service ports:

kubectl port-forward postgres-0 5432:5432 --address 0.0.0.0
kubectl port-forward mongo-0 27017:27017 --address 0.0.0.0

  1. Get and save passwords for connecting to the databases:

kubectl get secrets/elma365-db-connections -o json | jq '.data | map_values(@base64d)'

Step 2: Create a data dump

  1. Generate a data dump file for PostgreSQL using the pg_dump utility:

pg_dump -h localhost -p 5432 -U postgres elma365 -f /backup/<backup-postgresql-folder-name>

Enter the password obtained in the first step.

  1. Generate a data dump file for MongoDB using the mongodump utility:

mongodump --host localhost --port 27017 --db elma365 -u elma365 --out /backup/<backup-mongodb-folder-name>

Enter the password obtained in the first step.

Read more about these utilities in the External BRIX backup article.

Step 3: Restore data on external storage

  1. Restore from backup on empty databases and S3 storage. Clear them before restoration.
  2. Restore PostgreSQL data using the psql utility:

psql -h <external_db_host> -p <external_db_port> -U <username> -d <database_name> -f /backup/<backup-postgresql-folder-name>

Where:

  • <external_db_host>. The host of the external database.
  • <external_db_port>. The port of the external database.
  • <username>. The user name of the external database.
  • <database_name>. The name of the database on the external server.
  1. Restore MongoDB data using the mongorestore utility:

mongorestore --host <external_db_host> --port 27017 --db <database_name> -u <username> --dir=/backup/<backup-mongodb-folder-name>

Where:

  • <external_db_host>. The host of the external database.
  • <username>. The user name of the external database.
  • <database_name>. The name of the database on the external server.
  1. Restore the S3 storage data using the mc utility:

mc alias set <alias_source> <s3-endpoint_source> <access-key_source> <secret-key_source>
mc alias set <alias_destination> <s3-endpoint_destination> <access-key_destination> <secret-key_destination>

Where:

  • <alias_source>. The short name (alias) for the source.
  • <s3-endpoint_source>. The address of the S3-compatible source storage.
  • <access-key_source>. The access key for the source.
  • <secret-key_source>. The secret key for the source.
  • <alias_destination>. The short name (alias) for the destination.
  • <s3-endpoint_destination>. The address of the S3-compatible destination storage.
  • <access-key_destination>. The access key for the destination.
  • <secret-key_destination>. The secret key for the destination.

Then copy the S3 storage from the source to the destination:

mc cp --recursive <s3-endpoint_source>/bucket-name/ <s3-endpoint_destination>/bucket-name/

Where:

  • <alias_source>. The short name (alias) for the source.
  • <alias_destination>. The short name (alias) for the destination.
  • bucket-name. The bucket name.
  • --recursive. Recursive copying of all objects and files.
  1. After restoring the databases on external storage, reactivate the system.

Step 4: Update BRIX Enterprise application parameters

  1. After restoring data on external databases, make sure that all tables for PostgreSQL or collections for MongoDB are migrated correctly.
  2. To configure the BRIX Enterprise application to use external databases, change the database and S3 file storage connection settings in the values-elma365.yaml configuration file.
  3. Update the BRIX application parameters using the values-elma365.yaml configuration file.

Updating parameters via the Internet

HideShow all

Updating parameters without the Internet connection

HideShow all

  1. Restart all services with the command:

kubectl rollout restart deploy [-n namespace]

  1. Verify the system is working correctly and stop old databases inside Kubernetes:

kubectl scale statefulset <db-type> --replicas=0 [-n namespace]

Where <db-type> is the name of the StatefulSet: postgres, mongo, or minio.

dump_db.html embedded-to-external-database-enterprise-backupper.html