side menu

Data migration from embedded to external databases in BRIX On-Premises

You can migrate data from databases within a Kubernetes cluster to external storages using:

The main steps of data migration by external tools:

  1. Prepare to data dump creation.
  2. Create a data dump.
  3. Restore data on external storage.
  4. Update BRIX On-Premises application parameters.

начало внимание

After migrating databases to external storages, system reactivation is required.

конец внимание

Step 1: Prepare to data dump creation

  1. First, install the jq package.
  2. To access the databases within the cluster, forward the service ports:

kubectl port-forward postgres-0 5432:5432 --address 0.0.0.0
kubectl port-forward mongo-0 27017:27017 --address 0.0.0.0

  1. Get and save passwords for connecting to the databases:

kubectl get secrets/brix365-db-connections -o json | jq '.data | map_values(@base64d)'

Step 2: Create a data dump

  1. Generate a data dump file for PostgreSQL using the pg_dump utility:

pg_dump -h localhost -p 5432 -U postgres brix365 -f /backup/<backup-postgresql-folder-name>

Enter the password obtained in the first step.

  1. Generate a data dump file for MongoDB using the mongodump utility:

mongodump --host localhost --port 27017 --db brix365 -u brix365 --out /backup/<backup-mongodb-folder-name>

Enter the password obtained in the first step.

Read more about these utilities in the External BRIX backup article.

Step 3: Restore data on external storage

  1. Restore from backup on empty databases and S3 storage. Clear them before restoration.
  2. Restore PostgreSQL data using the psql utility:

psql -h <external_db_host> -p <external_db_port> -U <username> -d <database_name> -f /backup/<backup-postgresql-folder-name>

Where:

  • <external_db_host>. The host of the external database.
  • <external_db_port>. The port of the external database.
  • <username>. The user name of the external database.
  • <database_name>. The name of the database on the external server.
  1. Restore MongoDB data using the mongorestore utility:

mongorestore --host <external_db_host> --port 27017 --db <database_name> -u <username> --dir=/backup/<backup-mongodb-folder-name>

Where:

  • <external_db_host>. The host of the external database.
  • <username>. The user name of the external database.
  • <database_name>. The name of the database on the external server.
  1. Restore the S3 storage data using the mc utility:

mc alias set <alias_source> <s3-endpoint_source> <access-key_source> <secret-key_source>
mc alias set <alias_destination> <s3-endpoint_destination> <access-key_destination> <secret-key_destination>

Where:

  • <alias_source>. The short name (alias) for the source.
  • <s3-endpoint_source>. The address of the S3-compatible source storage.
  • <access-key_source>. The access key for the source.
  • <secret-key_source>. The secret key for the source.
  • <alias_destination>. The short name (alias) for the destination.
  • <s3-endpoint_destination>. The address of the S3-compatible destination storage.
  • <access-key_destination>. The access key for the destination.
  • <secret-key_destination>. The secret key for the destination.

Then copy the S3 storage from the source to the destination:

mc cp --recursive <s3-endpoint_source>/bucket-name/ <s3-endpoint_destination>/bucket-name/

Where:

  • <alias_source>. The short name (alias) for the source.
  • <alias_destination>. The short name (alias) for the destination.
  • bucket-name. The bucket name.
  • --recursive. Recursive copying of all objects and files.
  1. After restoring the databases on external storage, reactivate the system.

Step 4: Update BRIX On-Premises application parameters

  1. After restoring data on external databases, make sure that all tables for PostgreSQL or collections for MongoDB are migrated correctly.
  2. To configure the BRIX On-Premises application to use external databases, change the database and S3 file storage connection settings in the values-elma365.yaml configuration file.
  3. Update the BRIX application parameters using the values-elma365.yaml configuration file.

Updating parameters via the Internet

  1. Identify the version of the chart from which the BRIX application was installed or updated:

helm show chart brix365/brix365

 
Example of how to execute the command:

embedded-to-external-database-enterprise-1

The chart version will be specified in the version string. Save this value for the next step.

  1. Update the BRIX application parameters using the values-elma365.yaml configuration file. To do this, use the following command, specifying the installed chart version for the --version key instead of <elma365-chart-version>:

helm upgrade --install brix365 brix365/brix365 -f values-brix365.yaml --version <brix365-chart-version> --timeout=30m --wait [-n namespace]

 

Updating parameters without the Internet connection

 
Go to the folder with the downloaded BRIX chart and run the command:

helm upgrade --install brix365 ./brix365 -f values-brix365.yaml --timeout=30m --wait [-n namespace]

 
The parameter upgrade takes 10-30 minutes. Wait for it to complete.

 
  1. Restart all services with the command:

kubectl rollout restart deploy [-n namespace]

  1. Verify the system is working correctly and stop old databases inside Kubernetes:

kubectl scale statefulset <db-type> --replicas=0 [-n namespace]

Where <db-type> is the name of the StatefulSet: postgres, mongo, or minio.

 

 

embedded-to-external-database-enterprise-backupper.html enable-portable-services.html