Migration guide

Migrating from Single Machine to Clustered Bugsnag On-premise

Prerequisites

To get started with your migration from Single Machine to Clustered Bugsnag On-premise, you’ll need the following things prepared in advance:

  • Kubernetes cluster running 1.12 or higher
  • Docker registry
  • Current Bugsnag machine has the latest version

Running the migration

The migration can take some time and is dependent on the number of events you have, however you can continue to use your current Single Machine instance whilst the data is migrating. We estimate that it takes around 1 hour to migrate 8 million events of data. There will be some downtime after the data has been migrated and Clustered Bugsnag On-premise starts, this may take around 10-15 mins.

Configuration

Run migrating-settings.sh on the currently running Bugsnag machine. This will generate a copy of the config required for the migration: state.json.

Copy state.json under .ship folder from where you will be starting the migration.

Configure Bugsnag On-premise and generate Kubernetes YAML

Run the following to run the configuration tool:

docker pull replicated/ship && \
docker run -it -p 8800:8800 --rm -v `pwd`:/out \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -e HTTP_PROXY -e HTTPS_PROXY -e NO_PROXY \
  replicated/ship init --preserve-state \
  "replicated.app/bugsnag-k8s-ship"

You will be asked to enter your Bugsnag On-premise license. Enter the license you received from Bugsnag to continue to the next step.

The configuration tool will be available at http://localhost:8800

Migrate databases

  1. In the config section under Migration: Enable migration mode by ticking the checkbox: img
  2. (Optional) Configure the “MongoDB node port” and “Uploads storage node port” settings under the Migration section.
  3. Make sure to Enable high availability features under the Kubernetes section if you are going to be running Bugsnag in high availability mode after migration has completed. img
  4. Deploy by first clicking “Save and continue to next step” to generate the Kubernetes YAML and using instructions on how to apply it. On the next screen there will be a “Migration” section will values which will be required as configuration on your current Bugsnag instance so make sure to keep these values on hand. img
  5. Configure the current Bugsnag machine via Replicated settings to enable migration mode: img
  6. Configure this section using the values output earlier:
    • “Kubernetes MongoDB instance address”: This should be configured to address the MongoDB instance running on Kubernetes, usually this can be in the form of “Kubernetes Node IP:MongoDB node port”, this address should be accessible from the current Bugsnag machine.
    • “Current MongoDB instance address”: This should be configured to address the MongoDB instance running on the current Bugsnag machine, usually this can be in the form of “Current machine IP:57017”, this address should be accessible from the Kubernetes cluster.
    • “Address of uploads storage”: This should be configured to address the uploads storage instance running on Kubernetes usually this can be in the form of “Kubernetes Node IP:Uploads storage node port”
    • “Access key for uploads storage”: Use the value from the earlier output.
    • “Secret key for uploads storage”: Use the value from the earlier output.
  7. Save and restart Bugsnag when prompted, the migration will start once Bugsnag has restarted. You can check the progress of the migration using the instructions below:

Monitoring uploads migration progress

You can check the progress of the uploads migration by running the following on the Bugsnag machine:

docker logs bugsnag_migrate-uploads

Once this states “Migration of uploads completed.”, this migration is complete.

Monitoring MongoDB migration progress

You can check the progress of the MongoDB migration by checking the status of the replica set:

kubectl -n <NAMESPACE> exec -it mongo-0 -- mongo-local --eval "rs.status()"

If the MongoDB instance in Kubernetes is in STARTUP2 mode you can check the progress of the collection copying using:

kubectl -n <NAMESPACE> logs -f mongo-0 | grep "repl writer worker"

Once the instance is in SECONDARY mode you can use the following to check the replication lag:

kubectl -n <NAMESPACE> exec -it mongo-0 -- mongo-local --eval "rs.printSlaveReplicationInfo()"

Once this states that the MongoDB instance is 0 secs behind master, this migration is complete.

Migrate redis

Once MongoDB has finished migrating run migrate-redis.sh on the Bugsnag machine and select “Backup redis data” option to begin backing up the redis data from your current Bugsnag instance. Once that has completed you can use the “Restore redis data to kubernetes” option to begin the restore. If you do not have access to the Kubernetes cluster from that machine you should copy the output archive to machine which does and run migrate-redis.sh using the restore option and specifying the archive location to begin migrating the redis data.

Finishing up

Once all the migration are complete:

  1. Configure the current Bugsnag machine via Replicated settings to disable migration mode and restart: img

  2. Run the following to reconfigure Bugsnag:

docker pull replicated/ship && \
docker run -it -p 8800:8800 --rm -v `pwd`:/out \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -e HTTP_PROXY -e HTTPS_PROXY -e NO_PROXY \
  replicated/ship edit

Disable migration mode and apply the generated Kubernetes YAML.

img

  1. Once Bugsnag is running, Elasticsearch will require a reindex, which you can monitor using “Events in Elasticsearch/Mongo” graph under the “Support” dashboard on Grafana.