Backing up Appwrite to Backblaze

Table Of Contents

• Introduction
• Getting Backblaze secrets
• B2 CLI basics
• Appwrite MariaDB 👉 Backblaze
• Appwrite Docker Volumes 👉 Backblaze

Introduction

Recently, Appwrite has released an article on how to backup and restore all data on your production server. You can read it here:

Appwrite In Production: Backups and Restores

Bradley Schofield for Appwrite ・ Jul 10 '21

#webdev #opensource #devops #tutorial

This article got me thinking 🤔 Sure, you can backup most MySQL databases into a simple backup.sql file, but what if you have multiple massive projects inside Appwrite? Your disk storage may or may not be able to hold a backup of all Appwrite volumes. Additionally, you most likely need to backup periodically and need multiple versions of the backup. So, what are your options?

• Write a complex bash script to keep last three versions
• Throw all backups to a storage provider and let them take care of it

You should go with storage providers not only because they have a retention system already in place, also due to their experience in the field as they most likely care about security a lot and use raids to ensure your data won't get lost.

I wrote this article as I was implementing this on my own production server, and I will show you everything you need to know about backing up Appwrite into Backblaze.

Getting Backblaze secrets

First things first, what is Backblaze? Backblaze is one of the most, if not most, used commercial platform to backup your data for the lowest price possible. They provide solutions to backup your personal computer or your business data. The service we will be using is B2 Cloud Storage as this gives our flexibility just like an actual hard drive. You don't have to worry about pricing yet because their free plan gives you 10GB of free space. To learn more about this product and its pricing, you can take a look at: https://www.backblaze.com/b2/cloud-storage.html

"Over one exabyte of data storage under management, and counting." - Backblaze website

After creating an account and logging in (make sure to confirm your email), Backblaze may ask you to set up 2FA using your phone number for additional security.

To use Backblaze from our server, we will need:

• BUCKET_NAME
• ACCESS_KEY_ID
• APPLICATION_KEY

Backblaze uses buckets to store your data. In our case, we will only need one bucket as we will only back up two or three files at most (MySQL database and Appwrite Docker volumes). To create a bucket, navigate to Bucket and press Create a Bucket. Give it some name such as appwrite and click Create a Bucket again. You should see a newly created bucket on your list. Make sure to mark your Bucket Name as we will need this later.

If you enter App Keys, you can Add a New Application Key. For simplicity, we will ignore that, and we will use our master key to have all permissions. Simply click Generate a New Master Application Key, and this should give you keyID and applicationKey.

Wohoo 🎉 we have all secrets we need; let's continue!

B2 CLI basics

To make sure everything is working, let's try some simple action on B2:

• List files in a bucket
• Upload a file
• List files again

We need to have B2 CLI installed on our machine, but this can be easily avoided thanks to a Docker image mtronix/b2-cli:0.0.1. This is the only image I could find that was actually working.

To list all files in our bucket, we run:

docker run \
  --rm \
  mtronix/b2-cli:0.0.1 \
  bash -c \
  "b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 list-file-names <BUCKET_NAME>"

The output should be an empty list of files:

{
  "files": [],
  "nextFileName": null
}

Let's upload a file 😎
We need to have a file... I downloaded a cute cat image and saved it as cat.png. To upload the file, I run:

docker run \
  --rm \
  -v $PWD:/b2 \
  mtronix/b2-cli:0.0.1 \
  bash -c \
  "b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> cat.png cat.png"

The successful result should look something like:

{
  "action": "upload",
  "fileId": "4_z78da5cd2a05db73574a90515_f11841831f8e91ca6_d20210717_m092603_c002_v0001140_t0056",
  "fileName": "cat.png",
  "size": 136021,
  "uploadTimestamp": 1626513963000
}

Finally, run the list-file-names command from above again, and you should see your file in the array of files:

{
  "files": [
    {
      "accountId": "8ac20d754955",
      "action": "upload",
      "bucketId": "78da5cd2a05db73574a90515",
      "contentLength": 136021,
...

Backblaze let you do all of this function manually on their website. Exercise for you 😲 Remove our test file from the bucket. Do it however you want - you can remove it using B2 CLI or manually on the website.

Appwrite MariaDB 👉 Backblaze

To create an Appwrite MariaDB backup, you run this command:

docker-compose exec mariadb sh -c 'exec mysqldump --all-databases --add-drop-database -u "$MYSQL_USER" -p "$MYSQL_ROOT_PASSWORD"' > ./dump.sql

(Source: Appwrite article mentioned earlier)

To backup this export into Backblaze, I created a script that:

1. Export MariaDB data to dump.sql
2. Upload dump.sql to Backblaze
3. Remove dump.sql to free up the space on the machine

This is what the script looks like:

docker-compose exec \
  mariadb \
  sh -c \
  'exec mysqldump --all-databases --add-drop-database -u"$MYSQL_USER" -p"$MYSQL_ROOT_PASSWORD"' > ./dump.sql ; \
  docker run \
  --rm \
  -v $PWD:/b2 \
  mtronix/b2-cli:0.0.1 \
  bash -c \
  "b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> dump.sql dump.sql" ; \
  rm dump.sql

Each time you run this script, it will backup your MariaDB database into Backblaze. Backblaze can store multiple versions of a single file; just make sure to configure retention on a bucket, for example, 7 days.

Appwrite Docker Volumes 👉 Backblaze

Once again, let's get our base command from Appwrite's article:

Before running these commands is it highly recommended to shut down your Appwrite instance to ensure you get a complete backup.

mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/uploads && tar cvf /backup/uploads.tar ."

mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/functions && tar cvf /backup/functions.tar ."

These two commands will backup two different volumes. The first one back up all files in Appwrite storage while the second one backup your function tags. Let's start with backing up uploads.

Just like with MariaDB, we will:

1. Create a backup in backup/uploads.tar
2. Send the file to Backblaze
3. Remove the backup file

The full command to do this is:

mkdir -p backup && docker run \
  --rm \
  --volumes-from "$(docker-compose ps -q appwrite)" \
  -v $PWD/backup:/backup \
  ubuntu \
  bash -c \
  "cd /storage/uploads && tar cvf /backup/uploads.tar ."  ; \
  docker run \
  --rm \
  -v $PWD:/b2 \
  mtronix/b2-cli:0.0.1 \
  bash -c \
  "b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> backup/uploads.tar upload_backup.tar" ; \
  rm backup/uploads.tar

This should create a file called upload_backup.tar in your Backblaze bucket.

Exercise time 💪 You will need to backup Appwrite functions too! Using all knowledge from this article, you should be able to prepare a command that will do it.
If you are here only to copy&paste commands, here is the command:

mkdir -p backup && docker run \
  --rm \
  --volumes-from "$(docker-compose ps -q appwrite)" \
  -v $PWD/backup:/backup \
  ubuntu \
  bash -c \
  "cd /storage/functions && tar cvf /backup/functions.tar ."  ; \
  docker run \
  --rm \
  -v $PWD:/b2 \
  mtronix/b2-cli:0.0.1 \
  bash -c \
  "b2 authorize-account <ACCESS_KEY_ID> <ACCESS_KEY_ID> && b2 upload-file <BUCKET_NAME> backup/functions.tar upload_functions.tar" ; \
  rm backup/functions.tar

That's it! I hope this article helped you backing up your Appwrite server. If you have any questions, feel free to join Appwrite's Discord server and chat with their amazing community: https://appwrite.io/discord