DEV Community

Cover image for Appwrite In Production: Backups and Restores
Appwrite profile image Bradley Schofield
Bradley Schofield for Appwrite

Posted on • Updated on • Originally published at appwrite.io

Appwrite In Production: Backups and Restores

#webdev #opensource #devops #tutorial

Backing up and restoring data is an extremely important part of running servers. It's a virtual safety net against most bad things that can happen. Made a bad config change? Restore a backup. Messed up an update? Restore a backup. Corrupted Drives? Restore a backup.

Not only that, backups can also come in handy when migrating data to other systems, for instance migrating a development server into a production environment or vice versa ๐Ÿ”

To make this entire process as easy as possible we've written this simple and handy guide to explain everything you need to know about backing up and restoring your Appwrite instance.

Appwrite is broken down into multiple sections and most of Appwrite is stateless. This means that Appwrite only actually has two main things you need to backup: one is Appwrite's Database (MariaDB) and the other is the Docker volumes which store functions data and uploads. The rest - Appwrite can automatically handle and regenerate.

Please note that all these commands need to be run within the same directory as Appwrite's docker-compose.yml

With all that said, lets begin! ๐Ÿ˜Š

๐Ÿ’พ Backing up the MariaDB Database

Due to the fact that Appwrite uses a Docker image of MariaDB it is extremely easy to dump the entire database with just one command and likewise to restore the dump.

Creating a Database backup is just one command:

docker-compose exec mariadb sh -c 'exec mysqldump --all-databases --add-drop-database -u"$MYSQL_USER" -p"$MYSQL_PASSWORD"' > ./dump.sql
Enter fullscreen mode Exit fullscreen mode

Going into depth this command does a couple things:

  1. Docker-compose launches a temporary shell onto the MariaDB container to start work
  2. It runs mysqldump on the server with two specific options --all-databases and --add-drop-database these are important since they ensure that when the backup is restored old data doesn't get overlapped with new data.
  3. The output of mysqldump is piped into a dump.sql file. This is your backup โœ…๏ธ

โœ๏ธ Restoring the MariaDB Database

Restoring the database is similarly easy and also requires just one command to do so:

docker-compose exec -T mariadb sh -c 'exec mysql -u"$MYSQL_USER" -p"$MYSQL_PASSWORD"' < dump.sql
Enter fullscreen mode Exit fullscreen mode

This command is very simple once you break it down:

  1. Docker-compose launches a temporary shell onto the MariaDB container to start work
  2. Using the mysql command we restore the dump through a pipe

๐Ÿ’ฟ Backing up your Docker Volumes

Appwrite stores various things in Docker volumes. This includes your file uploads and Cloud Function data and docker volumes makes it easy for us to coordinate data between the central Appwrite container and our various Appwrite workers. Uploads are especially important to backup since they include all your app's file uploads, these commands can take a while to run depending on how much data you have to backup.

Before running these commands is it highly recommended to shutdown your Appwrite instance to ensure you get a complete backup.

To backup the functions volume the command is:

mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/functions && tar cvf /backup/functions.tar ."
Enter fullscreen mode Exit fullscreen mode

and to backup the uploads volume the command is:

mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/uploads && tar cvf /backup/uploads.tar ."
Enter fullscreen mode Exit fullscreen mode

Both these commands do similar things and when you break them down they are pretty simple.

  1. Start a new Docker container. This Docker container has a few special options

--rm will delete the container once it's done running. The reason we want this is because this container is only being used to package up our backup and give it to the host machine.

--volume-from This flag special as it will mount all of the volumes of the container we give it. To get the container ID we want we use a $(docker-compose ps -q appwrite) to get the ID within the command

-v This flag is being used to mount a volume onto our new container which will give us access to a backup folder we created using the mkdir command at the start

ubuntu is the image we are basing our new container on

  1. Finally with this command created we change directories into the normal Appwrite mount point for uploads and create a tarball which will be created in the backup directory where we will be able to access it.

Once these commands are run you should find a new backup folder which containsuploads.tar and functions.tar these are your backups. Keep them safe.

๐Ÿ“ Restoring your Docker Volumes

Restoring your Appwrite volumes is fairly simple as well. Move the backup folder you just created to your destination machine next to the docker-compose.yml file and simply run the following commands to restore the backup.

Please note that the Appwrite instance should be shutdown while running these commands.

Restoring functions volume:

docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/restore ubuntu bash -c "cd /storage/functions && tar xvf /restore/functions.tar --strip 1"
Enter fullscreen mode Exit fullscreen mode

Restoring uploads volume:

docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/restore ubuntu bash -c "cd /storage/uploads && tar xvf /restore/uploads.tar --strip 1"
Enter fullscreen mode Exit fullscreen mode

This command creates new temporary Docker container like the backup command but instead extracts the tar back into the functions and uploads endpoint which will restore the backup.

โญ๏ธ In Conclusion

To create a full Appwrite backup you will want to backup MariaDB and the two volumes specified here. Once you have done that make sure to keep it safe, the best backup will store the backup in multiple locations including locally and in multiple clouds. As with any Cloud Native Application, make sure that you backup your Appwrite instance regularly to ensure that you're never in a situation where you have to lose data due to a server failure.

This entire process can be easily done to help migrate an Appwrite installation easily by simply copying the backup files onto another server and running the restore steps.

We hope you enjoyed this article! We love contributions and encourage you to take a look at our open isuses and ongoing RFCs.

If you get stuck anywhere, feel free to reach out to us on our friendly support channels run by humans ๐Ÿ‘ฉโ€๐Ÿ’ป

Here are some handy links for more information:

Thank you to Taylor Vick for the server image.

Top comments (19)

Collapse
 
elreyes profile image
Fabian Reyes •

Hi, I have tried to replicate this guide and everything works fine except the database restore, I get this error: ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'mysqldump: Got error: 1045: "Access denied for user 'user'@'localhost' (using...' at line 1

Collapse
 
eldadfux profile image
Eldad A. Fux •

I think you need to make sure your target database has the same user credentials.

Collapse
 
elreyes profile image
Fabian Reyes • • Edited

For now, I managed to restore the files and the database without any problems. Now, there is a problem. The files do not work, apparently they were restored but it is not possible to access or download them.

dev-to-uploads.s3.amazonaws.com/up...
dev-to-uploads.s3.amazonaws.com/up...

Thread Thread
 
eldadfux profile image
Eldad A. Fux •

Make sure you use the same encryption key in your env file

Thread Thread
 
elreyes profile image
Fabian Reyes •

You mean, this one? _APP_OPENSSL_KEY_V1

Thread Thread
 
eldadfux profile image
Eldad A. Fux •

Yep :)

Collapse
 
stnguyen90 profile image
Steven •

I think there is a typo/bug in the user part of the database backup command. I think it should be:

docker-compose exec mariadb sh -c 'exec mysqldump --all-databases --add-drop-database -u"root" -p"$MYSQL_ROOT_PASSWORD"' > ./dump.sql
Enter fullscreen mode Exit fullscreen mode
Collapse
 
gewenyu99 profile image
Vincent Ge •

Thanks for pointing this out ;)

Took a while but we got to it! It should be fixed now <3

Collapse
 
stnguyen90 profile image
Steven •

Thanks! Looks like the same typo is in the mariadb restore command too.

Collapse
 
stnguyen90 profile image
Steven •

I think people have also had problems migrating if they don't migrate their appwrite-certificates and maybe appwrite-config volume. This is because the database has data about certificates generated, but the certificates don't actually exist on disk so HTTPS doesn't work.

If you didn't get a chance to backup those, you can try to manually delete the certificates info in the database manually. $ and > characters to indicate whether the command is executed in the shell or mysql CLI, respectively:

# shell into the mariadb container
$ docker-compose exec mariadb sh

# authenticate with mysql
# do not add whitespace between -p and password
$ mysql -u root -p"$MYSQL_ROOT_PASSWORD"

# use the appwrite database
> use appwrite;

# Find the documentID for your domain in the database:
> SELECT * FROM `app_console.database.properties` p where p.key = "domain";

# note the documentUid - that's what we'll use to query/delete them from the database
# We'll need to remove the document from two tables, here's the other one
> SELECT * FROM `app_console.database.documents` d where d.uid = "thedocumentUID";

# Now we can delete these two rows:
> DELETE FROM `app_console.database.properties` where documentUid = "thedocumentUID";
> DELETE FROM `app_console.database.documents` where uid = "thedocumentUID";
Enter fullscreen mode Exit fullscreen mode

Credits to @kodumbeats for the SQL queries.

Collapse
 
brianmutiso profile image
Brian Mutiso •

I was not able to back up maria db all the scripts in this article only produces a 0byte backup file

docker-compose exec mariadb sh -c exec mysqldump --all-databases --add-drop-database -u"root" -p"$MYSQL_ROOT_PASSWORD" > ./dump.sql

Collapse
 
subrotoxing profile image
subrotoxing • • Edited

I'm trying to backup my database ... but dump.sql seems to be empty... please advise

i'm typing this into cmd where the .env file located

docker-compose exec mariadb sh -c exec mysqldump --all-databases --add-drop-database -u user -p password > ./dump.sql

Collapse
 
meldiron profile image
Matej Baฤo • • Edited

Hey there ๐Ÿ‘‹ Tiny syntax mistake, it should be:

docker-compose exec mariadb sh -c exec mysqldump --all-databases --add-drop-database -u"user" -p"password" > ./dump.sql
Enter fullscreen mode Exit fullscreen mode
Collapse
 
heynicolasklein profile image
Nicolas Klein •

Still does not work for me on Appwrite v15

Collapse
 
meldiron profile image
Matej Baฤo •

You can't imagine how many times have I opened this article ๐Ÿ˜… Thanks a lot for this article!

Collapse
 
mandolf0 profile image
mandolf0 • • Edited

Your restore/backup gist helps tremendously. I just migrated local dev to the โ˜. One suggestion I'd like to make is to delete the cloud function deployments and use only the latest one. The backup I moved was 800MB because of those deployments. Other than that, 100% confirmed it works to a tee.

As of now with Appwrite 1.2.0, I deleted some rows in other tables.

delete from _console_domains;

//show domain permissions
select * from \_console_domains_perms;

//delete domain permissions
delete from_console_domains_perms;

rsync is our friend when moving to a new server.

rsync -avz --progress backups/backup-2023-03-21T02-05-16.tar.gz root@[targetServer]:/root/appwrite/backups/backup.tar.gz

2023-03-21T02-05-16.tar.gz is the source file to copy
backup.tar.gz is the target because that is the name restore.sh expects to find alongside it.

Issue certificates
docker-compose up -d
docker compose exec appwrite ssl domain="app.mydomain.com"

Collapse
 
fuadefendi profile image
FuadEfendi •

And how to backup/restore on MacOS?

Collapse
 
brianmutiso profile image
Brian Mutiso •

I don't seem to get it, even after running the above commands i still get an empty dump.sql file

Collapse
 
heynicolasklein profile image
Nicolas Klein •

I found this solution on discord:

here , what i did and it works
backup data :
docker-compose exec mariadb sh -c 'exec mysqldump --all-databases --add-drop-database -u"user" -p"password"' > ./dump.sql

restore data :
docker-compose exec -T mariadb sh -c 'exec mysql -uโ€userโ€-p"password"' < dump.sql

View full discussion (19 comments)

Read next

jetthoughts-dev profile image

Ruby on Rails 8: Simplifying Sharding with New Methods

JetThoughts Dev -

nenepo profile image

Optimizing React Re-Renders: Best Practices for Performance

Nenepo -

thinkthroo profile image

Generate a documentation site using Nextra

thinkThroo -

katorymnddev profile image

Exploring Pawapay Deposits, Refunds, and Payouts: A Hands-On Preview

katorymnddev -