Martilla is a project I've been working on recently. It's a CLI tool to help automate database backups. It's main objectives are to remain modular, configurable and simple to understand.
I was heavily inspired by the successful Backup project. The way they mix & match database engines, storage solutions, notifiers and other utilities made me a user of them. In the last couple of years support has decreased though, so I'm decided to work with this simplified concept (also to replace the DSL with a ruby-agnostic config file).
Installation & the configuration file
You'll need Ruby installed on your machine to get started. I'd highly recommend using a version manager
$ gem install martilla
$ martilla setup backup-config.yml
That's it! You now have Martilla working and a sample configuration file that looks like this
---
db:
type: postgres
options:
host: localhost
user: username
password: password
db: databasename
storage:
type: local
options:
filename: database-backup.sql
notifiers:
- type: none
Customize the config file
Let's say we have a Postgres database and we want backups stored on a S3 bucket at midnight everyday. You'll of course have to modify the db
directive with your database details, but also replace the default storage
details
storage:
type: s3
options:
bucket: sideproject_backups
filename: backup.sql
region: 'us-east-1'
access_key_id: XXXXXXXX
secret_access_key: YYYYYYYY
Once configured test your database backup to make sure it works as expected
$ martilla backup backup-config.yml
If everything's working fine all you need is to add this command to your crontab to run at midnight everyday. You've just automated database backups to S3!
In case you hate writing to crontab directly like me, check out the whenever gem if you haven't already.
Get notified
It's good to have backups run consistently, but it might be just as important to get notified when they fail.
Good news are that if we want an email notification to be sent and also a Slack notification we don't have to settle with just one or the other. Replace the type: none
notifier from the sample configuration with the following:
notifiers:
- type: 'slack'
options:
slack_channel: '#backups'
slack_webhook_url: https://hooks.slack.com/services/XXXXX/YYYYY/ZZZZZ
- type: ses
options:
from: backups@my_domain.com
to: fernando@visualcosita.com
region: 'us-east-1'
access_key_id: XXXXXXXX
secret_access_key: YYYYYYYY
The notifiers tag in the config file is an array, so you can add as many notifiers as you want. This means even multiple Slack notifications to different webhooks, just continue to chain notifiers on the list.
Now if a problem happens and your backup fails you'll have visibility of it right away!
Hacktoberfest
Thanks to Hacktoberfest we got our first contribution and that's how we have Slack notification support!
If you're looking for a Ruby project to contribute during this last stretch of "hacktober" take a look at our open issues. I'll be happy to guide you through questions you might have.
Conclusions
Martilla is meant to be a "simple backup tool for simple everyday use". High throughput and larger databases (probably > 50GB in size) will likely benefit more from backup solutions provided by most managed database services in the likes of AWS RDS.
For smaller deployments and side projects this might be an interesting addition to your toolbelt. For future releases I'm thinking of features like more integrations (databases, storages and notifiers), a command to automatically restore a backup and more customizable options.
That's it for now though, I hope this was helpful and check out our GitHub repo for more in-depth usage details. Pura vida!
This post was originally published on visualcosita.com
Top comments (2)
That sounds awesome :) I'm wondering how would you restore such a backup again?
Glad you found it interesting!
The CLI leverages existing tooling to dump the databases like
pg_dump
&mysqldump
. So to restore the backups you can usepg_restore
, directly restore usingpsql
, etc..One of the most important features I want to add is a command to restore a backup directly from the CLI. But this is something I'm still going back and forth thinking how it would work seamlessly with the config file and parameters passed in from the terminal.