DEV Community

Cover image for Martilla - A simple backup tool for simple everyday use
Fernando
Fernando

Posted on • Updated on • Originally published at fdo.cr

Martilla - A simple backup tool for simple everyday use

Martilla is a project I've been working on recently. It's a CLI tool to help automate database backups. It's main objectives are to remain modular, configurable and simple to understand.

I was heavily inspired by the successful Backup project. The way they mix & match database engines, storage solutions, notifiers and other utilities made me a user of them. In the last couple of years support has decreased though, so I'm decided to work with this simplified concept (also to replace the DSL with a ruby-agnostic config file).

Installation & the configuration file

You'll need Ruby installed on your machine to get started. I'd highly recommend using a version manager

$ gem install martilla
$ martilla setup backup-config.yml
Enter fullscreen mode Exit fullscreen mode

That's it! You now have Martilla working and a sample configuration file that looks like this

---
db:
  type: postgres
  options:
    host: localhost
    user: username
    password: password
    db: databasename
storage:
  type: local
  options:
    filename: database-backup.sql
notifiers:
- type: none
Enter fullscreen mode Exit fullscreen mode

Customize the config file

Let's say we have a Postgres database and we want backups stored on a S3 bucket at midnight everyday. You'll of course have to modify the db directive with your database details, but also replace the default storage details

storage:
  type: s3
  options:
    bucket: sideproject_backups
    filename: backup.sql
    region: 'us-east-1'
    access_key_id: XXXXXXXX
    secret_access_key: YYYYYYYY
Enter fullscreen mode Exit fullscreen mode

Once configured test your database backup to make sure it works as expected

$ martilla backup backup-config.yml
Enter fullscreen mode Exit fullscreen mode

If everything's working fine all you need is to add this command to your crontab to run at midnight everyday. You've just automated database backups to S3!

In case you hate writing to crontab directly like me, check out the whenever gem if you haven't already.

Get notified

It's good to have backups run consistently, but it might be just as important to get notified when they fail.

Good news are that if we want an email notification to be sent and also a Slack notification we don't have to settle with just one or the other. Replace the type: none notifier from the sample configuration with the following:

notifiers:
- type: 'slack'
  options:
    slack_channel: '#backups'
    slack_webhook_url: https://hooks.slack.com/services/XXXXX/YYYYY/ZZZZZ
- type: ses
  options:
    from: backups@my_domain.com
    to: fernando@visualcosita.com
    region: 'us-east-1'
    access_key_id: XXXXXXXX
    secret_access_key: YYYYYYYY
Enter fullscreen mode Exit fullscreen mode

The notifiers tag in the config file is an array, so you can add as many notifiers as you want. This means even multiple Slack notifications to different webhooks, just continue to chain notifiers on the list.

Now if a problem happens and your backup fails you'll have visibility of it right away!

Hacktoberfest

Thanks to Hacktoberfest we got our first contribution and that's how we have Slack notification support!

If you're looking for a Ruby project to contribute during this last stretch of "hacktober" take a look at our open issues. I'll be happy to guide you through questions you might have.

Conclusions

Martilla is meant to be a "simple backup tool for simple everyday use". High throughput and larger databases (probably > 50GB in size) will likely benefit more from backup solutions provided by most managed database services in the likes of AWS RDS.

For smaller deployments and side projects this might be an interesting addition to your toolbelt. For future releases I'm thinking of features like more integrations (databases, storages and notifiers), a command to automatically restore a backup and more customizable options.

That's it for now though, I hope this was helpful and check out our GitHub repo for more in-depth usage details. Pura vida!

This post was originally published on visualcosita.com

Top comments (2)

Collapse
 
tiim profile image
Tim Bachmann

That sounds awesome :) I'm wondering how would you restore such a backup again?

Collapse
 
fdocr profile image
Fernando

Glad you found it interesting!

The CLI leverages existing tooling to dump the databases like pg_dump & mysqldump. So to restore the backups you can use pg_restore, directly restore using psql, etc..

One of the most important features I want to add is a command to restore a backup directly from the CLI. But this is something I'm still going back and forth thinking how it would work seamlessly with the config file and parameters passed in from the terminal.