☕️ Let's automate a backup process of your GitHub accounts, organizations & repositories

Introduction

Good day/night, DEV people! 😉 I declare openly: GitHub is my favorite professional social network to communicate and share my experience with other developers around the World.

But sometimes, when servers crash (which have increased dramatically in the last six months), I would like to have backups of all my projects on my work computer.

I'm talking about a full backup with all history, commits, pull-requests and releases of GitHub repository... not a simple git clone command!

— So, what do I need to start backing up my GitHub account?

Let's work it out together! 👍

📝 Table of contents

• The problem
• Solution

The problem

If you've never seen GitHub Status page, I'm very happy for you! Because, it means I caught Internal Server Error (500), while working with GitHub... 😨

All the useful information, like "what's broken" or "when it's fixed", will be on this page. Also, you can subscribe to official Twitter account.

Yes, we spend thousands of hours (a la money) to write, support and improve projects. But let's not forget that the project doesn't really belong to us 100 percent, if it was uploaded to commercial service.

💭 GitHub is a commercial service for working with git (which also belongs to Microsoft Corporation), even if it has free plans.

Any downtime or rollback of such commercial service, can play a bad joke on you. Therefore, it's a good reason to have a full backup of all your projects. And on the whole, the backup hasn't bothered anyone yet!

↑ Table of contents

Solution

• Install Python (3.5 and later) and Ansible (2.9 and later):

brew install python3 ansible

☝️ This is example for macOS using the Brew package manager. How to install Python and Ansible on your system, please ask Google.

• Download (or git clone) the Useful Ansible playbooks collection and go into its folder:

git clone https://github.com/truewebartisans/useful-playbooks.git

cd useful-playbooks

• Go to your GitHub settings and generate a new personal access token with needed scopes.

• Create .env file with the following structure:

# GitHub users or organizations login names, 
# separated with commas (without spaces).
USERS=user123,org123

# GitHub personal access token.
GITHUB_TOKEN=0000000000000000000000000

# Max count of a saved backup files.
MAX_BACKUPS=10

# Time to delay.
# - d: days
# - h: hours
# - m: minutes
# - s: seconds
DELAY_TIME=1d

# Timezone for DELAY_TIME.
# See: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
TIME_ZONE=America/Chicago

☝️ This is required, because the Docker image, what we're using in this playbook (koddr/github-backup-automation), rest on these environment variables.

• Create a new virtual server with a pre-installed Docker (on DigitalOcean, for example, or on your favorite cloud provider. The minimum droplet's plan for $5/month, should be more than enough).

• And, finally, run Ansible playbook:
  

ansible-playbook \
                  github_backup-playbook.yml \
                  --user <USER> \
                  --extra-vars "host=<HOST>"

<USER> remote user's username (for example, root)
<HOST> hostname in your inventory (from /etc/ansible/hosts file)

All saved backups will be located at /var/github-backup folder of your remote server. That's it! 🎉

↑ Table of contents

Photos by

• GitHub Status page https://www.githubstatus.com

P.S.

If you want more articles (like this) on this blog, then post a comment below and subscribe to me. Thanks! 😻

And of course, you can help me make developers' lives even better! Just connect to one of my projects as a contributor. It's easy!

My projects that need your help (and stars) 👇

• 🔥 gowebly: A next-generation CLI tool for easily build amazing web applications with Go on the backend, using htmx & hyperscript and the most popular atomic/utility-first CSS frameworks on the frontend.
• ✨ create-go-app: Create a new production-ready project with Go backend, frontend and deploy automation by running one CLI command.
• 🏃 yatr: Yet Another Task Runner allows you to organize and automate your routine operations that you normally do in Makefile (or else) for each project.
• 📚 gosl: The Go Snippet Library provides snippets collection for working with routine operations in your Go programs with a super user-friendly API and the most efficient performance.
• 🏄‍♂️ csv2api: The parser reads the CSV file with the raw data, filters the records, identifies fields to be changed, and sends a request to update the data to the specified endpoint of your REST API.
• 🚴 json2csv: The parser can read given folder with JSON files, filtering and qualifying input data with intent & stop words dictionaries and save results to CSV files by given chunk size.