Sync GitHub User or Organization Repositories with a Simple Script

Sync GitHub User or Organization Repositories with a Simple Script

Keeping all repositories in a GitHub organization updated can be a tedious task, especially when working with multiple repositories. With a simple Bash script, you can automate the process of cloning and syncing all repositories in your organization. This article walks you through the script and how it works.

Why Automate Repository Syncing?

Manually cloning and pulling updates for multiple repositories is time-consuming and prone to errors. Automating this process offers the following benefits:

• Time-Saving: Quickly fetch and update all repositories without manual intervention.
• Consistency: Ensure every repository is up to date across local and remote systems.
• Efficiency: Handle repositories with minimal effort using Git and the GitHub API.

The Script: Clone and Sync Repositories

Here's the script to automate cloning and syncing repositories:

GITHUB_ORG=jdevto
BASE_DIR="." 

# Create the base directory
mkdir $GITHUB_ORG
cd $GITHUB_ORG

# Function to clone or sync a repository
sync_repo() {
  local repo_url="$1"
  local repo_name=$(basename "$repo_url" .git)
  local repo_dir="$BASE_DIR/$repo_name"

  if [ -d "$repo_dir" ]; then
    echo "Updating $repo_name..."
    git -C "$repo_dir" pull --rebase
  else
    echo "Cloning $repo_name..."
    git clone "$repo_url" "$repo_dir"
  fi
}

# Fetch repositories from GitHub API
echo "Fetching repositories for organization: $GITHUB_ORG..."
page=1

# Loop through each repository and sync it
while true; do
  repos=$(curl -s "https://api.github.com/orgs/$GITHUB_ORG/repos?per_page=100&page=$page" | jq -r '.[].ssh_url')
  [ -z "$repos" ] && break
  for repo_url in $repos; do
    sync_repo "$repo_url"
  done
  ((page++))
done

How It Works

1. Setup Organization and Base Directory

- The GITHUB_ORG variable holds your organization or GitHub user name (e.g., jdevto).
- The BASE_DIR variable determines where repositories will be cloned or updated.

1. Clone or Sync Repositories

- The sync_repo function checks if a repository exists locally.
- If found, it updates the repository using git pull --rebase. Otherwise, it clones the repository.

1. Fetch Repository List

- Using the GitHub API, the script fetches a list of repositories from the organization in pages of 100.


The script loops through all repositories and calls sync_repo for each.

Prerequisites

Before using the script, ensure the following:

• Git Installed: The script uses Git commands like git clone and git pull.
• GitHub API Access: The script fetches repositories using GitHub's REST API. If your organization has private repositories, configure authentication (e.g., with a Personal Access Token).
• jq Installed: This script uses jq to parse JSON responses from the API.

How to Use the Script

1. Save the script as sync_repos.sh

2. Make it executable
   

   chmod +x sync_repos.sh
   

3. Run the script
   

   ./sync_repos.sh
   

Key Takeaways

• Automating repetitive tasks like syncing repositories saves time and reduces errors.
• The GitHub API makes it easy to interact with your organization's repositories programmatically.
• With small tweaks, this script can handle additional tasks like branch switching or archiving inactive repositories.

Now, your GitHub repositories are always in sync with just one command! Happy automating!