Have you ever needed to change from an old computer to a new one and needed to manually clone all the Git repositories?
The below script keeps your folder structure and migrates all repositories to a new computer.
dirs=$(find . -name '.git' -type d | sed 's/\/\.git//')
for dir in $dirs; do
GIT_DIR=$dir/.git
# Creates the folder structure
echo mkdir -p $dir;
# Clones the repository in the same folder
echo git clone $(git --git-dir=$GIT_DIR remote get-url origin) $dir;
# Re-adds other remotes
for r in $(git --git-dir=$GIT_DIR remote | grep -v origin); do
echo git --git-dir=$GIT_DIR remote add $r $(git --git-dir=$GIT_DIR remote get-url $r);
done
echo
done > clone-all.sh
You just need to paste it into a shell session at the base folder where all your repositories are stored. The script creates a file named clone-all.sh. Next, you just need to copy and run this file in the new computer.
Script explanation
The script initially finds all sub-folders that contain a .git folder, which denotes it as a Git repository. From this list it starts to output commands that will be executed in the new computer.
The output commands (for each folder) are these:
- A command to create the folder, keeping the same structure;
- A command to clone the repository, to the same folder name, using the
originremote; - A command to add the other remotes (different than
origin) to the newly cloned repository.
Example
In the old computer, before run the script, the folder structure was this:
➜ pwd
/home/douglas/code
➜ tree -L 2
.
├── create-clone-script.sh
├── other
│ ├── audited_views
│ ├── bitmovin-ruby
│ ├── cucumber-ruby
│ ├── devise-i18n
│ ├── douglaslise.github.io
│ ├── gitlab-ci-monitor
│ ├── rubocop
│ ├── sendgrid-ruby
│ └── summernote
└── private
├── ping-monitor
└── qd
So I pasted the script in the shell and it generated this file clone-all.sh:
➜ cat clone-all.sh
mkdir -p ./other/bitmovin-ruby
git clone git@github.com:bitmovin/bitmovin-ruby.git ./other/bitmovin-ruby
git --git-dir=./other/bitmovin-ruby/.git remote add fork git@github.com:douglaslise/bitmovin-ruby.git
mkdir -p ./other/audited_views
git clone git@github.com:douglaslise/audited_views.git ./other/audited_views
mkdir -p ./other/cucumber-ruby
git clone git@github.com:cucumber/cucumber-ruby.git ./other/cucumber-ruby
mkdir -p ./other/rubocop
git clone git@github.com:rubocop-hq/rubocop ./other/rubocop
mkdir -p ./other/gitlab-ci-monitor
git clone git@github.com:globocom/gitlab-ci-monitor.git ./other/gitlab-ci-monitor
git --git-dir=./other/gitlab-ci-monitor/.git remote add fork git@github.com:douglaslise/gitlab-ci-monitor.git
mkdir -p ./other/summernote
git clone git@github.com:summernote/summernote.git ./other/summernote
mkdir -p ./other/devise-i18n
git clone git@github.com:tigrish/devise-i18n.git ./other/devise-i18n
git --git-dir=./other/devise-i18n/.git remote add fork git@github.com:douglaslise/devise-i18n.git
mkdir -p ./other/douglaslise.github.io
git clone git@github.com:douglaslise/douglaslise.github.io.git ./other/douglaslise.github.io
mkdir -p ./other/sendgrid-ruby
git clone git@github.com:sendgrid/sendgrid-ruby.git ./other/sendgrid-ruby
git --git-dir=./other/sendgrid-ruby/.git remote add fork git@github.com:douglaslise/sendgrid-ruby.git
mkdir -p ./private/ping-monitor
git clone ssh://hg@bitbucket.org/douglaslise/ping-monitor ./private/ping-monitor
git --git-dir=./private/ping-monitor/.git remote add heroku https://git.heroku.com/pingmonitor.git
mkdir -p ./private/qd
git clone ssh://hg@bitbucket.org/douglaslise/qd ./private/qd
➜
Next, I copied the generated file to the new computer and executed it in the base folder where I wanted to clone all the repositories:
➜ pwd
/home/douglas-new/code
➜ ls
clone-all.sh
➜ sh clone-all.sh
Cloning into './other/bitmovin-ruby'...
remote: Enumerating objects: 56, done.
remote: Counting objects: 100% (56/56), done.
remote: Compressing objects: 100% (43/43), done.
remote: Total 1707 (delta 21), reused 33 (delta 10), pack-reused 1651
Receiving objects: 100% (1707/1707), 245.62 KiB | 1.00 MiB/s, done.
Resolving deltas: 100% (1051/1051), done.
Cloning into './other/audited_views'...
remote: Enumerating objects: 314, done.
remote: Total 314 (delta 0), reused 0 (delta 0), pack-reused 314
Receiving objects: 100% (314/314), 48.93 KiB | 331.00 KiB/s, done.
Resolving deltas: 100% (81/81), done.
Cloning into './other/cucumber-ruby'...
remote: Enumerating objects: 371, done.
remote: Counting objects: 100% (371/371), done.
remote: Compressing objects: 100% (203/203), done.
remote: Total 58832 (delta 227), reused 264 (delta 163), pack-reused 58461
Receiving objects: 100% (58832/58832), 11.76 MiB | 4.73 MiB/s, done.
Resolving deltas: 100% (40075/40075), done.
Cloning into './other/rubocop'...
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 90142 (delta 0), reused 2 (delta 0), pack-reused 90131
Receiving objects: 100% (90142/90142), 29.81 MiB | 4.98 MiB/s, done.
Resolving deltas: 100% (68283/68283), done.
Cloning into './other/gitlab-ci-monitor'...
remote: Enumerating objects: 14, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 390 (delta 3), reused 3 (delta 0), pack-reused 376
Receiving objects: 100% (390/390), 471.38 KiB | 1.36 MiB/s, done.
Resolving deltas: 100% (210/210), done.
Cloning into './other/summernote'...
remote: Enumerating objects: 41, done.
remote: Counting objects: 100% (41/41), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 21101 (delta 21), reused 11 (delta 5), pack-reused 21060
Receiving objects: 100% (21101/21101), 13.13 MiB | 4.58 MiB/s, done.
Resolving deltas: 100% (13576/13576), done.
Cloning into './other/devise-i18n'...
remote: Enumerating objects: 114, done.
remote: Counting objects: 100% (114/114), done.
remote: Compressing objects: 100% (79/79), done.
remote: Total 4852 (delta 45), reused 56 (delta 19), pack-reused 4738
Receiving objects: 100% (4852/4852), 1.33 MiB | 2.81 MiB/s, done.
Resolving deltas: 100% (2928/2928), done.
Cloning into './other/douglaslise.github.io'...
remote: Enumerating objects: 463, done.
remote: Total 463 (delta 0), reused 0 (delta 0), pack-reused 463
Receiving objects: 100% (463/463), 4.46 MiB | 3.07 MiB/s, done.
Resolving deltas: 100% (92/92), done.
Cloning into './other/sendgrid-ruby'...
remote: Enumerating objects: 23, done.
remote: Counting objects: 100% (23/23), done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 2861 (delta 7), reused 14 (delta 4), pack-reused 2838
Receiving objects: 100% (2861/2861), 680.79 KiB | 1.60 MiB/s, done.
Resolving deltas: 100% (1313/1313), done.
Cloning into './private/ping-monitor'...
remote: Counting objects: 490, done.
remote: Compressing objects: 100% (436/436), done.
remote: Total 490 (delta 256), reused 75 (delta 25)
Receiving objects: 100% (490/490), 72.60 KiB | 352.00 KiB/s, done.
Resolving deltas: 100% (256/256), done.
Cloning into './private/qd'...
remote: Counting objects: 632, done.
remote: Compressing objects: 100% (418/418), done.
remote: Total 632 (delta 288), reused 332 (delta 128)
Receiving objects: 100% (632/632), 151.00 KiB | 322.00 KiB/s, done.
Resolving deltas: 100% (288/288), done.
Now we can see the newly created structure:
➜ tree -L 2
.
├── clone-all.sh
├── other
│ ├── audited_views
│ ├── bitmovin-ruby
│ ├── cucumber-ruby
│ ├── devise-i18n
│ ├── douglaslise.github.io
│ ├── gitlab-ci-monitor
│ ├── rubocop
│ ├── sendgrid-ruby
│ └── summernote
└── private
├── ping-monitor
└── qd
Conclusion
This could be used to clone all your projects just by copying a unique, small file, without the needing to copy all the files and folders.
If you have suggestions, please let me know in the comments.
Thanks.
Top comments (5)
Thanks for the script, I also wanted to just clone the projects, instead of copying files around.
I converted it into a Powershell script and used it on my windows machine successfully:
gist.github.com/Cyclodex/332802d13...
i usually zip the whole project folder and copy to external drive. this is very cool indeed
This is very good and useful; however, I found a work around.
On my new Mac, I restored my files via Time Machine backup and all my Git repositories were where they were on the old macbook. The problem was I could no longer PUSH to my Git repo as the id_rsa key was different. I simply followed the GIT instructions to generate new id_rsa key for the new macbook and everything worked fine. I could now simply continue where I left off on the old mac on my new mac.
docs.github.com/en/github/authenti...
Not a pro at this but it worked for me.
What was your reason for not just rsyncing the whole file structure over?
Good point. Rsync could be an option, but this script can help when you don't have a good connection with your old computer, or even when you do not know how to rsync them. Another use case is to backup the list of all your repositories. I do not backup them, but I regularly run this script and I store just the clone-all.sh file, so in the case I lost my old computer it will be easy to setup them in a new one.
But yes, rsync could be a great option too.