DEV Community

loading...

Clone azure repos in parallel

João Vitor
Updated on ・2 min read

Azure repos are hosted inside devops projects.

You can list azure projects with

az devops project list --detect true
Enter fullscreen mode Exit fullscreen mode

To list repositories in azure from one specific az devops project

az repos list --project project_id
Enter fullscreen mode Exit fullscreen mode

To grab the repo list from multiple projects in parallel and save it to a file:

org=$(basename $(pwd)); az devops project list --detect true | jq -r '.value[]|.id' | parallel -j 10 "az repos list --project {} | jq -r '.[]|.sshUrl'" | tee ${org}.txt
Enter fullscreen mode Exit fullscreen mode

As the repo ssh url is different from the github one the clone-missing.sh script had to be changed in a not compatible way.

Difference

github ssh url: git@github.com:${githuborg}/${repo}.git
azure repo url: git@ssh.dev.azure.com:v3/${devopsorg}/${devopsproject}/${repo}

New azure-clone-missing.sh command added to the parallel_commands to handle this url difference.

Cloning azure repositories in parallel

Given that you created the org.txt file the command below clones only the missing repositories.

cd ~/azure-orgs/azuredevopsorg/
org=$(basename $(pwd)); cat ${org}.txt | parallel  -j 25 'azure-clone-missing.sh {}; echo job {#} completed {};'
Enter fullscreen mode Exit fullscreen mode

If you want to exclude some repositories from your clone.

PS change the regex that excludes the repo.
grep -Ev '(repo1|repo2)' ${org}.txt

cd ~/azure-orgs/azuredevopsorg/
org=$(basename $(pwd)); grep -Ev '(repo1|repo2)' ${org}.txt | parallel  -j 25 'azure-clone-missing.sh {}; echo job {#} completed {};'
Enter fullscreen mode Exit fullscreen mode

Documentation reference

Discussion (0)