DEV Community

Cover image for How to clone a sub directory of a git project (not a zip)
darker
darker

Posted on

How to clone a sub directory of a git project (not a zip)

#git #bash #github #gitlab

Have you ever tried to get a folder from a git repository either on github, gitlab or bitbucket ?

Final Demo

hop hop hop, am not talking about Downloading a ZIP folder (like DownGit does), but only cloning a sub-directory with the git clone... command !

I got that question from a friend, and i wrote a small and interesting bash script to do so !

We're going to do that in 2 steps.

WRITE A BASH SCRIPT FUNCTION

We're going to write a function in our .bashrc that will update our sparse-checkout like this :

_git_clone_sub ()
{
    REPO_NAME="$(echo $2 | grep -oE '[^/]+$')";
    git clone --filter=blob:none --no-checkout $2
    cd $REPO_NAME;
    git sparse-checkout set --no-cone "$1/*"
    if [ -n "$3" ]; then
        git pull origin $3;
        git checkout $3;
    else
        git fetch origin;
        git checkout main
        [[ $? != 0 ]] && git checkout master;
    fi
}
Enter fullscreen mode Exit fullscreen mode

This script is quite simple, this is what it does:

  • Extract the repo name from the link (github.com/author/repo-name).
  • Create a git project with the repo name + add the origin from the link.
  • Then update the tree configuration of our project.
  • As an optional parameter, we have the branch, if it's provide, the script will clone a sub directory from that branch, otherwise, it will clone from master (you can change that with main if you want).

Now we can easily clone sub directories like this (after a source ~/.bashrc) :

_git_clone_sub subDir1 https://github.com/auth/repo
Enter fullscreen mode Exit fullscreen mode

or by specifying the branch name :

_git_clone_sub subDir1 https://github.com/auth/repo dev-branch
Enter fullscreen mode Exit fullscreen mode

WRITE A GIT ALIAS

So far so good... but we can do better !
We could call that bash function from git directly by adding an alias in our ~/.gitconfig file like this :

[alias]
    clone-sub = !bash -i -c '_git_clone_sub "$@"' -s
Enter fullscreen mode Exit fullscreen mode

Yeah yeah wait a minute, i know, that one looks weird, but let me explain... it's actually simple !

First, since we are going to run a bash function, we embrace that with 'bash -c'; then we call our function.
The $@ is to get all arguments got using the -s flag at the end of the alias.

That's been said, you can now do :

git clone-sub subDir1 https://github.com/auth/repo
Enter fullscreen mode Exit fullscreen mode 
git clone-sub subDir1 https://github.com/auth/repo dev-branch
Enter fullscreen mode Exit fullscreen mode

DEMO

DEMO

Thanks for reading, feel free to like and/or subscribe for more 🐼.

Oldest comments (7)

Collapse
 
ccoveille profile image
Christophe Colombier •

if I may

isn't this ?

    clone-sub = !bash -c 'source $HOME/.bashrc && _git_clone_sub "$@"' -s
Enter fullscreen mode Exit fullscreen mode

equal to this

    clone-sub = !bash -i -c '_git_clone_sub "$@"' -s
Enter fullscreen mode Exit fullscreen mode
Collapse
 
sanixdarker profile image
darker • • Edited

didn't knew the -i flag was equivalent to sourcing the bashrc, interesting !
Updated !

Collapse
 
ccoveille profile image
Christophe Colombier • • Edited 
git init $REPO_NAME;
cd $REPO_NAME;
git remote add origin $2;
git config core.sparsecheckout true;
echo "$1/*" >> .git/info/sparse-checkout;
Enter fullscreen mode Exit fullscreen mode

I would use the git command directly

git init $REPO_NAME;
cd $REPO_NAME;
git sparse-checkout set --no-cone '$1/*'
git remote add origin $2;
Enter fullscreen mode Exit fullscreen mode
Collapse
 
ccoveille profile image
Christophe Colombier • • Edited

interesting, you made me discover sparse checkout

I found this article, you may have read aymericbeaumet.com/faster-git-clon...
and the git documentation git-scm.com/docs/git-sparse-checkout

Here is the code I suggest you to try

_git_clone_sub ()
{
    REPO_NAME="$(echo $2 | grep -oE '[^/]+$')";
    git clone --filter=blob:none --no-checkout $2
    cd $REPO_NAME;
    git sparse-checkout set --no-cone "$1/*"
    if [ -n "$3" ]; then
        git pull origin $3;
        git checkout $3;
    else
        git fetch origin;
        git checkout main
        [[ $? != 0 ]] && git checkout master;
    fi
}
Enter fullscreen mode Exit fullscreen mode

I tested it locally by fetching a (private) 500Mo repository, and it's way faster this way (few seconds compared to minutes). Tell me if it suits your needs.

Collapse
 
sanixdarker profile image
darker •

Thanks for the article and the hint !
I just updated !

Collapse
 
ccoveille profile image
Christophe Colombier • • Edited

one more tips or suggestion

You are defining your alias with this

[alias]
    clone-sub = !bash -i -c '_git_clone_sub "$@"' -s
Enter fullscreen mode Exit fullscreen mode

because you defined the bash function like this

_git_clone_sub ()
{
 # ...
}
Enter fullscreen mode Exit fullscreen mode

I wouldn't have done like this.

First reason, your script is interesting, but bash is not used by everyone as a shell, so someone who might be interested by your script might be disappointed it's only available in bash

I would have defined a script available in your $PATH

such as:

  • bin
  • .local/bin

it depends on your distribution and can differ; simply do this to find the list of folder you have in your PATH variable

echo $PATH
Enter fullscreen mode Exit fullscreen mode

so let's say, ~/bin/ is in your PATH, I would have created ~/bin/git-clone-sub file, please make sure to make it executable with chmod +x ~/bin/git-clone-sub

I will explain the name I chose later.

Here would be content

#!/bin/bash

REPO_NAME="$(echo $2 | grep -oE '[^/]+$')";
git clone --filter=blob:none --no-checkout $2 $REPO_NAME
cd $REPO_NAME;
git sparse-checkout set --no-cone "$1/*"
if [ -n "$3" ]; then
    git pull origin $3;
    git checkout $3;
else
    git fetch origin;
    git checkout main
    [[ $? != 0 ]] && git checkout master;
fi
Enter fullscreen mode Exit fullscreen mode

Please note that here, I don't have to use -i for bash or -s because we are not using your alias you defined in .bashrc but a real binary.

Then what would I defined in my git config file for the alias, simply nothing.

Why ? because git is awesome, and accept to be extended easily. any executable binary starting with git- will be accepted and completed.

so now, you can simply type

git clone-sub <folder> <any repository>

and it will work, also git will complete the name of the function, it won't complete the argument of your function of course, as here it's not git that does it but your shell, so here bash.

This way, any one using zsh or fish would be able to use your script.

Collapse
 
yongchanghe profile image
Yongchang He •

Thank you for sharing this!

Read next

jmfayard profile image

Bash is a terrible as a programming language, but what's the alternative ?

Jean-Michel 🕵🏻‍♂️ Fayard -

mochafreddo profile image

How to Manage Documentation in a GitHub Repository: A Guide for Junior Developers

Geoffrey Kim -

reinhart1010 profile image

The (>_ )s need your help!

Reinhart Previano K. -

pizofreude profile image

Tracking Crypto Prices on Autopilot: GitHub Codespaces & Actions Guide

Pizofreude -