DEV Community

loading...

How And Why, I Moved From Docker Hub To GitHub Docker Registry.

Mpho Mphego
Software Engineer in Cape Town, South Africa
Originally published at blog.mphomphego.co.za on ・7 min read

post image


The Story

On August 2020, Docker announced that they are introducing rate-limiting for Docker container pulls for free or anonymous users, which meant if you did not login to your DockerHub registry via command-line you would be limited to 100 pulls per 6 hours. At first, this did not affect me as I rarely pulled 10 images per day, but recently I have been tinkering with Kubernetes, Prometheus, Jaeger (you can check this post on how to install Prometheus & Grafana on K3s cluster) and other tools which usually pulls multiple images per run. You can check

This meant that I would be pulling images more frequently than I used to in the past. That's when I got the dreaded error message, "429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit".

This meant that I either had to configure Kubernetes secrets to pull from an authenticated DockerHub registry or find an alternative registry that will not issue rate limits every 100th pull. Coincidentally, roughly at the same time GitHub introduced their container registry (offering private container registry that integrates easily with the existing CI/CD tooling), and we were saved. Since it's still in its beta stage usage is free.

This post will detail how I migrated from using DockerHub to GitHub as my Docker container registry.

TL;DR

Setup GitHub Container registry and configure Kubernetes pod container to pull from the private registry (Optional)

But wait, What is a Container registry?

According to RedHat,

A container registry is a repository, or collection of repositories, used to store container images for Kubernetes, DevOps, and container-based application development.

GitHub is the home for free and open-source software and it's in a great spot to offer a container registry, which integrates well with their existing services and operates as an extension of GitHub Packages. Thus making it a good competitor to DockerHub.

The How

After deploying my application on my Kubernetes cluster. I noticed a few errors and after troubleshooting, I found that the docker pull rate limit was hit, which drove me insane.

$ kubectl describe pod frontend-app-6b885c795d-9vbfx | tail

Events:
  Type     Reason          Age                From                Message
  ----     ------          ----               ----                -------
  Warning  FailedMount     72s                kubelet, dashboard  MountVolume.SetUp failed for volume "default-token-6zmpp" : failed to sync secret cache: timed out waiting for the condition
  Normal   SandboxChanged  71s                kubelet, dashboard  Pod sandbox changed, it will be killed and re-created.
  Warning  Failed          44s                kubelet, dashboard  Failed to pull image "mmphego/frontend:v7": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/mmphego/frontend:v7": failed to copy: httpReaderSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/mmphego/frontend/manifests/sha256:2994ce56c38abe2947935d7bc9d6a743dfc30186659aae80d5f2b51a0b8f37d1: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
  Warning  Failed          44s                kubelet, dashboard  Error: ErrImagePull
  Normal   BackOff         43s                kubelet, dashboard  Back-off pulling image "mmphego/frontend:v7"
  Warning  Failed          43s                kubelet, dashboard  Error: ImagePullBackOff
  Normal   Pulling         31s (x2 over 70s)  kubelet, dashboard  Pulling image "mmphego/frontend:v7"
Enter fullscreen mode Exit fullscreen mode

image

The Walk-through

Setting up your container registry is straight forward.

Steps

  1. Create a GitHub Personal Token on https://github.com/settings/apps (See images below)

    • Select Personal Access Tokens

      image

    • Click Generate a new token

      image

    • Add a Note, check write: packages and hit generate.

      image

    When done, you will be provided with a token that you need to backup.

  2. GitHub recommends placing the token into a file.

    Either add the token into your ~/.bashrc or ~/.bash_profile and risk exposing them as environmental variables or place them in a file under a secret directory with reading/writing privileges (I prefer the latter).

    vim ~/.secrets/github_docker_token
    

    Paste the token into the github_docker_token file.

  3. Login to GitHub Container Registry

    Setup your username as an environmental variable:

    $ cat ~/.bashrc
    
    export GH_EMAIL=$(git config user.email)
    export GH_USERNAME=$(git config user.username) # or hardcode your username (Not Recommended!)
    

    Log in to your container registry with your username and personal token.

    cat ~/.secrets/github_docker_token | docker login ghcr.io -u ${GH_USERNAME} --password-stdin
    

    image

    If successful, you should see an image similar to the one above.

    Note: Typing secrets on the command line may store them in your shell history unprotected, and those secrets might also be visible to other users on your PC.

  4. Confirm that you successfully logged in.

    To confirm that you logged in we need to build, tag and push our image to ghcr (GitHub Container Registry).

    export USERNAME="add information"
    export REPOSITORY="add information"
    export IMAGE="add information"
    export VERSION="add information"
    docker build . -t ghcr.io/${USERNAME}/${REPOSITORY}/${IMAGE}:${VERSION}
    docker push ghcr.io/${USERNAME}/${REPOSITORY}/${IMAGE}:${VERSION}
    

    or in my case, I have a docker-compose yaml file to make my life easier (I suppose).

    cat docker-compose-file.yaml
    
```yaml
version: '3'
services:
hello_world_app:
    build: ../../app
    image: ghcr.io/mmphego/jaeger-tracing-example/jaeger-tracing-example:v2
```
Enter fullscreen mode Exit fullscreen mode
Then build the tagged image.
Enter fullscreen mode Exit fullscreen mode
```bash
docker-compose -f deployment/docker/docker-compose-file.yaml build
```
Enter fullscreen mode Exit fullscreen mode

Screenshot from 2021-04-15 16-27-31

continues...
Enter fullscreen mode Exit fullscreen mode

Screenshot from 2021-04-15 16-27-15

After a successful build, we push the image to the registry.
Enter fullscreen mode Exit fullscreen mode
```bash
docker-compose -f deployment/docker/docker-compose-file.yaml push
```
Enter fullscreen mode Exit fullscreen mode

Screenshot from 2021-04-15 16-27-46

**Note**: The manual build and push steps will be used on GitHub Actions (so at this point ensure everything works 100%)
Enter fullscreen mode Exit fullscreen mode
  1. After pushing the image you should see new package(s) on your profile under Packages.

    Screenshot from 2021-04-15 16-28-29

  2. Setup GitHub Action workflow for auto-build and publish (Optional)

    Optionally, set up a workflow environment in your repository/project for the CI/CD to build and publish your container to the GitHub container registry.

    mkdir -p .github/workflows
    vim .github/workflows/docker-image-publisher.yaml
    

    Paste the following code snippet into your docker-image-publisher.yaml. This workflow will build and push images on pull requests and master branches.

    ---
    name: Docker Image CI
    
    on:
    workflow_dispatch: # Run workflow manually (without waiting for the cron to be called), through the Github Actions Workflow page directly
    push:
        branches:
        - master
    pull_request:
        branches:
        - '*'
    
    jobs:
    build:
        runs-on: ubuntu-latest
        steps:
        - uses: actions/checkout@v2
            with:
            fetch-depth: 0
    
        - name: Build the Docker image
            run: |
            export USERNAME=${{ github.repository_owner }}
            docker-compose -f deployment/docker/docker-compose-file.yaml build
    
        - name: Login and Push Docker images to GitHub container registry
            run: |
            export USERNAME=${{ github.repository_owner }}
            echo "${{ secrets.DOCKER_PASSWORD }}" | docker login ghcr.io -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
            docker-compose -f deployment/docker/docker-compose-file.yaml push
    

    Run the GitHub Action build manually...

    image

Alternatively, check out the Docker Build & Push Action or Build and push Docker images

  1. Configure Kubernetes to use your new container registry (Optional)

    Kubernetes supports a special type of secret that you can create which will be used to fetch images for your pods from any container registry that requires authentication.

    Create a Kubernetes Secret, naming it my-secret-docker-reg and providing credentials:

    kubectl create secret docker-registry my-secret-docker-reg \
        --docker-server=https://ghcr.io \
        --docker-username=${GH_USERNAME} \
        --docker-password=$(cat  ~/.secrets/github_docker_token) \
        --docker-email=${GH_EMAIL} -o yaml > docker-secret.yaml
        # or  kubectl apply the output of an imperative command in one line
        # --docker-email=${GH_EMAIL} -o yaml | kubectl apply -f -
    
    # You can then apply the file like any other Kubernetes 'yaml':
    kubectl apply -f docker-secret.yaml
    

    Inspect the Secret: my-secret-docker-reg

    kubectl get secrets
    

    image

Final Result

Successful pull from (then private) GitHub container registry!

image

Me:
After setting up my GitHub container registry and Kubernetes docker-secrets.

Conclusion

Hopefully, you have learned something new in this post (and enjoyed Denzel Washington gifs) and will consider using GitHub Container Registry to house your images and GitHub Actions to build and push them to your GitHub Container Registry!
Finally, to configure your Kubernetes cluster to use GitHub Container Registry for fetching images for your pods.

Reference

Discussion (0)