Last week I implemented a search functionality for a static website with Lunr.js. The way I implemented the search system, depends on 2 files to work, the pagesData.json
and the searchIndex.json
. These 2 files get generated by running an npm script (I called it yarn createSearchIndex
) which will parse all markdown files of the static website (pages and blog posts) and create the 2 aforementioned files. Obviously, I do not want to be responsible for running this script on my machine every time someone publishes a new blog post through our CMS (we use NetlifyCMS, so when someone wants to create a new blog post, they simply log in into the CMS, create the blog post, and the CMS will create a new commit on the branch NetlifyCMS connected to). What I want is the following:
- User commits a new blog post in branch
main
- A GitLab CI pipeline is triggered, which runs the
yarn createSearchIndex
script, which in turn generates the newpagesData.json
and thesearchIndex.json
files - GitLab CI commits the new
pagesData.json
andsearchIndex.json
files without triggering a new CI pipeline
Allowing GitLab runner to push to the git repo
We need to ensure that the GitLab runner machine can push new changes to our repository, so we need to have an SSH key and a git user.
Create SSH key pair
Create a public (deploy_key.pub
) and private (deploy_key
) SSH key pair without a passphrase with:
ssh-keygen -f deploy_key -N ""
Create environment variables
In the settings/ci_cd
page of your GitLab project, define 3 environment variables:
-
CI_GIT_USER_EMAIL
with the email of the git user that will appear in the commits made by the GitLab runner -
CI_GIT_USER_USERNAME
with the name of the git user that will appear in the commits made by the GitLab runner -
SSH_PRIVATE_KEY
with the contents of the private SSH key (you can copy the contents withpbcopy < deploy_key
)
Create deploy key
In the settings/repository
page of the GitLab project we need to create a deploy key. Create a new deploy key and paste the contents of the previously generated public key (pbcopy < deploy_key.pub
).
Make sure to enable Grant write permissions to this key, otherwise the runner won’t be able to push new commits.
Write .gitlab-ci.yml file
A basic version of my .gitlab-ci.yml
file looks like this:
image: node:14
stages:
- commit
update-index:
stage: commit
before_script:
# Set up the SSH key and the known_hosts file
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "${SSH_PRIVATE_KEY}" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- ssh-keyscan my.selfhostedgitlab.com >> ~/.ssh/known_hosts
script:
# Configure git
- git config --global user.email "${CI_GIT_USER_EMAIL}"
- git config --global user.name "${CI_GIT_USER_USERNAME}"
# Cone git repo
- git clone git@my.selfhostedgitlab.com:${CI_PROJECT_PATH}.git
- cd ${CI_PROJECT_NAME}
- git checkout main
# run scripts to generate new pagesData.json and searchIndex.json
- yarn && yarn createSearchIndex
# Check if there is something to commit, commit files, and push
- |
if [[-n $(git status --porcelain)]]; then
git add src/pagesData.json src/searchindex.json &&
git commit -m "Update search index" &&
git push origin main -o ci.skip
fi
only:
- main
Make sure to add the -o ci.skip
flag in the git push
command, so that you don’t trigger another pipeline.
The commit should then appear, with the name of the git user you have configured:
Resources
These resources helped me figure it out:
- https://forum.gitlab.com/t/git-push-from-inside-a-gitlab-runner/30554
- https://about.gitlab.com/blog/2017/11/02/automating-boring-git-operations-gitlab-ci/
- https://marcosschroh.github.io/posts/autobumping-with-gitlab/
- https://forum.gitlab.com/t/is-it-possible-to-commit-artifacts-into-the-git/22218/7
Photo by Pankaj Patel on Unsplash
Top comments (0)