Why?
One of the methods to install a package via poetry is to use deploy tokens
directly on the package URL, and that used to work on poetry too before version 1.0.3
.
What changed? The git URL parsing has changed significant to be more consistent.
Are they planning on fixing this within poetry? No.
They don't want to expose credentials on the pyproject.toml
file, which makes sense.
source: This issue on gitlab
How to do this then?
There are multiple ways, using git credential helpers, setting credentials globally for a particular URL.
The one thing that was easy enough to me was to tell git to replace the URL when calling gitlab via HTTPS.
After obtaining your deploy tokens from gitlab, this is how you tell git to use them
deploy_user=YOUR_DEPLOY_TOKEN_USER
deploy_token=YOU_DEPLOY_TOKEN
git config --global url."https://$deploy_user:$deploy_token@gitlab.com".insteadOf https://gitlab.com
Note this is intended to run in a docker container via CI, so it does not matter that much if you rewrite that URL, I do not recommend you to do this on your local machine, for that use SSH keys.
Which takes me to the next section
Poetry install Locally vs Gitlab CI
I do use SSH to clone mu libraries from gitlab, and the deploy token requires cloning via HTTPS, so now what?
I replace the urls on my before_script
, hacky but effective.
For example, let's say you have this library on your pyproject.toml
file
testrail-api-wrapper = {git = "git@gitlab.com:mundo03/sample_library.git", rev = "main"}
You can rewrite it using sed
like this:
export SSH_URL=git@gitlab\.com:mundo03\/sample_library.git
export TOKEN_URL=https:\\/\\/gitlab.com\\/mundo03\/sample_library.git
sed -i "s|$SSH_URL|$TOKEN_URL|g" pyproject.toml
your .gitlab-ci.yml
file would look like this, given you want to install you project and run pytest
stages:
- test
.build:
before_script:
- pip install poetry
- poetry config virtualenvs.in-project false #Ignore venv
- export SSH_URL=git@gitlab\.com:mundo03\/sample_library.git
- export TOKEN_URL=https:\\/\\/gitlab.com\\/mundo03\/sample_library.git
- sed -i "s|$SSH_URL|$TOKEN_URL|g" pyproject.toml
- git config --global url."https://$deploy_user:$deploy_token@gitlab.com".insteadOf https://gitlab.com
- poetry update
- poetry install -v
pytest_311:
image: python:3.11
extends: .build
stage: test
script:
- poetry run pytest --cov
retry: 2
coverage: '/TOTAL.*\s+(\d+%)$/'
Conclusion
Poetry did me dirty and I had to find another way to install dependencies in CI that come from private git repositories.
Top comments (0)