Welcome back! Let's use Github Actions to deploy your shining new package to PyPI!
Basic setup
We don't have a git repository yet, so create one:
git init
Next, let's create a Github Repository
Once you're done with that, let's link our git repository to our Github repository:
git remote add origin "https://github.com/USERNAME/REPOSITORY.git"
Next, we'll commit and push our changes:
git add .
git commit -m "First Commit!"
git push origin master
You should see your code live at your Github Repository.
Setting up your first action
A Github Action is a YAML file. It can be edited in any code editor, but I recommend Github's own Editor, because it provides intellisense when you press Ctrl
+Space
.
To create an action, head over to the Actions
tab. I'll create an action to automatically test and publish our package to TestPyPI whenever we've pushed our code to the master branch.
We can choose an action template, or we can start from scratch. I'm going to choose the latter, so click the set up a workflow yourself
link.
This will create a new file that you can name whatever. I'll call it test-and-upload-to-testpypi.yml
. Make sure that it has the extension yaml
or yml
, otherwise Github won't run it.
This should be the starter code you get:
# This is a basic workflow to help you get started with Actions
name: CI
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the master branch
push:
branches: [ master ]
pull_request:
branches: [ master ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
# Runs a single command using the runners shell
- name: Run a one-line script
run: echo Hello, world!
# Runs a set of commands using the runners shell
- name: Run a multi-line script
run: |
echo Add other actions to build,
echo test, and deploy your project.
Anatomy of a Github Action
Let's go through this file step-by-step.
name
: It is the name of your Action. It will appear in the Actions tab under that name
on
: Controls what events trigger your action. Currently, the action runs when there's a pull request or a push to the master
branch, but there are other events too, like creation of releases, issues, adding collaborators, etc. The workflow-dispatch
event will add a button in the Actions tab that'll allow you to run this action.
jobs
: Defines the jobs in the action. Here, we have a single job called build
.
Inside the build
job:
runs-on
: The OS to run on. Supports ubuntu-latest
, windows-latest
and macos-latest
.
steps
: The steps in your job.
Inside steps
uses
: Another Github Action to use (this is optional)
name
: Name of the step. Will appear in the Actions tab.
run
: The command to run.
The pipeline (
|
) that you see after the run command in some steps is the YAML multi-line string operator. It allows a string to span mutliple lines, each newline indicating a new command.
Special actions
You can see that our action uses version v2
of actions/checkout
. This is an action made by someone else and we're using it. It checks-out our code and copies it to the machine, so we can use it. There are several other actions like these available in the Github MarketPlace
Creating our action
We will have 5 main steps in our action:
- Checkout our code
- Install python
- Install our dependencies
- Test using unittest
- Publish to testpypi
For the first two steps, we have actions that will do it for us, namely action/checkout@v2
and actions/setup-python@v2
.
I want the action to run when we've pushed our code to the master branch, and I also want to have a button in the Actions tab to manually test & upload the project for me.
Here's the final code for our action:
# This is a basic workflow to help you get started with Actions
name: Test & Upload to TestPyPI
# Controls when the action will run.
on:
# Triggers the workflow on push to the master branch
push:
branches: [ master ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
# Sets up python3
- uses: actions/setup-python@v2
with:
python-version: 3.8
# Installs and upgrades pip, installs other dependencies and installs the package from setup.py
- name: "Installs and upgrades pip, installs other dependencies and installs the package from setup.py"
run: |
# Upgrade pip
python3 -m pip install --upgrade pip
# Install build deps
python3 -m pip install setuptools wheel twine
# If requirements.txt exists, install from it
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
# Install the package from setup.py
python3 setup.py install
# Tests with unittest
- name: Test with unittest
run: |
cd tests
python3 -m unittest discover
cd ..
# Upload to TestPyPI
- name: Build and Upload to TestPyPI
run: |
python3 setup.py sdist bdist_wheel
python3 -m twine upload dist/*
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.TWINE_TEST_TOKEN }}
TWINE_REPOSITORY: testpypi
What are secrets?
Secrets are Github's way of passing environment variables to our action. In this case, we used the secret TWINE_TEST_TOKEN
to hold our API key.
Creating a secret
Go to the settings tab and select secrets. Click Create Secret on the top right and put TWINE_TEST_TOKEN
for the name and paste your API key for the value.
Now, save the file and watch your action work from the Actions tab!
In our actions tab, we can see that this action is running:
After fixing a small bug with our tests, you can see that our action has failed, indicated by the red X icon
This is because whenever we upload a package to PyPI (or TestPyPI), we cannot upload the package of the same version twice, so let's up the version in setup.py
And our action has successfully executed!
Uploading to PyPI
We did TestPyPI. What about the real python package index? I'm not going to upload the package everytime we push master, since there could be some changes not ready for production. Instead, I'm going to use the releases feature of Github.
Create a new action by clicking New workflow in the Actions tab and following the same steps as before. This time, I'm naming my action upload-to-pip.yml
Here's my action:
# This is a basic workflow to help you get started with Actions
name: Upload to PIP
# Controls when the action will run.
on:
# Triggers the workflow when a release is created
release:
types: [created]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "upload"
upload:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
# Sets up python
- uses: actions/setup-python@v2
with:
python-version: 3.8
# Install dependencies
- name: "Installs dependencies"
run: |
python3 -m pip install --upgrade pip
python3 -m pip install setuptools wheel twine
# Build and upload to PyPI
- name: "Builds and uploads to PyPI"
run: |
python3 setup.py sdist bdist_wheel
python3 -m twine upload dist/*
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.TWINE_TOKEN }}
Adding another secret
Let's add another secret, this time called TWINE_TOKEN
, and give it the value of your PyPI API token.
Testing our Action
Let's create a new release by going to the releases section (to do that, click New Release on the right side of the main code tab).
I suggest naming the tag with the version in
setup.py
prefixed withv
.
And our project has successfully been uploaded to PyPI!
Congratulations 🥳! You just learnt the basics of Github Actions, PyPI projects and DevOps! Give yourself a pat on your back!
Stuck?
Github repo
PyPI
Github Actions Quickstart
Uploading a package on PyPI
If you enjoyed this post, and would like to see more, let me know in the comments, and also, follow me for more content like this 😀
Top comments (2)
I would love to have read this six month ago, took me like a solid week to figure everything out. Great writeup!
Thank you!